Workshops and tutorials were originally scheduled on July 17, 2020

However, due to the new virtual format, and the consequent rescheduling, they might be held on a different day. Please check the new schedule.

Accepted Workshops

Organizers: Liliana Ardissono, Noemi Mauro, George E. Raptis and Alan Wecker

11th International Workshop on Personalized Access to Cultural Heritage (PATCH 2020)

Following the successful series of PATCH workshops, PATCH 2020 will be again the meeting point between state-of-the-art cultural heritage research and personalization – using any kind of technology, while focusing on ubiquitous and adaptive scenarios, to enhance the personal experience in cultural heritage sites. The workshop is aimed at bringing together researchers and practitioners who are working on various aspects of cultural heritage and are interested in exploring the potential of state of the art of mobile technology (onsite as well as online) to enhance the CH visit experience. The expected result of the workshop is a multidisciplinary research agenda that will inform future research directions and hopefully, forge some research collaborations.

Organizers: Panagiotis Germanakos, Vania Dimitrova, Ben Steichen and Alicja Piotrkowicz

Human Aspects in Adaptive and Personalized Interactive Environments (HAAPIE 2020)

The vision of HAAPIE 2020 workshop is to bring more inclusively the “human-in-the-loop” in UMAP for increasing the usability, user experience and overall quality of systems and interactions. State-of-the-art approaches in adaptation and personalization research that consider information regarding the “traditional” user characteristics (i.e., experience, knowledge, interests, context), and related contextual or technology aspects (i.e., displays, connectivity, processing power) have shown significant improvements and benefits to the end-users. However, there is an urgent need for a step change in user modeling and adaptation that considers human aspects thoroughly, producing more holistic human-centered adaptation and personalization theories and practices. This requires broadening the scope including intrinsic human characteristics and abilities, such as perceptual, personality, visual, cognitive, and emotional factors as well as other diversity parameters ranging from more recognizable user characteristics, such as age, culture, status, to more inherent ones, such as motivation, self-actualization, and socio-cultural behavior. Accordingly, main goal of HAAPIE 2020 is to bring together researchers and practitioners working in the areas of human aspects in adaptation and personalization to shape new human-centered adaptive interactive environments and personalized platforms that can contribute towards viable long-term solutions.


Styliani Kleanthous, Bamshad Mobasher, Bettina Berendt, Michael Ekstrand, Jahna Otterbacher and Avital Shulner-Tal

3rd UMAP Workshop on Fairness in User Modeling, Adaptation, and Personalization (FairUMAP 2020)

Machine learning, recommender systems, and user modeling are key enabling technologies used in personalized intelligent systems. However, there has been a growing recognition that these underlying technologies raise novel ethical, policy, and legal challenges. System properties such as fairness, transparency, balance, openness to diversity, and other social welfare considerations are not always captured by typical metrics based on which data-driven personalized models are optimized. Bias, fairness, and transparency in machine learning are topics of considerable recent research interest. However, more work is needed to expand and extend this work into algorithmic and modeling approaches where user modeling and personalization is of primary importance. In particular, it is essential to address these challenges from the standpoint understanding stereotypes in users’ behavior and their influence on user or group decisions. The workshop aims to bring together a growing community of experts from academia and industry to discuss ethical, social, and legal concerns related to personalization and user modeling with the goal of exploring a variety of mechanisms and modeling approaches that help mitigate bias and achieve fairness in personalized systems.


Cataldo Musto, Nava Tintarev, Oana Inel, Marco Polignano, Giovanni Semeraro and Jurgen Ziegler

Explainable User Models and Personalized Systems (ExUM)

ExUM workshop aims to provide a forum to discuss and investigate the role of transparency and explainability in the development of novel methodologies to build user models and personalized systems. Research lines of interest for ExUM include: building scrutable user models and transparent algorithms, analyzing the impact of opaque algorithms on final users, studying the role of explanation strategies, investigating how to provide users with more control in personalized and adaptive systems.


Marios Belk, Christos Fidas, Juliana Bowles, Elias Athanasopoulos and Andreas Pitsillides

2nd International Workshop on Adaptive and Personalized Privacy and Security (APPS 2020)

Recent privacy and security incidents of famous online services have once more underpinned the necessity towards further investigating and improving current approaches and practices related to the design of efficient and effective privacy and security. In order to achieve this objective, one possible direction is related to providing adaptive and personalized characteristics to privacy- and security-related user tasks, given the diversity of the user characteristics (like cultural, cognitive, age, habits), the technology (like standalone, mobile, mixed-virtual-augmented reality, wearables) and interaction contexts of use (like being on the move, social settings, spatial limitations). Hence, adaptive and personalized privacy and security implies the ability of an interactive system or service to support its end-users, who are engaged in privacy- and/or security-related tasks, based on user models which describe in a holistic way what constitutes the user’s physical, technological and interaction context in which computation takes place. APPS 2020 aims to bring together researchers and practitioners working on diverse topics related to understanding and improving the usability of privacy and security software and systems, by applying user modeling, adaptation and personalization principles. Our special focus in 2020 will be on healthcare systems, more specifically on ensuring security and privacy of medical data in smart patient-centric healthcare systems.


Giorgio Delzanno, Giovanna Guerrini and Daniele Traversaro

Adaptation and Personalization in Computer Science Education (APCSE 2020)

In recent years a wide range of tools and applications have been developed for supporting Computer Science Education 
ranging from physical devices to visual programming languages and web applications. 
Individual and cooperative learning processes must be supported in a complex scenario in which learners have different education levels, motivations, and expectations. In this setting technological solutions play a central role in modeling user needs and in providing personalized support to improve the effectiveness and satisfaction of learning experiences. 
The workshop aims at bringing together researchers, practitioners and education stakeholders  to discuss challenges and future directions on adaptation and personalization in Computer Science Education.

Accepted Tutorials


Ludovico Boratto and Mirko Marras

Hands on Data and Algorithmic Bias in Recommender Systems


This tutorial provides a common ground for both researchers and practitioners interested in data and algorithmic bias in recommender systems. Guided by real-world examples in various domains, we introduce problem space and concepts underlying bias investigation in recommendation, and show two of the most frequently investigated use cases, addressing popularity bias and fairness. Then, we cover a range of techniques for evaluating and mitigating the impact of these biases in recommended lists, including pre-, in-, and post-processing procedures. This tutorial is accompanied by Jupyter notebooks putting into practice core concepts in data from real-world online platforms. Ludovico Boratto is senior research scientist in the Data Science and Big Data Analytics research group at Eurecat, in Barcelona (Spain). His research interests focus on Data Mining and Machine Learning approaches, mostly applied to recommender systems and social media analysis. The results of his research have been published in top-tier conferences and journals. His research activity also brought him to give talks and tutorials at top-tier conferences (e.g., ACM RecSys 2016, IEEE ICDM 2017) and research centers (Yahoo! Research). He is editor of the book “Group Recommender Systems: An Introduction”, published by Springer. He is editorial board member of the “Information Processing & Management” journal (Elsevier) and guest editor of several journal’s special issues. He is regularly part of the program committee of the main Data Mining and Web conferences, such as RecSys, KDD, SIGIR, WSDM, ICWSM, and TheWebConf. In 2012, he got a Ph.D. at the University of Cagliari (Italy), where he was research assistant until May 2016. In 2010 and 2014 he spent 10 months at Yahoo! Research in Barcelona as a visiting researcher. He is member of the ACM and of the IEEE. Mirko Marras is a PhD Candidate in Computer Science at Department of Mathematics and Computer Science of the University of Cagliari (Italy). His research interests focus on machine learning approaches for educational platforms, mostly applied on semantic-aware systems, recommender systems, biometric systems, and opinion mining systems. He has authored papers in top-tier international journals, such as Pattern Recognition Letters (Elsevier), Computers in Human Behavior (Elsevier), and IEEE Cloud Computing. He has given talks and demos at several international conferences and workshops, such as TheWebConf 2018, ECIR 2019, and INTERSPEECH 2019. He has been involving in the program committee of international conferences, such as ACL, AIED, EDM, ITICSE, ICALT, and UMAP. He has been co-chairing the Bias 2020 workshop on algorithmic bias in search and recommendation at ECIR 2020. In 2016, he received the MSc Degree in Computer Science (summa cum laude) from University of Cagliari. As a visiting student, he spent six months at EURECAT in Barcelona and two months at New York University. He is member of several national and international associations, including GRIN, CVPL, AIxIA, IEEE, and ACM.


Stephan Weibelzahl, Alexandros Paramythis and Judith Masthoff

Evaluation of Adaptive Systems


Adaptive systems are by definition interactive systems. As such they stand to benefit considerably from a development lifecycle that ensures user involvement from the early design stages, and embraces evaluation, in both formative and summative forms. Evaluating an adaptive system involves a number of specific problems and pitfalls that need to be addressed by the selection of specific methods, techniques and criteria. This tutorial aims to raise the awareness of participants regarding adequate evaluation methods and techniques, and reliable assessment criteria and metrics. It will address the following learning objectives:
  • Awareness of the specific problems involved in the evaluation of adaptive systems that differentiate them from their non-adaptive counterparts, and of ways to solve or circumvent these problems
  • Understanding of, and capacity to apply, the principles of layered evaluation of adaptive systems
  • Acquisition of the prerequisite knowledge for designing a targeted evaluation study for an adaptive system (e.g., addressing a given layer and set of criteria) by selecting appropriate methods and criteria).
Professor Dr. Stephan Weibelzahl is Professor of Business Psychology at PFH Private University of Applied Sciences Göttingen, Germany. Previously he held positions at National College of Ireland and Fraunhofer IESE, Germany. Stephan has longstanding experience in the evaluation of adaptive systems. In his PhD thesis he tried to integrate current research on this topic and explored suitable evaluation methods and criteria. He has published on the problems arising in the area and adequate methods to address these problems. Dr. Alexandros Paramythis has long-standing experience in the design and development of adaptive system frameworks, gained through participation in several international research projects in the field. His involvement in evaluation activities in this context has focused on an attempt to bridge the gap between the design of adaptive systems and the derivation of valid and reliable results concerning the effects of adaptation on interaction. Professor Dr. Ir. Judith Masthoff has performed research on a range of adaptive systems (educational, medical, recommender systems) which has always been accompanied by extensive evaluations. She has also published on the difficulties encountered in evaluating adaptive systems and on how to overcome some of these. She is Editor in Chief of the User Modeling and User Adapted Interaction journal, which is the leading personalization journal. She is also a director of User Modeling Inc, the association for user modelling and personalisation professionals.


Jim Torresen

Ethical Considerations in User Modeling and Personalization


Ethical considerations are getting increased attention with regards to providing responsible personalization for robots and autonomous systems. This is partly as a result of the currently limited deployment of such systems in human support and interaction settings. The tutorial will give an overview of the most commonly expressed ethical challenges and ways being undertaken to reduce their impact using the findings in an earlier undertaken review supplemented with recent work and initiatives. The tutorial will exemplify the challenges related to privacy, security and safety through several examples from own and others’ work.

Jim Torresen is a professor at University of Oslo where he leads the Robotics and Intelligent Systems (ROBIN) research group. He received his M.Sc. and (Ph.D) degrees in computer architecture and design from the Norwegian University of Science and Technology, University of Trondheim in 1991 and 1996, respectively. He has been employed as a senior hardware designer at NERA Telecommunications (1996-1998) and at Navia Aviation (1998-1999). Since 1999, he has been a professor at the Department of Informatics at the University of Oslo (associate professor 1999-2005). Jim Torresen has been a visiting researcher at Kyoto University, Japan for one year (1993-1994), four months at Electrotechnical laboratory, Tsukuba, Japan (1997 and 2000) and a visiting professor at Cornell University, USA for one year (2010-2011). His research interests at the moment include artificial intelligence, ethical aspects of AI and robotics, machine learning, robotics, and applying this to complex real-world applications. Several novel methods have been proposed. He has published over 200 scientific papers in international journals, books and conference proceedings. 10 tutorials and a number of invited talks have been given at international conferences and research institutes. He is in the program committee of more than ten different international conferences, associate editor of three international scientific journals as well as a regular reviewer of a number of other international journals. He has also acted as an evaluator for proposals in EU FP7 and Horizon2020 and is currently project manager/principal investigator in four externally funded research projects/centres. That includes being a principal investigator at the Centre of Excellence for Interdisciplinary Studies in Rhythm, Time and Motion (RITMO). He is a member of the Norwegian Academy of Technological Sciences (NTVA) and the National Committee for Research Ethics in Science and Technology (NENT) where he is a member of a working group on research ethics for AI. More information and a list of publications can be found here:


Flavian Vasile, David Rohde, Olivier Jeunen, Amine Benhalloum

A Gentle Introduction to Recommendation as Counterfactual Policy Learning


The objective of the course is to give a structured overview of the conceptual frameworks behind the current state-of-the-art recommender systems, explain their underlying assumptions, the resulting methods and their shortcomings and to introduce an exciting new class of approaches, that frames the task of recommendation as a counterfactual policy learning problem. This is a full-day tutorial and it is divided in two modules:In module 1, the participants will learn about the current approaches for building real-world recommender systems, that comprise mainly of two frameworks, namely: recommendation as optimal auto-completion of user behaviour and recommendation as reward modelling. In module 2, we present the framework of recommendation as a counterfactual policy learning problem and go over the theoretical guarantees that address the shortcomings of the previous frameworks. We then proceed to go over the associated algorithms and test them against classical methods in RecoGym, an open-source recommendation simulation environment. Overall, we believe the subject of the course is extremely actual and fills a gap between the consecrated recommendation frame- works and the cutting edge research and sets the stage for future advances in the field. Flavian Vasile is part of the Criteo AI Lab where he works as the ML Recommendations Solutions Architect, with his main focus being on the development of Deep Learning-based Recommendation Systems and on introducing aspects of Causal Inference to Recommendation.Before joining Criteo, he worked as a researcher in the Twitter Advertising Science team; before that, in the Yahoo! Research Lab where he mostly focused on Content Understanding problems. His current research interests include Deep Sequential Models for Recommendation and understanding Recommendation as a decision-making system with reward uncertainty. Among his recent research publications, the work on “Causal Embeddings for Recommendation” received the best paper award at RecSys 2018 and he was the co-organizer of the REVEAL Workshops on Offline Evaluation for Recommender Systems in conjunction with the ACM RecSys conference. David Rohde is a research scientist at Criteo. His research interests are around Bayesian machine learning, offline evaluation and causal inference. He is one of the original creators of the RecoGym environment and one of the authors of the RecoGym course which he delivered at the DS3 summer school in Paris, 2019. He regularly presents at machine learning conferences such as the REVEAL workshop and the causality workshops at NeurIPS 2018 and 2019 Furthermore, he regularly delivers teaching material internally at Criteo and externally at forums such as the DS3 summer school. He has numerous publications in applied and theoretical aspects of machine learning, from topics including variational approximations, causal inference, doubly intractable models to astronomy, analyzing massive public transport datasets and evaluating recommender systems. Olivier Jeunen is a PhD Student at the University of Antwerp, Belgium. His main line of research focuses on implicit-feedback recommender systems, with interests ranging from algorithms and efficiency to evaluation. He regularly collaborates with industrial research labs such as Technicolor, Froomle and Criteo. During this latter collaboration, he contributed to the RecoGym environment and co-authored and co-presented the original RecoGym course multiple times. Additionally, he has several years of experience teaching MSc-level Data Science courses at the University of Antwerp. Amine Benhalloum is a Senior Machine Learning Engineer at Criteo, working on building large scale representation learning and retrieval systems for recommendation, applying Deep learning to personalize billions of daily display ads, reaching billions of users and connecting them with millions of products. His areas of expertise are: large scale machine learning, natural language processing, information retrieval and data intensive systems. Before joining Criteo, Amine worked on a variety of topics ranging from Natural Language processing to fraud detection. He holds a master’s degree in Applied Mathematics.