Tutorials
Conducting User Experiments with Personalized Systems
Presenter: Bart Knijnenburg
Description: Studying user modeling, adaptation and personalization (UMAP) mechanisms inherently requires evaluating the effect of these mechanisms on end-users. The proper evaluation of the effects of UMAP systems on their users requires conducting user experiments, which can take the shape of controlled lab experiments, randomized field trials, or larger-scale A/B tests. This tutorial provides practical training in designing and conducting user experiments, and in statistically analyzing the results of such experiments. This tutorial will be useful for anyone seeking to conduct user-centric evaluations of UMAP systems—ranging from adaptive user interfaces to human-like conversational artificial intelligence (AI) systems powered by large language models (LLMs). It covers the development of a research question and hypotheses, the selection of study participants, the manipulation of system aspects and measurement of behaviors, perceptions and user experiences, and the evaluation of subjective measurement scales and study hypotheses.
Conducting Recommender Systems User Studies Using POPROX
Presenters: Robin Burke and Michael Ekstrand
Description: The Platform for OPen Recommendation and Online eXperimentation (POPROX) is a new resource to allow recommender systems and personalization researchers to conduct online user research without having to develop all of the necessary infrastructure and recruit users themselves. Our first domain is personalized news recommendations: POPROX 1.0 provides a daily newsletter (with content from the Associated Press) to users who have already consented to participate in research, along with interfaces and protocols to support researchers in conducting studies that assign subsets of users to various experimental algorithms and/or interfaces. This tutorial will introduce the platform and its capabilities to researchers in the UMAP community who may be interested using the system, and provide instructions on how to plan and run an experiment on POPROX. Participants will walk through the implementation of a sample experiment to demonstrate the mechanics of designing and running user studies with POPROX.
Human-Centered and Sustainable Recommender Systems
Presenters: Allegra De Filippo, Ludovico Boratto and Giuseppe Spillo
Description: This tutorial explores the intersection of sustainability and recommender systems, focusing on aligning user needs and values with sustainable practices. It emphasizes two dimensions: (1) understanding and modeling users to deliver more sustainable recommendations; and (2) fostering sustainability through system design and functionality. Participants can learn how recommender systems can encourage sustainable behaviors and how to enhance system efficiency while minimizing resource consumption and ethical challenges. Through theoretical insights and hands-on sessions, this tutorial proposes discussion and actionable strategies to design human-centered, sustainable recommender systems, addressing both societal impact and technological responsibility.
Data Access under the EU Digital Services Act and its Impact on User Modelling Research
Presenters: Erasmo Purificato, Ludovico Boratto and Joao Vinagre
Description: The Digital Services Act (DSA) establishes a regulatory framework for online platforms and search engines in the European Union, focusing on mitigating systemic risks such as illegal content dissemination, fundamental rights violations, and impacts on electoral processes, public health, and gender-based violence. Very Large Online Platforms (VLOPs) and Very Large Search Engines (VLOSEs), defined as those with over 45 million active recipients, must provide data access for research to enable investigations into these risks and the development of solutions. This tutorial is tailored for the UMAP community, addressing the implications of the DSA for user modelling research. It will cover the DSA’s key provisions and definitions, outline the procedural steps for accessing VLOP and VLOSE data, and discuss the technical aspects of data access requests. Participants will also explore the challenges and opportunities involved in working with this data. By the end of the tutorial, attendees will have a thorough understanding of the DSA’s data access provisions, the technical and procedural requirements for accessing VLOP and VLOSE data, and the regulation’s implications for user modelling research. They will be equipped to navigate the complexities of the DSA and contribute to the development of responsible and transparent online platforms.
Designing Intelligent User Interfaces for Well-Being
Presenters: Ernesto William De Luca, Julian Marvin Joers, and Marko Tkalčič
Description: In this tutorial, three separate areas of research will be combined to provide a comprehensive tutorial for considering well-being orientations in user interface design. First, the development of intelligent user interfaces will be highlighted from an artificial intelligence (AI) engineering perspective, using modality-based reasoning and algorithmic evaluation to build well-being-centred systems. Secondly, this perspective will be extended by the human-centred artificial intelligence (HCAI) approach, and finally, how the actual interaction can be developed from a human-centred design perspective in terms of well-being orientations that manifest specific interaction principles.