29th Conference on User Modeling, Adaptation and Personalization

Utrecht, the Netherlands & Online

June 21-25, 2021




universiteit utrecht

radboud universiteit


Multi-method evaluation: Why and how?

by Christine Bauer (University of Utrecht, the Netherlands)

When evaluating personalized or adaptive systems, we frequently rely on one single evaluation objective and one single method. This remains us with “blind spots”. A comprehensive evaluation may require a thoughtful integration of multiple methods.
This tutorial will (i) demonstrate the wide variety of dimensions to be evaluated, (ii) outline the methodological approaches to evaluate these dimensions, (iii) pinpoint the blind spots when using only one approach, (iv) demonstrate the benefits of multi-method evaluation, (v) and outline the basic options how multiple methods can be integrated into one evaluation design.
Participants will familiarize with the wide spectrum of opportunities how adaptive or personalized systems may be evaluated, and will be able to come up with evaluation designs that comply with the four basic options of multi-method evaluation.
The ultimate learning objective is to stimulate the critical reflection of one’s own evaluation practices and those of the community at large.


christine bauer

Christine Bauer is an assistant professor at Utrecht University, The Netherlands. Her research activities center on interactive intelligent systems. Recently, she focuses on context-aware (music) recommender systems. Core interests in her research are fairness in algorithmic decision-making and multi-method evaluations. Her research and teaching activities are driven by her interdisciplinary background. She holds a Doctoral degree in Social and Economic Sciences, a Master degree in Business Informatics, and a Diploma degree in International Business Administration. In addition, she pursued studies in jazz saxophone. She has authored more than 95 papers and holds several best paper awards as well as awards for her reviewing activities. Furthermore, she is an experienced teacher in a wide spectrum of topics, ranging from algorithms to adaptive interactive systems to research methods.


Ethical Considerations in User Modeling and Personalization

by Jim Tørresen (University of Oslo, Norway)

Ethical considerations are getting increased attention with regards to providing responsible personalization for robots and autonomous systems. This is partly as a result of the currently limited deployment of such systems in human support and interaction settings. The tutorial will give an overview of the most commonly expressed ethical challenges and ways being undertaken to reduce their impact using the findings in an earlier undertaken review supplemented with recent work and initiatives. The tutorial will exemplify the challenges related to privacy, security and safety through several examples from own and others’ work.


JimTorresenJim Tørresen is a professor at University of Oslo where he leads the Robotics and Intelligent Systems (ROBIN) research group. He received his M.Sc. and Dr.ing. (Ph.D) degrees in computer architecture and design from the Norwegian University of Science and Technology, University of Trondheim in 1991 and 1996, respectively. He has been employed as a senior hardware designer at NERA Telecommunications (1996-1998) and at Navia Aviation (1998-1999). Since 1999, he has been a professor at the Department of Informatics at the University of Oslo (associate professor 1999-2005). Jim Torresen has been a visiting researcher at Kyoto University, Japan for one year (1993-1994), four months at Electrotechnical laboratory, Tsukuba, Japan (1997 and 2000) and a visiting professor at Cornell University, USA for one year (2010-2011).

His research interests at the moment include artificial intelligence, ethical aspects of AI and robotics, machine learning, robotics, and applying this to complex real-world applications. Several novel methods have been proposed. He has published over 200 scientific papers in international journals, books and conference proceedings. 10 tutorials and a number of invited talks have been given at international conferences and research institutes. He is in the program committee of more than ten different international conferences, associate editor of three international scientific journals as well as a regular reviewer of a number of other international journals. He has also acted as an evaluator for proposals in EU FP7 and Horizon2020 and is currently project manager/principal investigator in four externally funded research projects/centres. That includes being a principal investigator at the Centre of Excellence for Interdisciplinary Studies in Rhythm, Time and Motion (RITMO). He is a member of the Norwegian Academy of Technological Sciences (NTVA) and the National Committee for Research Ethics in Science and Technology (NENT) where he is a member of a working group on research ethics for AI. More information and a list of publications can be found here: http://jimtoer.no