Contextual Integrity: Theory and Some Applications

  • Helen Nissenbaum

Creatures of Habit in Next Basket Recommendation

  • Maarten de Rijke

Personalized Persuasive Technologies for Engagement and Behaviour Change

  • Julita Vassileva

Full Papers

Uncovering Personal and Context-Dependent Display Preferences in Mobile Newsreader App

  • Emir Hasanbegović
  • Veljko Pejović

The smartphone has revolutionised the way we receive news, enabling on-demand, personalised content to be viewed in a range of different situations. Yet, while the content of the news is often adapted to the user’s preferences and the current environment (e.g. location), the actual interface of a mobile newsreader app often remains the same across users and contexts of use. In this work we first collect and examine real-world mobile news reading data to uncover the way contextual factors affect the perception of different aspects of the newsreader app interface, and then develop a method for modelling personalised context-dependent viewing preferences. Through a four-week long user study we demonstrate that our reinforcement and active learning-based personalisation approach leads to 26% higher user acceptance as compared to a generic context-aware mobile newsreader interface adaptation model.

Factors Influencing Privacy Concern for Explanations of Group Recommendation

  • Shabnam Najafian
  • Amra Delic
  • Marko Tkalcic
  • Nava Tintarev

Explanations can help users to better understand why items have been recommended. Additionally, explanations for group recommender systems need to consider further goals than single-user recommender systems. For example, we need to balance group members’ need for privacy with their need for transparency, since a transparent explanation might pose a privacy hazard. In an online experiment with real groups (n=114 participants: 38 groups of size 3), we seek to understand which factors influence people’s privacy concerns when a single explanation is presented to a group in the tourism domain. In particular, we study the direct effects of three factors on privacy concern: a) group members’ personality (using the ‘Big Five’ personality traits), b) specific preference scenarios (i.e., having minority or majority preferences compared to two other group members), c) the type of relationship they have in the group (i.e., loosely coupled heterogeneous, versus tightly coupled homogeneous). We find that for personality two traits, Extroversion, and Agreeableness, each significantly affects the privacy concern. Moreover, having the minority or majority preferences in the group, as well as the type of relationship people have in the group, have a strong and significant influence on participants’ privacy concern. These results suggest that explanations presented to groups need to be adapted to all three factors (personality, type of relationship, and preference scenario) when considering the privacy concern of users.

Learning to Trust: Understanding Editorial Authority and Trust in Recommender Systems for Education

  • Taha Hassan
  • Bob Edmison
  • Timothy Stelter
  • D. Scott McCrickard

Trust in a recommendation system (RS) is often algorithmically incorporated using implicit or explicit feedback of user-perceived trustworthy social neighbors, and evaluated using user-reported trustworthiness of recommended items. However, real-life recommendation settings can feature group disparities in trust, power, and prerogatives. Our study examines a complementary view of trust which relies on the editorial power relationships and attitudes of all stakeholders in the RS application domain. We devise a simple, first-principles metric of editorial authority, i.e., user preferences for recommendation sourcing, veto power, and incorporating user feedback, such that one RS user group confers trust upon another by ceding or assigning editorial authority. In a mixed-methods study at Virginia Tech, we surveyed faculty, teaching assistants, and students about their preferences of editorial authority, and hypothesis-tested its relationship with trust in algorithms for a hypothetical ‘Suggested Readings’ RS. We discover that higher RS editorial authority assigned to students is linked to the relative trust the course staff allocates to RS algorithm and students. We also observe that course staff favors higher control for the RS algorithm in sourcing and updating the recommendations long-term. Using content analysis, we discuss frequent staff-recommended student editorial roles and highlight their frequent rationales, such as perceived expertise, scaling the learning environment, professional curriculum needs, and learner disengagement. We argue that our analyses highlight critical user preferences to help detect editorial power asymmetry and identify RS use-cases for supporting teaching and research.

Assessing Cognitive Test Performance Using Automatic Digital Pen Features Analysis

  • Alexander Prange
  • Daniel Sonntag

Most cognitive assessments, for dementia screening for example, are conducted with a pen on normal paper. We record these tests with a digital pen as part of a new interactive cognitive assessment tool with automatic analysis of pen input. The clinician can, first, observe the sketching process in real-time on a mobile tablet, e.g., in telemedicine settings or to follow Covid-19 distancing regulations. Second, the results of an automatic test analysis are presented to the clinician in real-time, thereby reducing manual scoring effort and producing objective reports. The presented research describes the architecture of our cognitive assessment tool and examines how accurately different machine learning (ML) models can automatically score cognitive tests, without a semantic content analysis. Our system uses a set of more than 170 pen features, calculated directly from the raw digital pen signal. We evaluate our system with 40 subjects from a geriatrics daycare clinic. Using standard ML techniques our feature set outperforms previous approaches on the cognitive tests we consider, i.e., the Clock Drawing, the Rey-Osterrieth Complex Figure, and the Trail Making Test, by automatically scoring tests with up to 82% accuracy in a binary classification task.

Do Users Have Contextual Preferencesfor Smartphone Power Management?

  • Sophie Welber
  • Valerie Zhao
  • Claire Dolin
  • Olivia Morkved
  • Henry Hoffmann
  • Blase Ur

Smartphones must balance power and performance. While most smartphones offer a power-saving mode, they typically provide a binary choice between full performance and monolithic performance degradation (e.g., reducing both screen brightness and processing speed) to save power. Could smartphones improve the user experience by automatically degrading only selected features based on the usage context? To gauge whether preferences for power-saving strategies vary by context, we conducted a 304-participant, survey-based experiment. Each participant was assigned a context (e.g., navigation) and degradation level. They viewed a series of side-by-side simulations of one smartphone operating normally in that context and another operating with reduced GPS accuracy, processing speed, or screen brightness. Participants rated their willingness to accept each tradeoff to save power. Contrasting current power-saving modes, we found that participants’ preferences did indeed vary by context. Using factor analysis to cluster preferences, we identified key personas that pave the way toward context-aware and self-aware alternatives to smartphone power-saving modes.

Bayesian Preference Elicitation with Keyphrase-Item Coembeddings for Interactive Recommendation

  • Hojin Yang
  • Scott Sanner
  • Ga Wu
  • Jin Peng Zhou

Interactive (a.k.a. conversational) recommendation systems provide the potential capability to personalize interactions with increasingly prevalent dialog-based AI assistants. In the conversational recommendation setting, a user often has long-term preferences inferred from previous interactions along with ephemeral session-based preferences that need to be efficiently elicited through minimal interaction. Historically, Bayesian preference elicitation methods have proved effective for (i) leveraging prior information to incrementally estimate uncertainty in user preferences as new information is observed, and for (ii) supporting active elicitation of preference feedback to quickly zero in on the best recommendations in a session. Previous work typically focused on eliciting preferences in the space of items or a small set of attributes; in the dialog-based setting, however, we are faced with the task of eliciting preferences in the space of natural language while using this feedback to determine a user’s preferences in item space. To address this task in the era of modern, latent embedding-based recommender systems, we propose a method for coembedding user-item preferences with keyphrase descriptions (i.e., not explicitly known attributes, but rather subjective judgments mined from user reviews or tags) along with a closed-form Bayesian methodology for incrementally estimating uncertainty in user preferences based on elicited keyphrase feedback. We then combine this framework with well-known preference elicitation techniques that can leverage Bayesian posteriors such as Upper Confidence Bounds, Thompson Sampling, and a variety of other methods. Our empirical evaluation on real-world datasets shows that the proposed query selection strategies effectively update user beliefs, leading to high-quality recommendations with a minimal number of keyphrase queries.

Towards a User Integration Framework for Personal Health Decision Support and Recommender Systems

  • Katja Herrmanny
  • Helma Torkamaan

Supporting personal health with Decision Support Systems (DSS) and, specifically, recommender systems (RS) is a promising and growing area of research. Integrating the user in the loop is vital in such health systems due to the complexity of recommendations, gravity of the decisions and the reliance on user autonomy. However, for such a purpose, to the best of our knowledge there exists no profound or comprehensive framework nor model to guide system designers, to exploit the full potential of integrating users in the system’s reasoning process by design. In this paper, we present a multifaceted user integration framework in personal health-related DSS and RS. This framework, with three main components, has been derived from an iterative mixed-methods development and evaluation procedure, including expert workshops and extensive multidisciplinary literature reviews. Users are accordingly integrated into the whole process from system reasoning until decision making through the following actionable design strategies: (1) Empower: Enabling them to understand the result generation and implications, (2) Encourage: encouraging them to question and reflect system outcomes and to get involved in the generation process and (3) Engage: enabling them to take an active role by facilitating and providing opportunities for user control. The framework offers support to designers of personal health-related DSS and RS in properly integrating users into their systems.

Second Chance for a First Impression? Trust Development in Intelligent System Interaction

  • Suzanne Tolmeijer
  • Ujwal Gadiraju
  • Ramya Ghantasala
  • Akshit Gupta
  • Abraham Bernstein

There is a growing use of intelligent systems to support human decision-making across several domains. Trust in intelligent systems, however, is pivotal in shaping their widespread adoption. Little is currently understood about how trust in an intelligent system evolves over time and how it is mediated by the accuracy of the system. We aim to address this knowledge gap by exploring trust formation over time and its relation to system accuracy. To that end, we built an intelligent house recommendation system and carried out a longitudinal study consisting of 201 participants across 3 sessions in a week. In each session, participants were tasked with finding housing that fit a given set of constraints using a conventional web interface that reflected a typical housing search website. Participants could choose to use an intelligent decision support system to help them find the right house. Depending on the group, participants received a variation of accurate or inaccurate advice from the intelligent system throughout each session. We measured trust using a trust in automation scale at the end of each session.

We found evidence suggesting that trust development is a slow process that evolves over multiple sessions, and that first impressions of the intelligent system are highly influential. Our results echo earlier research on trust formation in single session interactions, corroborating that reliability, validity, predictability, and dependability all influence trust formation. We also found that the age of the participants and their affinity with technology had an effect on their trust in the intelligent system. Our findings highlight the importance of first impressions and improvement of system accuracy for trust development. Hence, our study is an important first step in understanding trust development, breakdown of trust, and trust repair over multiple system interactions, informing improved system design.

Characterising Student Engagement Modes through Low-Level Activity Patterns

  • Ann Gledson
  • Aitor Apaolaza
  • Sabine Barthold
  • Franziska Günther
  • He Yu
  • Markel Vigo

Existing approaches to characterise engagement in online learning focus on features of the interaction of students with the learning platform including the number of posts in forums, downloads of learning materials and time spent watching videos. However, little is known about what students actually do within the learning resources and whether these activities are indicators of learning outcomes. To bridge this gap, we associate low-level activity patterns with particular student engagement modes on a connectivist MOOC (cMOOC) that ran for four weeks and involved 224 students. Our findings indicate that our approach isolates meaningful interactive behavioural markers that are indicators of engagement, and are amenable to computation.

Generating Interesting Song-to-Song Segues With Dave

  • Giovanni Gabbolini
  • Derek Bridge

We introduce a novel domain-independent algorithm for generating interesting item-to-item textual connections, or segues. Pivotal to our contribution is the introduction of a scoring function for segues, based on their ‘interestingness’. We provide an implementation of our algorithm in the music domain. We refer to our implementation as Dave. Dave is able to generate 1553 different types of segues, that can be broadly categorized as either informative or funny. We evaluate Dave by comparing it against a curated source of song-to-song segues, called The Chain. In the case of informative segues, we find that Dave can produce segues of the same quality, if not better, than those to be found in The Chain. And, we report positive correlation between the values produced by our scoring function and human perceptions of segue quality. The results highlight the validity of our method, and open future directions in the application of segues to recommender systems research.

”It’s like a puppet master”: User Perceptions of Personal Autonomy when Interacting with Intelligent Technologies

  • Supraja Sankaran
  • Panos Markopoulos

Applications which use some form of artificial intelligence (AI) have become embedded in our everyday interactions. Very often, AI-based apps are personalized and modelled on users’ needs and preferences. However, such applications of AI tread a delicate balance between enhancing user experience and jeopardizing personal autonomy. Personal autonomy and sense of agency are crucial for human well-being and development. In this paper, we probe this fine balance aiming to capture users’ lived experiences and perceptions of interacting with AI-based apps. We present insights from a phenomenological study (N=15) regarding users’ perception of personal autonomy when interacting with AI in everyday contexts. We found that these experiences are transitory and largely influenced by contextual factors. Users experience a loss of autonomy when their privacy or identity is threatened or when their expectations are broken. To mitigate such loss of autonomy, mechanisms for providing intelligibility and control of AI are desired.

User-centered Evaluation of Popularity Bias in Recommender Systems

  • Himan Abdollahpouri
  • Masoud Mansoury
  • Robin Burke
  • Bamshad Mobasher
  • Edward Malthouse

Recommendation and ranking systems are known to suffer from popularity bias; the tendency of the algorithm to favor a few popular items while under-representing the majority of other items. Prior research has examined various approaches for mitigating popularity bias and enhancing the recommendation of long-tail, less popular, items. The effectiveness of these approaches is often assessed using different metrics to evaluate the extent to which over-concentration on popular items is reduced. However, not much attention has been given to the user-centered evaluation of this bias; how different users with different levels of interest towards popular items are affected by such algorithms. In this paper, we show the limitations of the existing metrics to evaluate popularity bias mitigation when we want to assess these algorithms from the users’ perspective and we propose a new metric that can address these limitations. In addition, we present an effective approach that mitigates popularity bias from the user-centered point of view. Finally, we investigate several state-of-the-art approaches proposed in recent years to mitigate popularity bias and evaluate their performances using the existing metrics and also from the users’ perspective. Our experimental results using two publicly-available datasets show that existing popularity bias mitigation techniques ignore the users’ tolerance towards popular items. Our proposed user-centered method can tackle popularity bias effectively for different users while also improving the existing metrics.

Emotion Detection through Smartphone's Accelerometer and Gyroscope Sensors

  • Orestis Piskioulis
  • Katerina Tzafilkou
  • Anastasios a. Economides

Emotion recognition is essential for assessing human emotional states and predicting user behavior to provide appropriate and personalized feedback. The wide range of Smartphones with accelerometers, microphones, GPSs, gyroscopes, and more motivate researchers to explore the automatic emotion detection through Smartphone sensors. To this end, mobile sensing can facilitate the data retrieval process in a non-intrusive way without disturbing the user's experience. This study seeks to contribute to the field of non-intrusive mobile sensing for emotion recognition by detecting user emotions via accelerometer and gyroscope sensors in Smartphones. A prototype gaming app was designed and a sensor log app for Android OS was used to monitor the users’ sensor data while interacting with the game. The recorded data from 40 users was processed and used to train different classifiers for two emotions: a positive (enjoyment) and a negative (frustration) one. The validation study demonstrates a high prediction of 87.90% for enjoyment and 89.45% for frustration. Our findings indicate that by analyzing accelerometer and gyroscope data, it is possible to make efficient predictions of a user's emotional state. The proposed model and its empirical development and validation are described in this paper.

Culture and Health Belief Model: Exploring the Determinants of Physical Activity Among Saudi Adults and the Moderating Effects of Age and Gender

  • Najla Almutari
  • Rita Orji

Physical inactivity is a significant risk factor for many non-communicable diseases such as heart disease, diabetes, and evidence shows that physical inactivity is one of the highest risk factors for death globally. Research has shown that theory-driven persuasive interventions are more effective at promoting behaviour change than generic ones. However, research on the determinants of physical activity and the moderating effect of age and gender among non-Western culture is limited. To close this gap, we conduct a large-scale study of the determinants of physical activity among 217 participants from Saudi Arabia using the extended Health Belief Model (HBM), a commonly applied behavioural model in health interventions design. We also assessed for the moderating effect of age and gender. Our findings show that Social influence, Cue to action and Perceived severity are the strongest determinants of physical activity in Saudi adults. We map these determinants to their corresponding persuasive strategies that can be used in operationalizing them in persuasive applications for promoting physical activity. Finally, we discuss the implication of our findings and offer design guidelines for persuasive interventions that appeal to both a broad audience and tailored to a particular group depending on their gender and age group.

Exploring the Effects of Natural Language Justifications in Food Recommender Systems

  • Cataldo Musto
  • Alain D. Starke
  • Christoph Trattner
  • Amon Rapp
  • Giovanni Semeraro

Users of food recommender systems typically prefer popular recipes, which tend to be unhealthy. To encourage users to select healthier recommendations by making more informed food decisions, we introduce a methodology to generate and present a natural language justification that emphasizes the nutritional content, or health risks and benefits of recommended recipes. We designed a framework that takes a user and two food recommendations as input and produces an automatically generated natural language justification as output, which is based on the user’s characteristics and the recipes’ features. In doing so, we implemented and evaluated eight different justification strategies through two different justification styles (e.g., comparing each recipe’s food features) in an online user study (N = 503). We compared user food choices for two personalized recommendation approaches, popularity-based vs our health-aware algorithm, and evaluated the impact of presenting natural language justifications. We showed that comparative justifications styles are effective in supporting choices for our healthy-aware recommendations, confirming the impact of our methodology on food choices.

Effects and Ways of Tailored Gamification in Software-Based Training in Cognitive Rehabilitation

  • Mareike Gabele
  • Juliane Weicker
  • Sebastian Wagner
  • Andrea Thoms
  • Steffi Huβlein
  • Christian Hansen

A high level of motivation and frequent training are relevant in software-based rehabilitation to improve cognitive functioning after acquired brain injury. We evaluated the benefit of tailored user-centered gamification elements in a clinical study with N=83 outpatients undergoing three weeks of cognitive training in their home environment. The use of gamification in relation to the patient’s player type was explored in three steps. First, we determined the individual player types and related requests for specific game elements by means of questionnaires. Afterwards, we examined the effect of gamified training based on a non-player character and training progress within a metaphor. We considered secondly the individual perception and emotional effect and thirdly the performance based on training duration. 37 elements were requested by patients of all types, 18 elements were partially requested, and 4 elements were rejected. A comparison shows that the requested game elements partly differ between healthy persons and patients. Overall, gamification was perceived positively and gamified training leads to an increase in enjoyment compared to non-gamified training. In detail, however, there were different effects on the individual player types: socialisers experienced more enjoyment while achievers perceived higher competence throughout gamified cognitive training. Also, differences in performance in training duration were found. Within gamified training, socialisers trained significantly more than patients not primarily assigned to this type. In contrast, no significant difference was found for achievers. By showing modulating requests and effects in player types, our results support user-centered tailoring of game elements in the development of software-based cognitive training in rehabilitation.

Privacy as a Planned Behavior: Effects of Situational Factors on Privacy Perceptions and Plans

  • A. K. M. Nuhil Mehdy
  • Michael D. Ekstrand
  • Bart P. Knijnenburg
  • Hoda Mehrpouyan

To account for privacy perceptions and preferences in user models and develop personalized privacy systems, we need to understand how users make privacy decisions in various contexts. Existing studies of privacy perceptions and behavior focus on overall tendencies toward privacy, but few have examined the context-specific factors in privacy decision making. We conducted a survey on Mechanical Turk (N=401) based on the theory of planned behavior (TPB) to measure the way users’ perceptions of privacy factors and intent to disclose information are affected by three situational factors embodied hypothetical scenarios: information type, recipients’ role, and trust source. Results showed a positive relationship between subjective norms and perceived behavioral control, and between each of these and situational privacy attitude; all three constructs are significantly positively associated with intent to disclose. These findings also suggest that, situational factors predict participants’ privacy decisions through their influence on the TPB constructs.

Knowledge Tracing for Complex Problem Solving: Granular Rank-Based Tensor Factorization

  • Chunpai Wang
  • Shaghayegh Sahebi
  • Siqian Zhao
  • Peter Brusilovsky
  • Laura O. Moraes

Knowledge Tracing (KT), which aims to model student knowledge level and predict their performance, is one of the most important applications of user modeling. Modern KT approaches model and maintain an up-to-date state of student knowledge over a set of course concepts according to students’ historical performance in attempting the problems. However, KT approaches were designed to model knowledge by observing relatively small problem-solving steps in Intelligent Tutoring Systems. While these approaches were applied successfully to model student knowledge by observing student solutions for simple problems, such as multiple-choice questions, they do not perform well for modeling complex problem solving in students. Most importantly, current models assume that all problem attempts are equally valuable in quantifying current student knowledge. However, for complex problems that involve many concepts at the same time, this assumption is deficient. It results in inaccurate knowledge states and unnecessary fluctuations in estimated student knowledge, especially if students guess the correct answer to a problem that they have not mastered all of its concepts or slip in answering the problem that they have already mastered all of its concepts. In this paper, we argue that not all attempts are equivalently important in discovering students’ knowledge state, and some attempts can be summarized together to better represent student performance. We propose a novel student knowledge tracing approach, Granular RAnk based TEnsor factorization (GRATE), that dynamically selects student attempts that can be aggregated while predicting students’ performance in problems and discovering the concepts presented in them. Our experiments on three real-world datasets demonstrate the improved performance of GRATE, compared to the state-of-the-art baselines, in the task of student performance prediction. Our further analysis shows that attempt aggregation eliminates the unnecessary fluctuations from students’ discovered knowledge states and helps in discovering complex latent concepts in the problems.

Progression Trajectory-Based Student Modeling for Novice Block-Based Programming

  • Fahmid Morshed Fahid
  • Xiaoyi Tian
  • Andrew Emerson
  • Joseph B. Wiggins
  • Dolly Bounajim
  • Andy Smith
  • Eric Wiebe
  • Bradford Mott
  • Kristy Elizabeth Boyer
  • James Lester

Block-based programming environments are widely used in computer science education. However, these environments pose significant challenges for student modeling. Given a series of problem-solving actions taken by students in block-based programming environments, student models need to accurately infer problem-solving students’ programming abilities in real time to enable adaptive feedback and hints that are tailored to students’ abilities. While student models for block-based programming offer the potential to support student-adaptivity, creating student models for these environments is challenging because students can develop a broad range of solutions to a given programming activity. To address these challenges, we introduce a progression trajectory-based student modeling framework for modeling novice student block-based programming across multiple learning activities. Student trajectories utilize a time series representation that employs code analysis to incrementally compare student programs to expert solutions as students undertake block-based programming activities. This paper reports on a study in which progression trajectories were collected from more than 100 undergraduate students engaging in a series of block-based programming activities in an introductory computer science course. Using progression trajectory-based student modeling, we identified three distinct trajectory classes: Early Quitting, High Persistence, and Efficient Completion. Analysis of these trajectories revealed that they exhibit significantly different characteristics with respect to students’ actions and can be used to accurately predict students’ programming behaviors on future programming activities compared to competing baseline models. The findings suggest that progression trajectory-based student models can accurately model students’ block-based programming problem solving and hold potential for informing adaptive support in block-based programming environments.

Data-Driven Modeling of Learners’ Individual Differences for Predicting Engagement and Success in Online Learning

  • Kamil Akhuseyinoglu
  • Peter Brusilovsky

Individual differences have been recognized as an important factor in the learning process. However, there are few successes in using known dimensions of individual differences in solving an important problem of predicting student performance and engagement in online learning. At the same time, learning analytics research has demonstrated that the large volume of learning data collected by modern e-learning systems could be used to recognize student behavior patterns and could be used to connect these patterns with measures of student performance. Our paper attempts to bridge these two research directions. By applying a sequence mining approach to a large volume of learner data collected by an online learning system, we build models of student learning behavior. However, instead of following modern work on behavior mining (i.e., using this behavior directly for performance prediction tasks), we attempt to follow traditional work on modeling individual differences in quantifying this behavior on a latent data-driven personality scale. Our research shows that this data-driven model of individual differences performs significantly better than several traditional models of individual differences in predicting important parameters of the learning process, such as success and engagement.

Intelligent Shifting Cues: Increasing the Awareness of Multi-Device Interaction Opportunities

  • Ngoc Thi Nguyen
  • Agustin Zuniga
  • Huber Flores
  • Hyowon Lee
  • Simon Tangi Perrault
  • Petteri Nurmi

The ever-increasing ubiquity of smart devices is creating new opportunities for people to interact and engage with digital information using multiple devices. In the simplest case this can refer to choosing which device to use for a particular task (e.g., phone, laptop or smartwatch), whereas a more complex example is simultaneously taking advantage of the capabilities of different devices (e.g., laptop and smart TV). Despite these types of opportunities becoming increasing available, currently the full potential of multi-device interactions is not being realized as people struggle to take advantage of them. As our first contribution, we study people’s willingness to engage with multi-device interactions and rank the factors that mediate this response through an online survey (N = 60). Our results show that users are strongly in favour of using multiple devices, but lack the awareness or information to engage with them, or feel that establishing the interactions is too laborious and would disrupt the fluidity of the interactions. Motivated by this result, as our second contribution we design and evaluate intelligent shifting cues, visualizations that offer information about available interaction opportunities and how to establish them, and study how they influence users willingness to engage in multi-device interactions. Results of our study show that the cues can be effective in helping people to engage with multiple devices, but that the suitability of the proposed device and fit with task are important mediating factors. We end the paper by deriving design implications for intelligent systems that can support people in engaging with multi-device interactions.


Short Papers

Exploring Multi-List User Interfaces for Similar-Item Recommendations

  • Dietmar Jannach
  • Mathias Jesse
  • Michael Jugovac
  • Christoph Trattner

On many e-commerce and media streaming sites, the user interface (UI) consists of multiple lists of item suggestions. The items in each list are usually chosen based on pre-defined strategies and, e.g., show movies of the same genre or category. Such interfaces are common in practice, but there is almost no academic research regarding the optimal design and arrangement of such multi-list UIs for recommenders. In this paper, we report the results of an exploratory user study that examined the effects of various design alternatives on the decision-making behavior of users in the context of similar-item recommendations. Our investigations showed, among other aspects, that decision-making is slower and more demanding with multi-list interfaces, but that users also explore more options before making a decision. Regarding the selection of the algorithm to retrieve similar items, our study furthermore reveals the importance of considering social-based similarity measures.

Psychographic Matching between a Call Center Agent and a Customer

  • Akihiro Kobayashi
  • Yuichi Ishikawa
  • Roberto Sebastian Legaspi

The interpersonal compatibility of a company agent and a customer significantly affects the outcome of their communication. In existing research, however, compatibility has been studied only in terms of the similarity in personality and values. That is, agents and customers were compared only on the same dimensions that make up their personality and values, e.g., on the same trait of the Big Five (compared agent's Extraversion and customer's Extraversion) or same values as per Schwartz's Basic Values (agent's Conformity and customer's Conformity). In this paper, we studied compatibility from a broader perspective, i.e., in addition to the similarity, we investigated interactions across different dimensions (e.g., an agent's Extraversion and a customer's Conformity, or the former's Extraversion and the latter's Neuroticism). Examining 7,594 real call logs collected from telemarketing call centers, we have confirmed that such different dimensional interactions significantly affect a customer making a purchase (i.e., a customer's conversion) or not. A simulation where we matched agents and customers demonstrated that our compatibility model that incorporated the interactions across different dimensions yielded significant conversion lift, i.e., +46% on the average, compared from one that used only similarity in personality and values.

Personality and Engagement with Digital Mental Health Interventions

  • Mohammed Khwaja
  • Svenja Pieritz
  • A. Aldo Faisal
  • Aleksandar Matic

Personalisation is key to creating successful digital health applications. Recent evidence links personality and preference for digital experience — suggesting that psychometric traits can be a promising basis for personalisation of digital mental health services. However, there is still little quantitative evidence from actual app usage. In this study, we explore how different personality types engage with different intervention content in a commercial mental health application. Specifically, we collected the Big Five personality traits alongside the app usage data of 126 participants using a mobile mental health app for seven days. We found that personality traits significantly correlate with the engagement and user ratings of different intervention content. These findings open a promising research avenue that can inform the personalised delivery of digital mental health content and the creation of recommender systems, ultimately improving the effectiveness of mental health interventions.

How Do User Opinions Influence Their Interaction With Web Search Results?

  • Luyan Xu
  • Mengdie Zhuang
  • Ujwal Gadiraju

Understanding the influence of users’ opinions on their search behavior together with their inherent biases in web search has garnered widespread interest in recent times. This is largely due to the implications of promoting critical thinking, explaining phenomena such as political polarization, or the manifestation of echo chambers. It is important to understand how personal opinions can bias users’ interaction with search results. Moreover, there is a lack of understanding of the impact of user search intents, namely non-purposeful browsing versus searching with a pre-defined goal, on users’ interactions with search results. We take a step towards bridging this knowledge gap through an empirical study in this paper. To do so, we select two controversial topics in abortion and gun control, and invite users to learn about them through ‘Purposeless’ and ‘Purposeful’ web searching. Our findings suggest that users with strong personal opinions exhibit biased interactions with the search results. However, the effect of users’ opinions on their interactions with search results can differ depending on whether users search purposelessly or with a purpose. Our findings advance the current understanding of the effect of users’ opinions in web search sessions, and show that users’ search intents shape their interaction with search results. This work has broad design implications on dealing with bias in interactive information retrieval systems.

Translating a Typing-Based Adaptive Learning Model to Speech-Based L2 Vocabulary Learning

  • Thomas Wilschut
  • Maarten van der Velde
  • Florian Sense
  • Zafeirios Fountas
  • Hedderik van Rijn

Memorising vocabulary is an important aspect of formal foreign language learning. Advances in cognitive psychology have led to the development of adaptive learning systems that make vocabulary learning more efficient. These computer-based systems measure learning performance in real time to create optimal study strategies for individual learners. While such adaptive learning systems have been successfully applied to written word learning, they have thus far seen little application in spoken word learning. Here we present a system for adaptive, speech-based word learning. We show that it is possible to improve the efficiency of speech-based learning systems by applying a modified adaptive model that was originally developed for typing-based word learning. This finding contributes to a better understanding of the memory processes involved in speech-based word learning. Furthermore, our work provides a basis for the development of language learning applications that use real-time pronunciation assessment software to score the accuracy of the learner’s pronunciations. Speech-based learning applications are educationally relevant because they focus on what may be the most important aspect of language learning: to practice speech.

Personalisation in Cyber-Physical-Social Systems: A Multi-stakeholder aware Recommendation and Guidance

  • Bereket Abera Yilma
  • Yannick Naudet
  • Hervé Panetto

The evolution of smart devices has led to the transformation of many physical spaces to the so-called smart environments collectively termed as Cyber-Physical-Social System (CPSS). In CPSS users co-exist with different stakeholders influencing each other while being influenced by different environmental factors. Additionally, these environments often have their own desired goals and corresponding set of rules in place expecting people to behave in certain ways. Hence, in such settings classical approaches to personalisation which solely optimise for user satisfaction are often encumbered by competing objectives and environmental constraints which are yet to be addressed jointly. In this work we set out to (i) formalise the general problem of personalisation in CPSS from a multi-stakeholder perspective taking into account the full environmental complexity, (ii) extend the general formalisation to the case of exhibition areas and propose a personalised Multi-stakeholder aware Recommendation and Guidance method on a case study of National Gallery, London.

Right Way, Right Time: Towards a Better Comprehension of Young Students’ Needs when Looking for Relevant Search Results

  • Monica Landoni
  • Mohammad Aliannejadi
  • Theo Huibers
  • Emiliana Murgia
  • Maria Soledad Pera

We report on the exploration we conducted to understand better children’s needs for the design of Search Engine Result Pages (SERP) that can help them notice relevant resources when performing online inquiry tasks in a classroom context. We analyse children’s interactions with traditional and emoji-enriched SERP and look for trends linking children’s engagement with SERP and search success (based on experts’ assessments). We also identify areas remaining to be interpreted and considered in future studies. On this mixed ground, we discuss the complexity of this design space and the need to bypass the one-size-fits-all approach in favor of adaptive SERP to cater to children’s different and ever-evolving skills in searching and recognising useful results if we aim to support learning.

Dynamic Modeling of User Preferences for Stable Recommendations

  • Oluwafemi Olaleke
  • Ivan Oseledets
  • Evgeny Frolov

In domains where users tend to develop long-term preferences that do not change too frequently, the stability of recommendations is an important factor of the perceived quality of a recommender system. In such cases, unstable recommendations may lead to poor personalization experience and distrust, driving users away from a recommendation service. We propose an incremental learning scheme that mitigates such problems via the dynamic modeling approach. It incorporates a generalized matrix form of a partial differential equation integrator that yields a dynamic low-rank approximation of time-dependent matrices representing user preferences. The scheme allows extending the famous PureSVD approach to time-aware settings and significantly improves its stability without sacrificing the accuracy in standard top-n recommendations tasks.

Pivoting Image-based Profiles Toward Privacy: Inhibiting Malicious Profiling with Adversarial Additions

  • Zhuoran Liu
  • Zhengyu Zhao
  • Martha Larson

Users build up profiles online consisting of items that they have shared or interacted with. In this work, we look at profiles that consist of images. We address the issue of privacy-sensitive information being automatically inferred from these user profiles, against users’ will and best interest. We introduce the concept of a privacy pivot, which is a strategic change that users can make in their sharing that will inhibit malicious profiling. Importantly, the pivot helps put privacy control into the hands of the users. Further, it does not require users to delete any of the existing images in their profiles, nor does it require a radical change in their sharing intentions, i.e., what they would like to communicate with their profile. Previous work has investigated adversarial images for privacy protection, but has focused on individual images. Here, we move further to study image sets comprising image profiles. We define a conceptual formulation of the challenge of the privacy pivot in the form of an “Anti-Profiling Model”. Within this model, we propose a basic pivot solution that uses adversarial additions to effectively inhibit the predictions of profilers using set-based image classification.

Fairness and Transparency in Recommendation: The Users’ Perspective

  • Nasim Sonboli
  • Jessie J. Smith
  • Florencia Cabral Berenfus
  • Robin Burke
  • Casey Fiesler

Though recommender systems are defined by personalization, recent work has shown the importance of additional, beyond-accuracy objectives, such as fairness. Because users often expect their recommendations to be purely personalized, these new algorithmic objectives must be communicated transparently in a fairness-aware recommender system. While explanation has a long history in recommender systems research, there has been little work that attempts to explain systems that use a fairness objective. Even though the previous work in other branches of AI has explored the use of explanations as a tool to increase fairness, this work has not been focused on recommendation. Here, we consider user perspectives of fairness-aware recommender systems and techniques for enhancing their transparency. We describe the results of an exploratory interview study that investigates user perceptions of fairness, recommender systems, and fairness-aware objectives. We propose three features – informed by the needs of our participants – that could improve user understanding of and trust in fairness-aware recommender systems.

Model-Agnostic Counterfactual Explanations of Recommendations

  • Vassilis Kaffes
  • Dimitris Sacharidis
  • Giorgos Giannopoulos

Explanations for algorithmically generated recommendations is an important requirement for transparent and trustworthy recommender systems. When the internal recommendation model is not inherently interpretable (e.g., most contemporary systems are complex and opaque), or when access to the system is not available (e.g., recommendation as a service), explanations have to be generated post-hoc, i.e., after the system is trained. In this common setting, the standard approach is to provide plausible interpretations of the observed outputs of the system, e.g., by building a simple surrogate model that is inherently interpretable, and explaining that model. This however has several drawbacks. First, such explanations are not truthful, as they are rationalizations of the observed inputs and outputs constructed by another system. Second, there are privacy concerns, as to train a surrogate model, one has to know the interactions from users other than the one who seeks an explanation. Third, such explanations may not be scrutable and actionable, as they typically return weights for items or other users that are difficult to comprehend, and hard to act upon so to improve the quality of one’s recommendations.

In this work, we present a model-agnostic explanation mechanism that is truthful, private, scrutable, and actionable. The key idea is to provide counterfactual explanations, defined as those small changes to the user’s interaction history that are responsible for observing the recommendation output to be explained. Without access to the internal recommendation model, finding concise counterfactual explanations is a hard search problem. We propose several strategies that seek to efficiently extract concise explanations under constraints. Experimentally, we show that these strategies are more efficient and effective than exhaustive and random search.


Doctoral Consortium

Adaptive Visualisations Using Spatiotemporal and Heuristic Models to Support Piano Learning

  • Jordan Aiko Deja

Learning the piano is hard and many approaches including piano-roll visualisations have been explored in order to support novices and seasoned learners in this process. However, existing piano roll prototypes have not considered the spatiotemporal component (user’s ability to press on a moving target) when generating these visualisations and user modelling. In this PhD, we are going to look into two different approaches: (i) exploring whether existing techniques in single-target spatiotemporal modelling can be adapted to a multi-target scenario such as when learners use several fingers to press multiple moving targets when playing the piano, and (ii) exploring heuristics defined by experts marking various difficult parts of songs, and deciding on specific interventions needed for these marked parts. Using models and input from the experts we will design and build an adaptive piano roll training system. We will evaluate and compare these models in various user studies involving users trying to play piano pieces and develop their improvisation skills. We intend to uncover whether these adaptive visualisations will be helpful in the overall training of piano learners. Additionally, these models and adaptive visualisations will allow us to discover affordances that can potentially improve piano learning in general.

Adaptive Visualizations for Enhanced Data Understanding and Interpretation

  • Christos Amyrotos

In a data driven economy where data volume and dimensions are explosively increasing, businesses rely on business intelligence and analytics (BI&A) platforms for analysing their data and coming to beneficial decisions. With the ever-growing generation of data, the process of data analysis is becoming more complicated for the business users, as the exploration of more demanding use cases increases. While the existing BI&A platforms provide myriads of data visualizations that support data exploration, none of those account for the user’s individual differences, needs or requirements, and thus may hinder the user’s understanding of visual data and consequently their decision-making processes. This work embarks on an interdisciplinary endeavour to introduce a human-centred adaptive data visualizations framework in the context of business, as the core of an adaptive data analytics platform, that aims to enhance the business user’s decision making by increasing her understanding of data. The framework is built using a multi-dimensional human-centred user model that goes beyond traditional user characteristics and accounts for cognitive factors, domain expertise and experience and factors related to the business context i.e., data, visualizations and tasks; a data visualization engine that will recommend to the unique-user the best-fit data visualizations based on the abovementioned user model; and an intelligent data analytics component that enhances the efficiency and effectiveness of the data exploration process by leveraging user interactions during the explorations to further inform the user model on the user’s expertise and experience.

Climber Behavior Modeling and Recommendation

  • Iustina Ivanova

Sport climbing is becoming more and more popular, even among non-specialists. While new routes are built each year, both indoor and outdoor, there is no effective tool for supporting climbers to choose the most appropriate routes, either for training or simply enjoying. Route recommendation is hard and risky because a reliable evaluation of the climber’s capabilities, status and subjective difficulty perception is necessary. This can be achieved only with the exploitation of Internet of Things (IoT) sensors for the automatic recording of climbers’ activity. In this research, we want to further extend the still young research subject of activity recognition in sport climbing and combine this with new recommender systems (RSs) techniques for route suggestion. We have developed an initial solution for unobtrusively and automatically detecting climbers’ activities in a gym, and we are now connecting this information with the manual inserted diary data of climbers by means of a mobile application. We present the design and the open research questions for a system that leverages sensor data and explicit feedback to generate a climber’s profile and recommend suitable routes.

User-Centric Item Characteristics for Modeling Users and Improving Recommendations

  • Elham  Motamedi

The PhD research presented in this paper discusses the idea of using user-centric item characteristics (UCIC) such as the eudaimonic/hedonic quality of multimedia items for achieving a better performance of the recommender systems. UCIC is the characteristics of the item that has its root in the way users perceive the item. Different users have different perceptions and therefore UCIC is a distribution of these perceptions of the item characteristic. For example, movies have a different value of UCIC concerning the induced emotion in users. Therefore, the quality of induced emotion is the characteristic of the item that its value is changed based on how the item is triggering different emotional reactions in users. One of our objectives is to predict the value of UCIC for the items. One possibility of describing UCIC can be by predicting mean and standard variation of perception of users of the specific item characteristic. Another objective is to use the variance in perception of the item characteristic in different users to personalize the recommendations based on the predicted score of the perceived item characteristic for different users. The thesis is composed of three original scientific contributions: (i) devise a method for labeling items with UCIC value and in particular the UCIC of hedonic/eudaimonic quality, (ii) devise models of user behavior based on the perception of item characteristic and in particular eudaimonic/hedonic perception, (iii) devise recommender systems with the incorporation of UCIC concepts for generating more accurate recommendations.

Leveraging Unstructured Text Within the Context of Conversational Agents

  • Maria Phillips

We propose exploring alternative designs for a conversational agent developed as a tool to provide feedback within the education domain for pre-service teachers, students pursuing their teaching certificate, to practice their questioning skills in a given scenario. We utilize a component-based approach in the design of our conversational agent and this research focuses on proposing methods within the knowledge base component specifically leveraging unstructured text as the foundation of the knowledge base. Through leveraging unstructured text we intend to explore the possibilities of improving conversational agent response quality while minimizing resources required of domain experts in scenario development.

Adapting Online Messaging Based on Emotional State

  • Moeen Mostafavi

Online messaging is an increasingly popular form of communication, yet it has difficulty facilitating an accurate interpretation of users’ emotions. In the absence of body language and vocal characteristics, emojis, gifs, and stickers help users communicate their emotions. A challenging but effective approach to improving these communication types requires interpreting emotions accurately and adapting the conversation accordingly. In this research, we focus on modeling emotional state in an online messaging session. Using Affect Control Theory (ACT), machines can predict emotional change during the interaction. For example, a chatbot that understands the emotional states can adapt its messages and offers. This framework helps the chatbot to communicate naturally and efficiently. Similarly, a messaging platform uses this framework to recommend more appropriate stickers and gifs to a user.



Ethical Considerations in User Modeling and Personalization

  • Jim Torresen

Ethical considerations are getting increased attention with regards to providing responsible personalization for robots and autonomous systems. This is partly as a result of the currently limited deployment of such systems in human support and interaction settings. The tutorial will give an overview of the most commonly expressed ethical challenges and ways being undertaken to reduce their impact using the findings in an earlier undertaken review supplemented with recent work and initiatives. The tutorial will exemplify the challenges related to privacy, security and safety through several examples from own and others’ work.

Multi-Method Evaluation of Adaptive Systems

  • Christine Bauer

When evaluating personalized or adaptive systems, we frequently rely on one single evaluation objective and one single method. This remains us with “blind spots”. A comprehensive evaluation may require a thoughtful integration of multiple methods. This tutorial (i) demonstrates the wide variety of dimensions to be evaluated, (ii) outlines the methodological approaches to evaluate these dimensions, (iii) pinpoints the blind spots when using only one approach, (iv) demonstrates the benefits of multi-method evaluation, and (v) outlines the basic options how multiple methods can be integrated into one evaluation design. Participants familiarize with the wide spectrum of opportunities how adaptive or personalized systems may be evaluated, and have the opportunity to come up with evaluation designs that comply with the four basic options of multi-method evaluation. The ultimate learning objective is to stimulate the critical reflection of one’s own evaluation practices and those of the community at large.