UMUAI Journal

Latest Results for User Modeling and User-Adapted Interaction

Deprecated: DateTime::__construct(): Passing null to parameter #1 ($datetime) of type string is deprecated in /home/umorg/public_html/libraries/src/Date/Date.php on line 126
06 December 2023

• Solving the imbalanced data issue: automatic urgency detection for instructor assistance in MOOC discussion forums

  Abstract

  In MOOCs, identifying urgent comments on discussion forums is an ongoing challenge. Whilst urgent comments require immediate reactions from instructors, to improve interaction with their learners, and potentially reducing drop-out rates—the task is difficult, as truly urgent comments are rare. From a data analytics perspective, this represents a highly unbalanced (sparse) dataset. Here, we aim to automate the urgent comments identification process, based on fine-grained learner modelling—to be used for automatic recommendations to instructors. To showcase and compare these models, we apply them to the first gold standard dataset forUrgent iNstructorInTErvention (UNITE), which we created by labelling FutureLearn MOOC data. We implement both benchmark shallow classifiers and deep learning. Importantly, we not only compare, for the first time for the unbalanced problem, several data balancing techniques, comprising text augmentation, text augmentation with undersampling, and undersampling, but also propose several new pipelines for combining different augmenters for text augmentation. Results show that models with undersampling can predict most urgent cases; and 3X augmentation + undersampling usually attains the best performance. We additionally validate the best models via a generic benchmark dataset (Stanford). As a case study, we showcase how the naïve Bayes with count vector can adaptively support instructors in answering learner questions/comments, potentially saving time or increasing efficiency in supporting learners. Finally, we show that the errors from the classifier mirrors the disagreements between annotators. Thus, our proposed algorithms perform at least as well as a ‘super-diligent’ human instructor (with the time to consider all comments).

• What influences users to provide explicit feedback? A case of food delivery recommenders

  Abstract

  Although various forms of explicit feedback such as ratings and reviews are important for recommenders, they are notoriously difficult to collect. However, beyond attributing these difficulties to user effort, we know surprisingly little about user motivations. Here, we provide a behavioral account of explicit feedback’s sparsity problem by modeling a range of constructs on the rating and review intentions of US food delivery platform users, using data collected from a structured survey (n = 796). Our model, combining the Technology Acceptance Model and Theory of Planned Behavior, revealed that standard industry practices for feedback collection appear misaligned with key psychological influences of behavioral intentions. Most notably, rating and review intentions were most influenced by subjective norms. This means that while most systems directly request feedback in user-to-provider relationships, eliciting them through social ties that manifest in user-to-user relationships is likely more effective. Secondly, our hypothesized dimensions of feedback’s perceived usefulness recorded insubstantial effect sizes on feedback intentions. These findings offered clues for practitioners to improve the connection between providing behaviors and recommendation benefits through contextualized messaging. In addition, perceived pressure and users’ high stated ability to provide feedback recorded insignificant effects, suggesting that frequent feedback requests may be ineffective. Lastly, privacy concerns recorded insignificant effects, hinting that the personalization-privacy paradox might not apply to preference information such as ratings and reviews. Our results provide a novel understanding of explicit feedback intentions to improve feedback collection in food delivery and beyond.

• Leveraging response times in learning environments: opportunities and challenges

  Abstract

  Computer-based learning environments can easily collect student response times. These can be used for multiple purposes, such as modeling student knowledge and affect, domain modeling, and cheating detection. However, to fully leverage them, it is essential to understand the properties of response times and associated caveats. In this study, we delve into the properties of response time distributions, including the influence of aberrant student behavior on response times. We then provide an overview of modeling approaches that use response times and discuss potential applications of response times for guiding the adaptive behavior of learning environments.

• Clustering of conversational bandits with posterior sampling for user preference learning and elicitation

  Abstract

  Conversational recommender systems elicit user preference via conversational interactions. By introducing conversational key-terms, existing conversational recommenders can effectively reduce the need for extensive exploration required by a traditional interactive recommender. However, there are still limitations of existing conversational recommender approaches eliciting user preference via key-terms. First, the key-term data of the items needs to be carefully labeled, which requires a lot of human efforts. Second, the number of the human labeled key-terms is limited and the granularity of the key-terms is fixed, while the elicited user preference is usually from coarse-grained to fine-grained during the conversations. In this paper, we propose a clustering of conversational bandits algorithm. To avoid the human labeling efforts and automatically learn the key-terms with the proper granularity, we online cluster the items and generate meaningful key-terms for the items during the conversational interactions. Our algorithm is general and can also be used in the user clustering when the feedback from multiple users is available, which further leads to more accurate learning and generations of conversational key-terms. Moreover, to learn the user clustering structure more efficiently in more complex user clustering structure, we further propose a simple yet effective soft user clustering module to perform exploration on user clustering via sampling the posterior user representations. We analyze the regret bound of our learning algorithm. In the empirical evaluations, without using any human labeled key-terms, our algorithm effectively generates meaningful coarse-to-fine grained key-terms and performs as well as or better than the state-of-the-art baseline.

• A novel group recommender system for domain-independent decision support customizing a grouping genetic algorithm

  Abstract

  Group formation is a complex task requiring computational support to succeed. In the literature, there has been considerable effort in the development of algorithms for composing groups as well as their evaluation. The most widely used approach is the Genetic Algorithm, as, it can handle numerous variables, generating optimal solutions according to the problem requirements. In this study, a novel genetic algorithm was developed for forming groups using innovative genetic operators, such as a modification of 1-point and 2-point crossover, the gene and the group crossover, to improve its performance and accuracy. Moreover, the proposed algorithm can be characterized as domain-independent, as it allows any input regardless of the domain problem; i.e., whether the groups concern objects, items or people, or whether the field of application is industry, education, healthcare, etc. The grouping genetic algorithm has been evaluated using a dataset from the literature in terms of its settings, showing that the tournament selection is better to be chosen when a quick solution is required, while the introduced gene and group crossover operators are superior to the classic ones. Furthermore, the combination of up to three crossover operators is ideal solution concerning algorithm’s accuracy and execution time. The effectiveness of the algorithm was tested in two grouping cases based on its acceptability. Both the students participated in forming collaborative groups and the professors participated in evaluating the groups of courses created were highly satisfied with the results. The contribution of this research is that it can help the stakeholders achieve an effective grouping using the presented genetic algorithm. In essence, they have the flexibility to execute the genetic algorithm in different contexts as many times as they want until to succeed the preferred output by choosing the number of operators for either greater accuracy or reduced execution time.