The following tutorials will be given during the UMAP2014:

More details about individual tutorials and authors can be found below.

User Affect and Sentiment Modelling

Björn W. Schuller - Imperial College London / TU München

Date: 7th of July 2014 (morning session)

Abstract: Affect and Sentiment Recognition have matured to the degree where they are ready for real-world application. At the same time, the interest in such technology has tremendously grown over the last decade, and expectancy is nothing short of a potential to drastically change our way we interact with technology. Affect and emotion are omni-present - be it if a system does not react in the way we want it to, or - in the better case - because it does. In addition, users may be affected by other influences and humans certainly take such cues into account when interacting. Accordingly, there is great interest in having future interactive systems do so as well to make communication with them feel more natural, sensitive, and "intelligent". Modelling of "affect" thereby comprises an increasingly broader range of mental states beyond the "big six" basic emotions. Today's approaches target increasingly more subtle affective states including also social emotions or mental states such as cognitive and physical load, intoxication, pain, or sleepiness, just to name a few. Models to represent affect are various, and dimensional modelling is increasingly more used rather than looking at a closed inventory of discrete labels. Sentiment can be considered as one of these dimensions, highly related to valence, but usually in connection with an object or target the sentiment is directed at.

This tutorial introduces the principles, methods, and state-of-the-art in user affect and sentiment modelling focussing first on speech and text, but also including facial expression, body gestures, physiological sensors, interaction patterns and context. It further introduces typical databases, tools, and benchmarks in the field, and touches upon use-cases and examples of affect-aware systems and their engineering. Emphasis is thereby laid on modelling and recognition "in the wild" and independent of the user.

In detail, it will guide through the following parts:

  • Motivation for User Affect and Sentiment Modelling
  • Affect and Sentiment Modelling
    • Classes
    • Dimensions
    • Other representation forms
  • Affect and Sentiment Data & Benchmarks
    • Database creation
    • Monomodal databases
    • Multimodal databases
    • Benchmark tests and challenges
  • Affect and Sentiment Recognition
    • Modalities
    • Pre-processing and de-noising
    • Features and selection
    • Classification and regression
    • Autonomous learning and adaptation
  • Affective Output and Feedback
    • Affect synthesis
    • Feedback generation
  • System integration aspects
    • Confidence measures
    • Context integration
    • Distributed processing
    • Encoding
  • Tools
    • Labelling Toolkits
    • Feature Extraction Toolkits
    • Learning Toolkits
  • Use-Cases
  • The (likely) "next big things" in the field


Björn W. Schuller received his diploma in 1999, his doctoral degree for his study on Automatic Speech and Emotion Recognition in 2006, and his habilitation in 2012 all in electrical engineering and information technology from TUM/Germany.

He is a Senior Lecturer in Imperial College London's Machine Learning Group in London/UK (since 2013) and a tenured faculty member heading the Machine Intelligence and Signal Processing Group at TUM’s Institute for Human-Machine Communication since 2006 as well as CEO of audEERING UG (limited). He is also a permanent Visiting Professor at the Harbin Institute of Technology, Harbin/P.R. China. Previously, he was heading the Institute for Sensor Systems as full professor at the University of Passau/Germany, a Visiting Professor at the Université de Genève/Switzerland in the Centre Interfacultaire en Sciences Affectives remaining an appointed associate of the institute, with JOANNEUM RESEARCH, in Graz/Austria, remaining an expert consultant, and with the CNRS-LIMSI Spoken Language Processing Group in Orsay/France among others. Best known are his works advancing Affective Computing.

Dr. Schuller is president of the Association for the Advancement of Affective Computing (AAAC, former HUMAINE Association), elected member of the IEEE Speech and Language Processing Technical Committee, and (co-)authored 5 books and more than 400 publications in the field (5800+ citations, h-index = 39). He was co-founding member and secretary of the steering committee and guest editor, and still serves as associate editor of the IEEE Transactions on Affective Computing, is associate editor for the Computer Speech and Language, associate editor IEEE Signal Processing Letters, IEEE Transactions on Cybernetics, and the IEEE Transactions on Neural Networks and Learning Systems, and guest editor for the IEEE Intelligent Systems Magazine, Neural Networks, Speech Communication, Image and Vision Computing, Cognitive Computation, and the EURASIP Journal on Advances in Signal Processing. Further he is/was co-general chair of ACM ICMI 2014, program chair of the ACM ICMI 2013, IEEE SocialCom 2012, and ACII 2011, and workshop and challenge organizer including the INTERSPEECH 2009-2014 annual Computational Paralinguistics Challenges and 2011-2014 Audio/Visual Emotion Challenges and Workshops. Advisory board activities comprise his membership as invited expert in the W3C Emotion Markup Language Incubator Groups.


Social Information Access

Peter Brusilovsky - University of Pittsburgh

Date: 11th of July 2014 (afternoon session)

Abstract:  The power of the modern Web, which is frequently called the Social Web or Web 2.0, is frequently traced to the power of users as contributors of various kinds of contents through Wikis, blogs, and resource sharing sites. However, the community power impacts not only the production of Web content, but also the access to all kinds of Web content. A number of research groups worldwide explore what we call social information access techniques that help users get to the right information using “collective wisdom” distilled from actions of those who worked with this information earlier.

Social information access can be formally defined as a stream of research that explores methods for organizing users' past interaction with an information system (known as explicit and implicit feedback), in order to provide better access to information to the future users of the system. It covers a range of rather different systems and technologies from social navigation to collaborative filtering.  An important feature of all social information access systems is self-organization. Social information access systems are able to work with little or no involvement of human indexers, organizers, or other kinds of experts. They are truly powered by a community of users. Due to this feature, social information access technologies are frequently considered as an alternative to the traditional (content-oriented) technologies. The goal of this tutorial is to provide an overview of the emerging social information access research stream and to provide some practical guidelines for building social information access systems.

Peter Brusilovsky is a Professor of Information Science and Intelligent Systems at the University of Pittsburgh, where he directs Personalized Adaptive Web Systems (PAWS) lab. Peter has been working in the field of adaptive educational systems, user modeling, and intelligent user interfaces for over 20 years. He published numerous papers and edited several books on adaptive hypermedia and the adaptive Web. He was holding visiting faculty appointments at the Moscow State University (Russia), Sussex University (UK), Tokyo Denki University (Japan), University of Trier (Germany), Free University of Bolzano (Italy), National College of Ireland, and Carnegie Mellon University. Peter is the Associate Editor-in-Chief of IEEE Transactions on Learning Technologies and a board member of several journals including User Modeling and User Adapted Interaction, ACM Transactions on the Web, and Web Intelligence and Agent Systems. He is  also the current President of User Modeling Inc., a professional association of user modeling researchers.


Personalization for behaviour change

Julita Vassileva - University of Saskatchewan

Judith Masthoff - University of Aberdeen

Date: 11th of July 2014 (morning session)

Abstract: Digital behaviour intervention is a growing area of research which investigates how interactive systems can encourage and support people to change their behaviour, for their own or communal benefits. Personalization plays an important role in this, as the most effective persuasive and motivational strategies are likely to depend on user characteristics such as the user’s personality, affective state, existing attitudes, behaviours, knowledge, and goals. Example application areas include healthcare (e.g., encouraging people to eat more healthily and exercise more), education (e.g., motivating learners to study more), environment (e.g., encouraging people to use less energy and more public transport), and collaborative content development (e.g., incentivising people to annotate resources, participate online). This tutorial will cover the role of personalization in behaviour change technology, and methods and techniques to design personalized behaviour change technology. The tutorial will include both traditional and more recent approaches (such as gamification). It will be highly interactive, with short interactive lectures, exercises and mini-experiments.

Julita Vassileva is a professor of Computer Science at the University of Saskatchewan, Canada. Her research areas involve human issues in decentralized software environments: user modeling and personalization, and in designing incentive mechanisms for encouraging participation and facilitating trust in web, cloud and mobile applications. She is interested in how to use personalized recommendations, games and social influence to support people in their learning and engaging in beneficial behaviours. She serves on the Editorial Boards of User Modelling and User Adapted Interaction, Computational Intelligence and IEEE Transactions on Learning Technologies. She has served on the PC of Persuasive since 2006.

Judith Masthoff is an associate professor of Computing Science at the University of Aberdeen, UK. Her research is in personalisation and intelligent user interfaces. She is interested in personalizing behavior change mechanisms for encouraging people to live more healthily and sustainably, and in adapting motivating and emotional support messages to personality. She has co-organized four workshops on behavior change technology. She serves on the editorial board of User Modeling and User-Adapted Interaction, and is current guest-editing a special issue on Personalization and Behavior Change. She was program chair of the UMAP conference in 2012 and is a director of User Modeling Inc., the professional association of user modeling researchers. She has served on the PC of Persuasive since 2006.