Mehdi Elahi, Himan Abdollahpouri, Masoud Mansoury and Helma Torkamaan


Problem: Fairness is one of the crucial aspects of modern Recommender Systems which has recently drawn substantial attention from the community. Many recent works have addressed this aspect by studying the fairness of the recommendation through different forms of evaluation methodologies and metrics. However, the majority of these works have mainly concentrated on the recommendation algorithms and hence measured the fairness from the algorithmic viewpoint. While such a viewpoint may still play an important role, it does not necessarily project a comprehensive picture of how the users may perceive the overall fairness of a recommender system.


Solution: This paper extends the prior works and goes beyond the al- algorithmic fairness in recommender systems by highlighting the non-algorithmic viewpoint on fairness in these systems. The paper proposes an evaluation methodology that can be used to assess the fairness of a recommender system perceived by its users. We have adopted a well-known model and re-formulated it to suit the particular characteristics of the recommender systems, and ac- accordingly, their corresponding users. Our proposed methodology can be used in order to elicit the feedback of the users, along with three important dimensions, i.e., Engagement, Representation, and Action & Expression. We have formed a set of survey questions that address the aforementioned dimensions, as a set of examples to assess the fairness in a recommender system.

Image From UMAP 2021 Fairness Measurement 27


Main contributions: 

  • We extend the notion of fairness in recommender systems to cover other aspects of the recommendations that are widely non-algorithmic.
  • We propose a novel framework based on a well-known the- theoretical foundation that can be used to evaluate different aspects of fairness in recommender systems.