Recommender System Quality Factors


  • Recommender System - Accuracy or RelevanceRecommender System - Accuracy or Relevance

    [[Being accurate is not enough - how accuracy metrics have hurt recommender systems]]

    Accuracy hurt recommender system

    #recommender-system #recommender-system/qual...
    • The recommendation suggested to a user should be relevant because that a user has a high propensity to purchase the recommended items and rate them highly.
    • Prediction accuracy is not enough
  • Recommender System - Novelty
    • the recommender system should provide users with options that are not already known to the users.
  • Recommender System - Diversity
    • A list of recommendations suggested to a user should be diverse to increase the chance of a conversion.
  • Recommender System - SerendipityRecommender System - Serendipity

    [[Beyond accuracy - evaluating recommender systems by coverage and serendipity]]

    Serendipity - Unexpected and usefulness (usefulness judged by user)

    [[Being accura...
    • Recommendations can help a user discover products that are unexpected and surprising, as well as a novel
  • Recommender System - CoverageRecommender System - Coverage

    [[Beyond accuracy - evaluating recommender systems by coverage and serendipity]]


    Prediction Coverage
    Catalogue Coverage

    • The percentage of users or items over which the system can make recommendations

  • Beyond-accuracy measures: Novelty, Diversity, Serendipity, and Coverage
    • Diversity, Serendipity, Novelty, and Coverage - A survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems
    • A survey and empirical analysis of beyond-accuracy objectives in recommender system
      • by quantifying diversity based on pair-wise item similarities
    • Rank and relevance in novelty and diversity metrics for recommender systems
      • by determining novelty based on item popularity
    • Avoiding monotony: Improving the diversity of recommendation list
      • Balance accuracy with other quality factors on a global level
    • Solving the apparent diversity-accuracy dilemma of the recommender system
      • Balance accuracy with other quality factors on a global level
    • Efficient optimization of multiple recommendation quality factors according to individual user tendencies
      • Balance accuracy with other quality factors on an individual level
    • Contextual bandits (Explore-Exploit problem)
      • Unbiased offline evaluation of contextual bandit based news article recommendation algorithms
        • Serve novel content (newly published articles in the news domain), then use the feedback to learn if this content should also be recommended to others
      • contextual combinatorial bandit and its application on diversified online recommendation
      • Learning diverse rankings with multi-armed bandits
      • Graph bandit for diverse user coverage in online recommendation
    • Recommender systems evaluation - A3D Benchmark
    • Improving recommendation lists through topic diversificationImproving recommendation lists through topic diversification

      Accuracy is not enough
      Introduce intra-list similarity metric to measure diversity which improves user satisfaction
      Propose topic diversification method to increase diversity

      The Intra-...
      • intra-list diversity
  • blockbuster effect (rich-get-richer)


  • Measuring the business value of recommender systemsMeasuring the business value of recommender systems
    What is being measured

    [[Click Through Rate]]

    Easy to measure and established
    Not the ultimate goal

    [[Adoption and Conversion Rates]]

    Easy to measure
  • Algorithmic MarketingAlgorithmic Marketing

    Promotions and advertisements
    Recommender Systems
    Pricing and assortment


    Algorithmic Marketing book - ILYA KATSOV

#recommender-system #recommender-system/quality #metrics