• Recommender System - Accuracy or RelevanceRecommender System - Accuracy or Relevance
  Supports
  
  
  [[Being accurate is not enough - how accuracy metrics have hurt recommender systems]]
  
  Accuracy hurt recommender system
  
  
  
  
  #recommender-system #recommender-system/qual...
  • The recommendation suggested to a user should be relevant because that a user has a high propensity to purchase the recommended items and rate them highly.
  • Prediction accuracy is not enough
• Recommender System - Novelty
  • the recommender system should provide users with options that are not already known to the users.
• Recommender System - Diversity
  • A list of recommendations suggested to a user should be diverse to increase the chance of a conversion.
• Recommender System - SerendipityRecommender System - Serendipity
  Supports
  
  
  [[Beyond accuracy - evaluating recommender systems by coverage and serendipity]]
  
  Serendipity - Unexpected and usefulness (usefulness judged by user)
  
  
  [[Being accura...
  • Recommendations can help a user discover products that are unexpected and surprising, as well as a novel
• Recommender System - CoverageRecommender System - Coverage
  Supports
  
  
  [[Beyond accuracy - evaluating recommender systems by coverage and serendipity]]
  
  Coverage
  
  Prediction Coverage
  Catalogue Coverage
  
  ...
  • The percentage of users or items over which the system can make recommendations

• Beyond-accuracy measures: Novelty, Diversity, Serendipity, and Coverage
  • Diversity, Serendipity, Novelty, and Coverage - A survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems
  • A survey and empirical analysis of beyond-accuracy objectives in recommender system
    • by quantifying diversity based on pair-wise item similarities
  • Rank and relevance in novelty and diversity metrics for recommender systems
    • by determining novelty based on item popularity
  • Avoiding monotony: Improving the diversity of recommendation list
    • Balance accuracy with other quality factors on a global level
  • Solving the apparent diversity-accuracy dilemma of the recommender system
    • Balance accuracy with other quality factors on a global level
  • Efficient optimization of multiple recommendation quality factors according to individual user tendencies
    • Balance accuracy with other quality factors on an individual level
  • Contextual bandits (Explore-Exploit problem)
    • Unbiased offline evaluation of contextual bandit based news article recommendation algorithms
      • Serve novel content (newly published articles in the news domain), then use the feedback to learn if this content should also be recommended to others
    • contextual combinatorial bandit and its application on diversified online recommendation
    • Learning diverse rankings with multi-armed bandits
    • Graph bandit for diverse user coverage in online recommendation
  • Recommender systems evaluation - A3D Benchmark
  • Improving recommendation lists through topic diversificationImproving recommendation lists through topic diversification
    
    Accuracy is not enough
    Introduce intra-list similarity metric to measure diversity which improves user satisfaction
    Propose topic diversification method to increase diversity
    
    
    
    The Intra-...
    • intra-list diversity
• blockbuster effect (rich-get-richer)

Reference

• Measuring the business value of recommender systemsMeasuring the business value of recommender systems
  What is being measured
  
  
  [[Click Through Rate]]
  
  Easy to measure and established
  Not the ultimate goal
  
  
  [[Adoption and Conversion Rates]]
  
  Easy to measure
  ...
• Algorithmic MarketingAlgorithmic Marketing
  
  Promotions and advertisements
  Search
  Recommender Systems
  Pricing and assortment
  
  
  
  
  Reference
  
  Algorithmic Marketing book - ILYA KATSOV
  
  

#recommender-system #recommender-system/quality #metrics