You can't Google your way to a recommender system

Schaun Wheeler and DJ Rich delve into the intricacies of building recommender systems in this podcast hosted by Arpit Choudhury. The discussion highlights the steps to developing a recommender system, practical advice for startups, and the evolving landscape of recommender technologies. 

---

KEY POINTS: 

1. Identify the Problem: Recommender systems address the "discovery problem" by helping users sift through vast amounts of options to find relevant content quickly. Recognizing this problem is crucial before diving into solutions.

2. System Components: Recommender systems are complex and involve multiple components such as:

   • Item Inventory: Detailed metadata about items (e.g., descriptions, categories).

   • User Interaction History: Data on user interactions with items (e.g., views, purchases).

   • Recommendation Model: The core model that filters and ranks items based on user preferences.

   • The Learner: An important component, which trains the model and separates it from the model's deployment phase.

3. Build vs. Buy: Should one build a recommender system from scratch or use existing solutions? For many startups, buying an off-the-shelf system can be more practical due to advances in data infrastructure and the complexity of developing a bespoke system. Buying a system can also free up resources for other critical areas.

4. Practical Recommendations for Startups: Instead of getting bogged down by complex models initially, startups are encouraged to start with simpler models and leverage existing infrastructure to implement a functional recommender system.

5. Innovations in Recommender Systems: Schaun is interested in combining traditional methods with reinforcement learning to enhance system performance. DJ is excited about research addressing causal questions and handling sequential recommendations.

---

REFERENCES:

• "Are we really making progress?" This paper is a replication study on recommender algorithms and shows that many DL approaches couldn't be reproduced or could be beaten with linear methods.

• "Deep Exploration for Recommender Systems" This paper talks about sequential decisioning for RSs (where you consider more than just one item recommendation).

• "Two Decades of Recommender Systems at Amazon" This paper is a retrospective on what's work well at Amazon.

---