Recommender systems
Recommender Systems as Decision Pipelines
Build a systems-level map of recommendation from eligible inventory and candidate generation to ranking, presentation, feedback, and long-term effects.
By the end you can
- Explain why a recommender system is a decision pipeline rather than a single prediction model
- Distinguish prediction, ranking, re-ranking, presentation, and policy execution
- Identify stage-specific evidence and failure modes in a recommendation surface
- Design a rollback-aware recommendation decision before selecting an algorithm
Example
Bing shipped a bug and both tracked metrics went up
A bug shipped into a Bing experiment and made the search results very poor. Both of the metrics the organisation watched went up. Microsoft's experimentation team published the episode in 2012, in its own words: “When Bing had a bug in an experiment, which resulted in very poor results being shown to users, two key organizational metrics improved significantly: distinct queries per user went up over 10%, and revenue per user went up over 30%!”
The mechanism is not mysterious. Bad results make people search again. Searching again means more queries typed and more ads clicked. Kohavi and his co-authors give the episode as the reason a short-term proxy cannot be the Overall Evaluation Criterion. The two numbers rose through the failure, not despite it. The system had got worse. Everything the team measured said better.
- Surface: Bing chooses which results a query returns. In this experiment a bug made those results very poor.
- Proxy: The two key organizational metrics were distinct queries per user and revenue per user. Fast, abundant, and both up — over 10% and over 30%.
- Decision chain: The bug travelled the whole chain before it reached a metric. Worse results changed what users saw. What they saw changed what they did. What they did was all the logs recorded.
- Delayed consequence: Neither number reported the thing that had got worse. The improvement was recorded first. The diagnosis came from knowing a bug had shipped, not from the metrics themselves.
- System lesson: A short-term proxy cannot serve as the Overall Evaluation Criterion. The system is a policy that allocates attention, not merely a predictor of clicks.
Recommendation is a sequence of constrained decisions
A recommender system chooses which items become visible, in what order, under which context, and with what opportunity for action. Netflix put a size on that choice in 2015. Gomez-Uribe and Hunt, who ran the system, wrote: “Our recommender system is used on most screens of the Netflix product beyond the homepage, and in total influences choice for about 80% of hours streamed at Netflix.” Two of every three streamed hours are discovered on the homepage, across roughly 40 rows of up to 75 videos each. They put personalization and recommendations together at more than $1B per year.
The model is one component inside that pipeline. Inventory rules, retrieval, ranking, re-ranking, interface design, experimentation, logging and governance are the rest of it. So the central design question is not “Which algorithm should we use?” It is “Which decision should change, for whom, under what evidence, and with which safeguards?”
Swapping the model changes almost nothing if nobody has named the decision, the evidence behind it, and the safeguards around it.
Case
Two stages at YouTube, 1,500 candidates at X
Two published systems make the shape concrete. YouTube's recommender comes in two pieces. Google described it in 2016 as “split according to the classic two-stage information retrieval dichotomy: first, we detail a deep candidate generation model and then describe a separate deep ranking model”.
X released its home timeline code on 31 March 2023, and the shape is the same. Roughly 1,500 candidate posts are pulled “from a pool of hundreds of millions”. Half come from the search index — “~50% of posts come from this candidate source” — and the rest from the user-tweet-entity-graph, tweet-mixer and follow-recommendation-service. A “~48-million-parameter neural network” then orders them, showing on average 50% in-network and 50% out-of-network posts.
Filtering is a stage of its own there. Visibility filtering is “Responsible for filtering X content to support legal compliance, improve product quality, increase user trust, protect revenue through the use of hard-filtering, visible product treatments, and coarse-grained downranking”. Retrieval is a design decision too, not a given. Facebook's search team published embedding-based retrieval in 2020. Until then, “Facebook search was still mainly based on a Boolean matching model”. Their paper describes “the system to serve embedding-based retrieval in a typical search system based on an inverted index”.
The model is one box in that chain.
Analogy
A theatre program is more than critic scores
A festival director assembles one evening from hundreds of eligible performances. Reviews help. Scheduling, venue capacity, audience mix, contractual rules and the order of the program determine the actual experience. A recommender also turns many signals into a constrained slate.
Then the analogy breaks, and the break is the useful part. Next year's reviews arrive whatever this festival programmed. Tomorrow's interaction data arrives only for the items this slate made visible. A 2018 paper gave that asymmetry a name — algorithmic confounding — and a simulation to go with it. Its abstract states the problem plainly: “These systems are often evaluated or trained with data from users already exposed to algorithmic recommendations; this creates a pernicious feedback loop.” The authors simulated recommenders trained on data that earlier recommendations had generated. The confounded loop homogenises user behaviour, with no gain in utility.
A critic reviews the performances that existed. A recommender only ever sees the ones it chose to stage.
Recommendation quality emerges from the whole program, not from a score in isolation.
Visual
The recommendation control loop
Five boxes, and the model is only one of them. Eligible inventory settles what may be shown at all. At X that box holds visibility filtering, for legal compliance and product quality, standing outside the ranker. Candidate generation narrows the corpus, and X keeps roughly 1,500 candidates out of hundreds of millions. That is how aggressive the narrowing is. Presentation is not a neutral window either. Netflix's homepage lays out roughly 40 rows of up to 75 videos each, and two of every three streamed hours are discovered there. Feedback and learning settles what the next model will believe. Everything teams argue about sits between the first box and the last.
- 1
Eligible inventory
Apply availability, policy, geographic, contractual, and safety constraints before personalization.
- 2
Candidate generation
Retrieve a manageable set with complementary sources and freshness rules.
- 3
Ranking and re-ranking
Estimate utility, combine objectives, and construct an ordered slate.
- 4
Presentation
Place cards, rows, explanations, and controls in an interface that changes behavior.
- 5
Feedback and learning
Log exposure and outcomes, evaluate policy effects, and update only under controlled evidence.
Comparison
Predictor, ranker, and decision policy are not synonyms
The three words get used interchangeably in meetings, and they name three different objects. A predictor produces a number. A ranker produces an order. Only the decision policy settles what anyone actually sees — which is why it is the one that creates tomorrow's training data.
The middle box is not one predictor either. Google's published account of YouTube's next-video ranker, from 2019, opens on the difficulty: “The system faces many real-world challenges, including the presence of multiple competing ranking objectives, as well as implicit selection biases in user feedback.” The system runs Multi-gate Mixture-of-Experts across those competing objectives. It adds a Wide & Deep framework to model and remove the selection bias baked into logged feedback. A deployed “ranker” therefore already holds several objectives in tension. It also carries a correction for the bias its own past exposure created.
Predictor
Estimates an outcome such as click, watch time, purchase, or satisfaction.
- Produces a score or distribution
- Can be evaluated without choosing a final slate
- May ignore inventory and capacity
- Example: probability of completing a course
Ranker
Orders candidates according to one or more modeled objectives.
- Consumes candidate features
- Optimizes relative placement
- Interacts with position and page design
- Example: ordering ten eligible courses
Decision policy
Combines scores with rules, constraints, exploration, and user controls.
- Determines actual exposure
- Allocates scarce attention
- Creates future training data
- Example: rank, diversify, cap repetition, and abstain
Example
Failure modes that survive a good model
Every failure below survives a model that scores well offline. One of them has been watched directly. In 2005 Joachims and four colleagues put an eye-tracker on people using a search engine. Phase I recruited 34 participants, and 29 of them gave usable eye-movement data. Phase II recruited 22, of whom 16 gave usable data: 6 normal, 5 swapped, 5 reversed. The finding fits in one sentence: “It is very interesting that users click substantially more often on the first than on the second link, while they view the corresponding abstract with almost equal frequency.” People looked at both results. They clicked the top one. The study calls that a trust bias, and names a quality bias alongside it.
The same study shows what does survive the presentation. Relative preferences extracted with Click > Skip Above agreed with human relevance judges 80.8% of the time. The ceiling was 89.5%, set by agreement between the judges themselves. Clicks are not worthless. They are worthless read as absolute votes.
Objective mismatch and operational blindness are the two worth separating in a review. One means the system is optimizing the wrong thing. The other means nobody can tell afterwards what it optimized.
- Objective mismatch: The system predicts the chosen proxy accurately while degrading the intended outcome. On very poor results, Bing's distinct queries per user rose over 10% and revenue per user over 30%.
- Candidate omission: A strong ranker never sees relevant items because retrieval removed them. Whatever X's candidate sources miss out of hundreds of millions is absent from the roughly 1,500 that reach the neural ranker.
- Presentation confounding: A card receives clicks because of position or artwork rather than item value. Rank 1 was clicked substantially more often than rank 2, while both abstracts were viewed with almost equal frequency.
- Feedback amplification: Exposure creates interactions that make already-visible items appear increasingly preferable. The simulated confounded loop raises homogeneity without raising utility.
- Operational blindness: The team cannot reconstruct which version, rule, and inventory state produced a slate.
Steps
Design a recommendation decision before choosing a model
Name the intervention before any modeling. Set the rollback conditions after all of it. The three steps in between turn a recommendation into a decision someone else can review. Define value and harm across immediate, delayed, user, provider, platform and societal outcomes. Draw the pipeline, and mark where eligibility, retrieval, ranking, re-ranking, presentation and feedback begin and end. Then assign offline tests, online experiments, guardrails and qualitative review to each stage.
Step 5 is not only prudence. In the EU it is law. Very large online platforms and very large online search engines must “provide at least one option for each of their recommender systems which is not based on profiling”. That is Article 38 of the Digital Services Act, Regulation (EU) 2022/2065, adopted on 19 October 2022. Article 27 separately obliges them to set out the main ranking parameters in plain and intelligible language. TikTok announced the switch-off on 4 August 2023, ahead of the 28 August 2023 deadline: “As part of our efforts to meet DSA requirements, we will soon be giving our European community another way to discover content on TikTok by allowing them to turn off personalisation.”
The non-personalized fallback is a shipped surface with a date on it, not a paragraph in a design document.
1. Name the intervention
Specify the surface, user action, timing, and number of items the system will choose.
2. Define value and harm
List immediate, delayed, user, provider, platform, and societal outcomes.
3. Draw the pipeline
Mark eligibility, retrieval, ranking, re-ranking, presentation, and feedback boundaries.
4. Assign evidence
Choose offline tests, online experiments, guardrails, and qualitative review for each stage.
5. Set rollback conditions
State when the policy must fall back to a simpler or non-personalized experience.
Key idea
A high offline score cannot rescue a wrong objective
A high offline score cannot rescue an objective that rewards the wrong user behavior or ignores the rest of the decision pipeline. Two published results say so from opposite directions.
From inside a production recommender, the Netflix paper admits the limit: “Thus, while we do rely on offline experiments heavily, for lack of a better option, to decide when to A/B test a new algorithm and which new algorithms to test, we do not find them to be as highly predictive of A/B test outcomes as we would like.” The offline setup has to assume members would have engaged with the product in exactly the same way had the new algorithm served the recommendations. Gomez-Uribe and Hunt flag that assumption as not always true.
From outside, a 2019 audit went after the offline literature itself. Its authors took 18 neural recommendation algorithms presented at top-level research conferences and tried to run them. “Only 7 of them could be reproduced with reasonable effort.” Of those 7, 6 could often be outperformed by simple nearest-neighbour or graph-based heuristics. The published scores were real. What they measured was the open question.
Offline evaluation ranks models against the objective it was handed; nothing in that score audits the handoff itself.
Key idea
The first release question
Can the team explain what the system chooses, which outcomes justify that choice, and how it will stop when evidence turns against it? Bing's experiment answers the first and fails the second. The system chose results. The outcomes that justified it moved over 10% and over 30% in the wrong direction of quality. Article 38 of the Digital Services Act answers the third for very large online platforms and very large online search engines, by making the non-profiling option compulsory rather than optional. The first two questions are about the system. The third is about the team.
Ship when those three answers exist in writing; the team that improvises them after launch is improvising under pressure.
Key takeaways
- A recommender is trustworthy only when its complete decision policy can be measured, challenged and reversed. Under Article 38 of the Digital Services Act, the reversal has to be a shipped, non-profiling option, not an intention.
- A recommender system chooses which items become visible, in what order, under which context, and with what opportunity for action. At Netflix that choice influences about 80% of hours streamed, and two of every three streamed hours are discovered on one page.
- A high offline score cannot rescue an objective that rewards the wrong user behavior or ignores the rest of the decision pipeline. Netflix reports its offline experiments are not as highly predictive of A/B test outcomes as it would like, and a 2019 audit of 18 neural algorithms could reproduce only 7 with reasonable effort.
- Availability, policy, geographic, contractual and safety constraints apply before personalization. X keeps visibility filtering as a stage of its own, for legal compliance and product quality, around the ranker rather than inside it.
- Objective mismatch remains the everyday risk: the system predicts the chosen proxy accurately while degrading the intended outcome. A Bing bug served very poor results, and distinct queries per user rose over 10% while revenue per user rose over 30%.
- Name the conditions that force a fall back to a simpler or non-personalized experience in the design of the policy, not in the incident that exposes it. TikTok announced its personalisation switch-off on 4 August 2023, ahead of the 28 August 2023 deadline.