Content-Based Recommendation

Content-based systems know the item before they know the user. Item representation comes first. The profile is assembled from what this user accepted and refused. The last step is the awkward one — taking corrections without freezing a passing mood into permanent taste.

Content-based systems encode items with metadata, text, image, audio, taxonomy, or learned embeddings. They then aggregate the items a user has interacted with into a preference representation. That is enough to recommend a new item before any collaborative feedback exists. The boundary is representational. The system can match only the properties its encoders and metadata make visible. Provider-generated content, missing attributes, and historical interaction bias all shape the profile.

A career platform represented jobs using titles and employer-written descriptions. Two roles shared keywords. They differed sharply in seniority, schedule, accessibility, and actual responsibilities. No scoring function could separate them, because the qualities that separated them were never handed to one.

Control and coverage trade against each other here. Curated metadata is inspectable and expensive to maintain. A learned content embedding scales to anything and can encode style signals nobody asked for. Structured relation features sit between the two. They earn their place where an item is too new for behaviour and too specific for a category.

The embedding column is not a thought experiment. Google published the design in 2019, and stated the substitution in its own abstract: “Apart from many content-aware systems based on matrix factorization, we consider a modeling framework using two-tower neural net, with one of the towers (item tower) encoding a wide variety of item content features.”

That framework retrieves personalised suggestions for YouTube, out of a corpus with tens of millions of videos. It was validated by offline experiments on two real-world datasets and by live A/B tests. What makes an item retrievable there is its content features, not its identifier. That is the property the middle and right-hand columns here are competing to supply.

Provider gaming comes from outside the team and semantic overclaim from inside it. Sellers stuff metadata to enter a profile. Engineers treat embedding proximity as proof that two items are interchangeable. Both end in a recommendation nobody can defend. Gaming is well past the hypothetical stage: the two largest app catalogs have written the prohibition, and the enforcement power, into their published rules.

Evaluate content-only performance on genuinely new items and new catalog regions. Not on items whose collaborative history leaked into pretraining or feature engineering. Compare curated attributes, learned embeddings, and hybrid variants under the same inventory. Measure coverage, duplication, novelty, profile correction, and failure under sparse or manipulated metadata. New-item recall is useful only if the items are eligible and the representation captures decision-relevant properties.

That demand is not a stylistic preference. There is a counted result behind it. Ferrari Dacrema and colleagues set out to reproduce the neural top-n recommenders the conferences had been publishing, and reported the attrition plainly: “Specifically, we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort.”

Of those 7, 6 could often be outperformed by comparably simple heuristics based on nearest-neighbour or graph-based techniques. The remaining one clearly beat the baselines. It did not consistently beat a well-tuned non-neural linear ranking method. The work appeared in 2019, at the same conference as the two-tower paper above. Read someone's new-item recall figure against that background before you spend a quarter on it.

An audit of the item features opens this build. A content model can match only what its encoders and metadata already expose. It closes on the explanations. Those are the part a user actually reads.

Step four is the step teams cut when the roadmap tightens, and there is a measurement attached to cutting it. In 2020 Anderson and colleagues scored listening diversity for over 100 million Spotify users, using a generalist–specialist score over a song embedding. Algorithm-driven listening was typically much less diverse than user-driven listening. The users who became more diverse did so by shifting away from algorithmic consumption.

Then the retention side: “Controlling for activity (the number of songs played), generalist users are up to 25 percentage points less likely to leave the platform and up to 35 percentage points more likely to become a paid subscriber.”

Reserving candidate slots for adjacent topics is not a concession to editorial taste. On that measurement it sits on the same side as churn and subscription.