Deep Ranking Models and Feature Interactions

Modern ranking models combine sparse IDs, dense features, embeddings, context, sequence summaries, and explicit crosses. Wide & Deep joins memorization with neural generalization. Deep & Cross Networks learn bounded-degree feature crosses. Others reach for attention or interaction layers. What a ranker buys with all of it is the pairwise and contextual reasoning a separate-tower retriever cannot compute at all.

That reasoning is affordable only because of the two-stage split. Google put numbers on the split in 2016, in its YouTube paper. Candidate generation winnows a corpus of millions down to hundreds. Only then does the ranker apply hundreds of features to each impression, roughly evenly split between categorical and continuous. The models learn approximately one billion parameters, trained on hundreds of billions of examples. The paper puts the trade plainly: “During ranking, we have access to many more features describing the video and the user's relationship to the video because only a few hundred videos are being scored rather than the millions scored in candidate generation.” The stage in front is what buys the budget for interaction.

The same paper cautions that live A/B results are not always correlated with offline experiments. That is the other half of the constraint. Capacity pays only when the candidate set, the feature freshness, the latency budget and the evaluation all support it.

Work the failure through on a hypothetical before reading the documented ones below. A high-cardinality merchant embedding sits inside a ranker that also sees user, item, price, context, and seller features. It captures historical promotion privileges and dominates genuine product relevance. Each path in the architecture offers a different way for that to happen. The published cases in this lesson give each of them a measured counterpart.

Memorization, generalization, and interaction by construction. The wide component remembers combinations it has already seen. The deep component generalizes to ones it has not. A cross network builds interactions of bounded order deliberately, rather than hoping depth discovers them.

The cross network has a published price list. Deep & Cross Network was tested in 2017 on the Criteo Display Ads data: 11 GB of user logs over 7 days, roughly 41 million records, 13 integer and 26 categorical features. Best test logloss was 0.4419 for DCN. Deep Crossing scored 0.4425, a plain DNN 0.4428, a factorization machine 0.4464 and logistic regression 0.4474. The authors state that on this dataset a 0.001 logloss improvement is practically significant. Their margin over the DNN is 0.0009, just under their own bar. That is exactly why the memory line matters: “In particular, it outperforms the state-of-art DNN model but uses only 40% of the memory consumed in DNN.” Efficient interaction modeling is a claim with a number behind it, not an adjective.

The successor ran the cost-matched ablation on a production system. DCN-V2, published in 2021, reports from a Google production ranker trained on hundreds of billions of examples: “When compared with production model, DCN-V2 yielded 0.6% AUCLoss (1 - AUC) improvement. For this particular model, a gain of 0.1% on AUCLoss is considered a significant improvement.” The paper then swaps the cross layers for same-sized ReLU layers and prices the difference: -0.15% for 2 ReLU layers against -0.45% for 2 DCN-V2 cross layers. Same parameter budget, different structure, published margin.

Identity and history embeddings, context, cross-features and multi-task heads are the easy part of this stack. The serving contract at the bottom is the constraint: every value has to arrive point-in-time, inside a latency budget, with a defined answer for the moment it does not arrive at all.

That contract has a published price. The Wide & Deep paper prints it instead of asserting it: “At peak traffic, our recommender servers score over 10 million apps per second. With single threading, scoring all candidates in a single batch takes 31 ms. We implemented multithreading and split each batch into smaller sizes, which significantly reduced the client-side latency to 14 ms (including serving overhead).” The engineering that bought the model its place in production was batch splitting across threads, not a new interaction layer. That is the level at which a latency budget is real: 31 ms of single-threaded scoring, 14 ms after the fix, at over 10 million scores per second.

Linear and tree baselines set the number a deep ranker has to beat. The last step prices what beating it costs, in feature access, tail latency, fallback, and rollback. Everything between the two — ablating ID, content, context and aggregate feature families, validating point-in-time joins, measuring calibration and slices — is an argument about whether the extra capacity earned its place.

Step 1 exists because tuned simple baselines win more often than the literature suggests. In 2020 Rendle and three colleagues re-ran the neural collaborative filtering benchmark on the published Movielens-1M and Pinterest splits. A properly tuned dot-product matrix factorization beat the MLP-based NeuMF on every metric. On Movielens: HR@10 0.7294 against 0.7093, NDCG@10 0.4523 against 0.4349. On Pinterest: HR@10 0.8895 against 0.8777, NDCG@10 0.5794 against 0.5576. Their abstract states it without hedging: “First, we show that with a proper hyperparameter selection, a simple dot product substantially outperforms the proposed learned similarities.” Those NeuMF figures are the non-cherry-picked results, reproduced by the independent meta-study of Dacrema and colleagues. The numbers the original NCF paper reported for NeuMF and MLP were cherry-picked in the neural model's favour: the metrics were taken from the best iteration, selected on the test set. The matrix factorization baseline beat even those.

Step 2 has a published template. The DCN-V2 ablation replaces cross layers with equal-parameter ReLU layers. It reports -0.15% for 2 ReLU layers against -0.45% for 2 DCN-V2 cross layers, on a production ranker where 0.1% AUCLoss is the team's own significance threshold. Step 5 has one too: 31 ms of single-threaded batch scoring reduced to 14 ms of client-side latency, at over 10 million apps scored per second at peak.