Recommender systems
Graph Neural Recommenders and LightGCN
Understand graph convolution for recommendation, LightGCN simplification, layer aggregation, oversmoothing, sampling, and fair baselines.
By the end you can
- Explain neighborhood propagation and layer aggregation in graph recommenders
- Describe the simplifications introduced by LightGCN
- Identify oversmoothing, degree bias, leakage, and baseline weaknesses
- Design depth, degree, sampler, and compute-matched ablations
Example
The variant without layer combination peaked at two hops and was worst at four
Gowalla is a real user-item graph: 29,858 users, 40,981 items, and 1,027,370 interactions joining them. Density 0.00084. Almost every cell is empty. It is one of the three datasets behind the LightGCN experiment, published in 2020.
The heavier design was not the one that paid. LightGCN-PyTorch, an unofficial third-party reimplementation, runs the model at one, two, three and four layers with the seed fixed to 2020. Top-20 recall on Gowalla comes out at 0.1687, 0.1786, 0.1824, 0.1825. The curve flattens by the third hop. It never climbs again. The model that stayed is the one defined by what it removed.
- Message passing: User and item embeddings absorb signal from their interaction neighbors. On Gowalla that is 1,027,370 interactions linking 29,858 users to 40,981 items, at density 0.00084.
- Layer depth: Extra hops mix wider collaborative neighborhoods for less and less. In the released reimplementation the fourth hop moves top-20 recall from 0.1824 to 0.1825. Full LightGCN, which combines embeddings from several depths, does not degrade at 4 layers.
- Architecture excess: The abstract says it flatly: “the two most common designs in GCNs -- feature transformation and nonlinear activation -- contribute little to the performance of collaborative filtering”. In pure ID-based collaborative filtering there are no informative node features for those components to feed on.
- Oversmoothing: LightGCN-single reads out only the final propagation layer instead of combining layers. It peaks at 2 layers and is at its worst at 4.
- Benchmark lesson: A model built by deleting feature transformation and nonlinear activation, not by adding anything, is the comparator later graph work has to beat.
Comparison
NGCF-style complexity and LightGCN-style simplicity, as an ablation that was actually run
Read this row left to right and it is an ablation. The ablation was actually run. Before proposing LightGCN at all, its authors took NGCF at 2 layers, held NGCF's optimal hyper-parameters fixed, and started cutting. NGCF-f lost the feature transformation matrices. NGCF-n lost the nonlinear activation. NGCF-fn lost both. Removing the nonlinearity alone barely moved accuracy. Removing the pair did this: “As a whole, feature transformation and nonlinear activation impose rather negative effect on NGCF, since by removing them simultaneously, NGCF-fn demonstrates large improvements over NGCF (9.57% relative improvement on recall).” The left column's machinery was not neutral in this setting. It was worth 9.57% on recall, in the wrong direction.
The right-hand column is not a straw baseline either. Rendle's group re-ran the neural collaborative filtering experiments in 2020 and reported: “First, we show that with a proper hyperparameter selection, a simple dot product substantially outperforms the proposed learned similarities.” They also found the learned MLP similarities too costly for production item recommendation. That is what careful tuning of the classical column buys. Here it bought the reversal of a headline result.
Feature-rich graph convolution
Uses transformations, nonlinearities, and interaction terms.
- Can exploit informative node features
- Adds parameters and optimization complexity
- May overfit sparse graphs
- Useful beyond pure ID-based CF
LightGCN propagation
Keeps neighborhood aggregation and removes several neural components.
- Strong pure-CF baseline
- Efficient and easier to inspect
- Still inherits graph and sampling bias
- Limited without side information
Matrix factorization
Uses direct user-item embeddings without graph propagation.
- Cheap and well understood
- No explicit higher-hop aggregation
- Strong baseline under careful tuning
- Useful for measuring graph lift
Graph neural recommenders learn by propagating collaborative signal
In a user-item graph, message passing updates each node from its neighbors. Multi-hop propagation reaches higher-order collaborative structure. Layer aggregation then combines the local evidence with the broader kind.
For pure collaborative filtering, removing feature transformation and nonlinear activation improved results. Measured against NGCF under exactly the same experimental setting, the gain averaged 16.52% on recall and 16.87% on NDCG across three datasets. Read that as a result about matching an architecture to the information actually available. Here the nodes are bare identifiers, nothing more. It is not proof that graph networks never need transformations.
Match the architecture to the information you actually have. Machinery the signal cannot feed costs quality instead of adding it.
Case
LightGCN removed two components and gained 16.52% on recall and 16.87% on NDCG
LightGCN is an ablation before it is an architecture. The abstract rounds the result to about 16.0% relative improvement on average over NGCF, under exactly the same experimental setting. The paper itself gives the two averages that summary compresses: “On average, the recall improvement on the three datasets is 16.52% and the ndcg improvement is 16.87%, which are rather significant.”
One concrete pair sits under those averages. On Gowalla the best recall@20 reported in the NGCF paper is 0.1570; LightGCN reaches 0.1830 in the 4-layer setting. That is 16.56% higher. It is also the baseline the index in the figure is drawn against. And depth was not what the earlier model had been buying.
The comparison cuts both ways. In 2022 Rendle's group took implicit alternating least squares matrix factorisation, iALS, back to benchmarks where it had been written off: “We revisit four well-studied benchmarks where iALS was reported to perform poorly and show that with proper tuning, iALS is highly competitive and outperforms any method on at least half of the comparisons.”
That sentence is a floor, not a share. It says how rarely the tuned classical method lost, at minimum, and leaves the top of its win rate unstated. Which is exactly enough to disqualify a graph result whose comparator was never tuned.
Figure
Steps
Evaluate a graph recommender responsibly
Freeze the temporal graph rules before tuning anything. Otherwise the depth ablation measures leakage instead of propagation. Then tune the baselines as hard as the graph model, ablate depth and components, and slice by degree — all before anyone reads a headline number.
Step 5 is the one teams skip, and it has a documented price list. PinSage, published in 2018, is the graph recommender that was actually deployed at scale: “We deploy PinSage at Pinterest and train it on 7.5 billion examples on a graph with 3 billion nodes representing pins and boards, and 18 billion edges.” What justified a build that size was not an offline table. It was an online number. The production A/B test on homefeed recommendations reports “Depending on the particular setting, we observe 10-30% improvements in repin rate over the Annotation and Visual embedding based recommendations.”
A named online metric, and a measured online lift. That is the standard the audit holds your graph build, neighbor access, retraining and serving complexity against.
1. Freeze temporal graph rules
Construct train and evaluation graphs without future edges.
2. Tune strong baselines
Include MF, BPR, item-item, and LightGCN.
3. Ablate depth and components
Compare propagation hops, layer weights, and feature transforms.
4. Slice by degree
Measure head, mid, tail, new, and sparse users.
5. Audit operational cost
Track graph build, neighbor access, retraining, and serving complexity.
Example
GNN recommendation failures
Oversmoothing hurts the model, and benchmark inflation hides that it did. Neither is folklore. Both come with a number.
The mechanism was named in a 2018 AAAI paper: “First, we show that the graph convolution of the GCN model is actually a special form of Laplacian smoothing, which is the key reason why GCNs work, but it also brings potential concerns of over-smoothing with many convolutional layers.” Smoothing is why graph convolution works. Keep smoothing long enough and it is also what kills the representation.
The inflation has been counted as well. A 2019 audit went through the literature: “Specifically, we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort. For these methods, it however turned out that 6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or graph-based techniques.” Eighteen published methods went in. Six of the seven that could be rebuilt lost to a heuristic.
- Oversmoothing: Repeated aggregation makes users and items less distinguishable — the failure the 2018 AAAI paper traced to Laplacian smoothing, and the one LightGCN cites it for when it says embeddings “will be over-smoothed” as layers increase.
- Popularity propagation: High-degree items spread into many user representations.
- Graph leakage: Evaluation graphs contain future or held-out interactions.
- Sampler confounding: Neighbor and negative sampling choices dominate architecture comparisons.
- Benchmark inflation: Weakly tuned BPR or MF baselines exaggerate GNN gains. Of 18 neural top-n algorithms from top-level conferences, only 7 could be reproduced with reasonable effort — and 6 of those 7 could often be beaten by simple nearest-neighbor or graph-based heuristics.
Visual
The LightGCN-style computation
The model deliberately computes very little. Trainable initial embeddings, degree-aware averaging over neighbors, a few hops, one combination step. Then pairwise scoring inside a ranking objective does the learning.
The combination step is not decoration. LightGCN-single skips it and peaks at 2 layers, at its worst by 4. The full model does not degrade at 4 layers. One averaging step is the whole difference.
Initial embeddings
Assign trainable vectors to users and items.
Normalized propagation
Aggregate neighboring embeddings with degree-aware weights.
Multiple hops
Repeat propagation to reach higher-order collaborative structure.
Layer combination
Average or weight embeddings from several depths.
Pairwise scoring
Use user-item similarity within a ranking objective such as BPR.
Key idea
More layers widen mixing without deepening personalization
More graph layers create wider mixing, not better personalization. The paper measured where the widening turns into damage. Read out only the last layer, with no layer combination, and this is the curve: “The peak point is on layer 2 in most cases, while after that it drops quickly to the worst point of layer 4.”
Full LightGCN combines embeddings from several depths, and does not degrade at 4 layers. The fix in that experiment was the combination step, not fewer hops. Which is why a depth sweep is something you run, not something you guess at.
Treat layer count as an experiment with a stopping rule. Without layer combination the curve peaked at 2 layers and bottomed out at 4 — and a dip of that shape is the kind nobody traces back to depth.
Key idea
The graph-model gate
Adopt a graph neural recommender only when it beats strong simpler baselines under identical graph construction, sampling and compute. Strong means tuned the way iALS was tuned in 2022, and the way the plain dot product was tuned in 2020. In both cases, proper tuning of the classical method was enough to overturn the published verdict. The data did not change. The metric did not change.
An unmatched comparison is not evidence of a graph effect; it is evidence that one system got more attention than the other.
Key takeaways
- LightGCN is an ablation before it is an architecture. Deleting feature transformation and nonlinear activation gained 16.52% on recall and 16.87% on NDCG on average across three datasets against NGCF, under exactly the same experimental setting.
- Message passing updates each node from its neighbors. On Gowalla that means 29,858 users and 40,981 items joined by 1,027,370 interactions, at density 0.00084.
- More layers mean wider mixing, not better personalization: LightGCN-single peaks at 2 layers and is at its worst at 4, while the full model with layer combination does not degrade at 4.
- Trainable vectors for users and items carry the model, with degree-aware averaging and one combination step doing the rest. In the released LightGCN-PyTorch reimplementation, seed 2020, that is 0.1824 top-20 recall on Gowalla at three layers.
- Oversmoothing has a named mechanism: graph convolution is a special form of Laplacian smoothing, shown in a 2018 AAAI paper, which is why repeated aggregation makes users and items less distinguishable.
- Graph build, neighbor access, retraining and serving are recurring costs the graph has to repay. PinSage justified a 3-billion-node, 18-billion-edge system with 10-30% improvements in repin rate in a production A/B test. And a properly tuned iALS still outperformed any method on at least half of the comparisons across four benchmarks.