SPRInG: Continual LLM Personalization via Selective Parametric Adaptation and Retrieval-Interpolated Generation

AI summary

Overview

Research area: Continual learning and personalization of large language models (LLMs), specifically long-form personalized text generation.

Technical level: Intermediate. The paper assumes familiarity with LoRA adapters, retrieval-augmented generation, and continual learning concepts such as replay buffers and catastrophic forgetting, but its core ideas are explained through intuitive terms (novelty, quality, drift).

Scope: The paper proposes and evaluates a semi-parametric framework, SPRInG, that adapts a user-specific adapter only to interactions signaling genuine preference drift while fusing parametric and retrieved knowledge at inference time.

What This Paper Is About

Most LLM personalization methods treat a user's preferences as fixed: they either retrieve from a frozen history or train an adapter once and stop. In reality, user interests evolve continuously, and the interaction streams that reveal these changes are noisy, so indiscriminate updating risks overfitting to transient context or forgetting important history. SPRInG tackles this by deciding which interactions are worth learning from during training and when retrieval is worth trusting during inference.

Key Contributions

1. Drift-driven selective adaptation. A likelihood-based drift score combining a novelty term (how much the adapter lags the base model on a sample) and a quality term (how well-formed the sample is under the base model) is used to select the top 30% of each incoming batch for adapter updates, filtering pseudo-novelty from genuine preference shifts.
2. Residual replay buffer construction. After each update, candidates (new drift samples plus the previous buffer) are re-scored with the updated adapter, and the highest-scoring, still-hard samples are retained under a fixed budget of N_max = 50 per user, preserving sparse historical nuances that weight updates fail to internalize.
3. Selective retrieval-interpolated generation. At inference, a relevance gating mechanism (threshold σ at the third quartile of the retrieval score distribution) filters retrieved history, and the parametric and retrieval-augmented next-token distributions are fused by probability interpolation with mixing coefficient λ = 0.5.
4. Empirical validation on a long-form personalized generation benchmark. SPRInG is evaluated on the LongLaMP benchmark (Abstract Generation and Review Writing) across five temporally ordered periods, together with a temporal drift analysis confirming multidimensional non-stationarity in the evaluation streams.

Main Findings

• SPRInG leads across all periods on both tasks. Period-average scores reach ROUGE-1 0.350 / ROUGE-L 0.190 on Abstract Generation and ROUGE-1 0.326 / ROUGE-L 0.142 on Review Writing, versus the strongest CL baseline CAMA at 0.337 / 0.164 and 0.312 / 0.125 respectively.
• Relative gains over the strongest CL baseline. 3.86% in ROUGE-1 and 15.85% in ROUGE-L on Abstract Generation; 4.49% in ROUGE-1 and 13.60% in ROUGE-L on Review Writing.
• Standard personalization can be harmful. In Review Writing, RAG each falls 8.60% below the non-personalized Base model in ROUGE-1, while SPRInG achieves an 18.12% relative improvement over RAG each.
• Non-stationarity is real and grows over time. Comparing Period 0 to Period 4 against a random within-user baseline, drift exceeds the baseline by 1.86–7.13× for Abstract Generation and 2.22–4.33× for Review Writing, and P0→P4 drift is consistently greater than P0→P1 drift.
• Each component adds a distinct benefit (ablation). The user adapter alone yields relative gains of 7.6% in ROUGE-1 and 5.4% in ROUGE-L on Abstract Generation and 3.6% in ROUGE-1 on Review Writing; adding retrieval interpolation yields the largest gain, 8.3% in ROUGE-1 and 23.2% in ROUGE-L over the adapter-only setting on Abstract Generation; relevance gating adds a further 2.2% in ROUGE-1 on Review Writing while leaving Abstract Generation nearly unchanged.
• Selective updating beats updating on everything. Top-30% selection outperforms All-100% (0.324 vs 0.300 ROUGE-1 on Abstract Generation, an 8.0% improvement, and 14.1% over Bottom-30%) despite using 70% fewer update samples, and matches or improves on a random 30% subset.
• Gating is task-dependent. Under σ = Q3, retrieval still triggers for 92.3% of Abstract Generation queries but only 41.4% of Review Writing queries; gating filters out 58.6% of Review Writing queries yet still yields a 2.19% relative ROUGE-1 gain, and reduces inference cost by falling back to single-path decoding.
• Scalability across interaction volumes. In the Large Interactions group, baselines such as Base and OPPU degrade on Review Writing, likely from noise accumulation, whereas SPRInG leverages the extended histories to improve performance.

Methodology in Plain English

The setup assumes a frozen base LLM (Gemma-3-4B-IT) with a lightweight LoRA adapter per user. Each user's chronologically ordered history is split into five periods (Period 0–4); Period 0 is the initial observed history, and Periods 1–4 arrive sequentially. Within a period, the earliest 90% of samples are available for updating and the most recent 10% are held out for evaluation.

Training. Every incoming interaction is scored by comparing its mean token log-likelihood under the base model versus the current adapter. The novelty term rewards samples the adapter handles worse than the base model; the quality term rewards samples the base model finds well-formed, which guards against selecting gibberish just because it is hard. Only the top 30% by this score are used to fine-tune the adapter, which is initialized from the previous period to inherit prior preferences. The drift samples are then merged with the existing buffer and re-scored using the updated adapter; the samples that remain hardest to predict are kept in a buffer capped at 50 items per user.

Inference. For a new query, the system retrieves the top-2 interactions from the buffer using BM25 and keeps only those whose similarity exceeds a threshold set at the third quartile of the retrieval score distribution. If nothing passes, generation falls back to the adapter alone. Otherwise, two distributions are computed at each decoding step—one from the query alone (parametric) and one from the query plus retrieved context (retrieval-augmented)—and mixed with equal weight (λ = 0.5) before selecting the next token by greedy decoding.

Why This Matters

Impact on research. The paper reframes LLM personalization as a problem of distinguishing genuine preference evolution from transient noise, rather than as generic continual learning. Its drift score differs from prior replay-selection criteria (interference, confidence, activation magnitude, spaced repetition) by selecting current interactions for adaptation rather than past examples for retention. The temporal drift analysis also provides a methodological template for verifying that a continual-learning setup is genuinely non-stationary.

Real-world applications:

• Long-term conversational assistants that must track shifting user interests without re-reading an ever-growing history.
• Personalized writing tools that adapt to a user's evolving style and topic focus, as in the abstract-generation task drawn from real researchers.
• Recommendation and review platforms where consumer sentiment and preferences drift over years, as in the Amazon-derived review-writing task.
• Any deployment needing cost control over personalization, since selective training uses only 30% of the data and gating allows fallback to single-path decoding for queries without strong retrieved evidence.

Industry relevance. The approach is built on parameter-efficient LoRA adapters and standard BM25 retrieval, so it fits existing serving stacks. Gating acts as a practical quality-cost knob: under σ = Q3, nearly 58.6% of Review Writing queries avoid the dual-path computation. The paper acknowledges that scaling user-specific adapters remains an infrastructure challenge as the user base grows.

Future Directions

• Reducing dual-path inference cost. The paper notes that two forward passes per decoding step introduce latency, and suggests adaptive interpolation such as terminating the dual path mid-generation or reverting to a single parametric path when the two distributions converge.
• Hardening drift detection against false drift. Transient noise and abrupt topic shifts could be mistaken for persistent preference change; the authors point to denser retrieval or cross-encoder re-ranking as ways to isolate semantically relevant evidence more precisely.
• Scaling adapter infrastructure. Hierarchical or shared adapter architectures that group users with similar preference trajectories could reduce storage burdens and enable cross-user knowledge transfer.
• Examining alternative retention and selection policies. The paper defers comparison of retention policies and analyses of selection ratio and drift-score formulation to its appendices, leaving room for further optimization.

Target Audience

Researchers and practitioners working on LLM personalization, continual learning, and retrieval-augmented generation, particularly those deploying user-adaptive systems where preferences change over time. It is also relevant to engineers interested in parameter-efficient fine-tuning with LoRA and in the cost trade-offs of dual-path inference. Readers without some background in adapters or continual learning would benefit from reviewing those concepts first.

Authors’ abstract

Personalizing Large Language Models typically relies on static retrieval or one-time adaptation, assuming user preferences remain invariant over time. However, real-world interactions are dynamic, where user interests continuously evolve, posing a challenge for models to adapt to preference drift without catastrophic forgetting. Standard continual learning approaches often struggle in this context, as they indiscriminately update on noisy interaction streams, failing to distinguish genuine preference shifts from transient contexts. To address this, we introduce SPRInG, a novel semi-parametric framework designed for effective continual personalization. During training, SPRInG employs drift-driven selective adaptation, which utilizes a likelihood-based scoring function to identify high-novelty interactions, selectively updating the user-specific adapter on drift signals while preserving hard-to-learn residuals in a replay buffer. During inference, we apply strict relevance gating and fuse parametric knowledge with retrieved history via probability interpolation. Experiments on the long-form personalized generation benchmark demonstrate that SPRInG significantly outperforms existing baselines, validating its robustness for real-world continual personalization.