Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs

AI summary

Overview

Research area: Large language model knowledge editing, specifically lifelong/sequential model editing and the internal mechanisms by which transformer feedforward networks store factual knowledge.

Technical level: Advanced. The paper assumes familiarity with transformer internals (FFN key-value memory decomposition, unembedding projections), the locate-then-edit editing paradigm, and evaluation protocols such as edit success, generalization, and locality.

Scope: The paper proposes NMKE (Neuron-Specific Masked Knowledge Editing), a fine-grained editing framework that combines neuron-level attribution with entropy-guided dynamic sparse masking, and evaluates it against seven baselines on thousands of sequential edits across three LLMs.

What This Paper Is About

Lifelong knowledge editing aims to keep updating facts inside a large language model — for example, changing "the year of the next Olympic Games" from 2024 to 2028 — without retraining the model from scratch. Existing methods, whether they add external parameters or directly rewrite internal weights, accumulate errors as edits pile up: editing accuracy and generalization both decay, and the model's general abilities (math, reasoning, coding) collapse. The paper's goal is an editing method that makes far fewer parameter changes per edit, so that the model can absorb thousands of sequential edits without forgetting what it already knew.

Key Contributions

1. Diagnosis of lifelong editing degradation. The authors show experimentally that performance decline in lifelong editing is driven by cumulative disruption of neurons caused by coarse-grained parameter updates, and they characterize the distinct roles of Knowledge-General and Knowledge-Specific neurons in storing factual knowledge.

2. The NMKE framework. A fine-grained editing framework that uses neuron-level attribution plus dynamic sparse masking to modify only the neurons most relevant to the knowledge being updated, reducing unintended disruption to the rest of the model.

3. Entropy-guided dynamic selection ratios. Rather than fixing the fraction of neurons to edit, NMKE derives the selection ratio from the distribution of attribution scores using normalized entropy measures, separately for knowledge-general and knowledge-specific neurons.

4. Large-scale empirical validation. Evaluation on thousands of sequential edits across LLaMA3-8B-Instruct, GPT2-XL, and Qwen2.5-7B shows NMKE maintains high editing success rates and preserves general capabilities where baselines collapse.

Main Findings

• Knowledge is sparsely localized, and neuron type determines damage. On LLaMA2-7B with MMLU biology and chemistry subtasks, masking the top-10 highest-attribution knowledge-general neurons dropped accuracy from 37.5% to 4.17% (a ↓33.33% drop), and masking the top-50 dropped it to 0.69% (↓36.81%). Masking task-specific neurons in the same domain cost only ↓2.04% and ↓6.25%, and masking domain-specific neurons from a different domain had a negligible effect.

• Three observed neuron categories. Cross-task activation analysis identified knowledge-general neurons (active across all tasks, the most frequent), domain-specific neurons (active within a subject), and task-specific neurons (active in one task only, the least frequent). The paper consolidates these into knowledge-general and knowledge-specific neurons for editing purposes.

• NMKE avoids the sequential-edit collapse seen in baselines. At T=2000 on ZsRE with LLaMA3-8B-Instruct, NMKE scored Rel. 0.94 / Gen. 0.85 / Loc. 0.71, versus AlphaEdit at 0.32 / 0.28 / 0.06. On CounterFact at T=2000, NMKE scored 0.98 / 0.67 / 0.38 versus AlphaEdit at 0.22 / 0.13 / 0.04.

• AlphaEdit's decline is quantified. AlphaEdit's editing success accuracy fell by ↓0.67 on ZsRE and ↓0.78 on CounterFact over 2000 edits, with catastrophic forgetting at T>1500. WISE's editing success rate fell by ↓0.46 after 2000 edits.

• General capabilities survive. FT destroyed GSM8K and HumanEval performance after only 100 edits; ROME and MEMIT showed significant drops beyond T=500; AlphaEdit at T=1500 saw accuracy on both GSM8K and HumanEval drop to 0. After 5000 editing steps, NMKE kept MMLU at 0.59.

• Internal weight distributions shift less. t-SNE visualization of down-projection weights in the 8th FFN layer after 2000 ZsRE edits showed AlphaEdit producing a dispersed, distorted geometry relative to the pre-edited model, while NMKE stayed compact and closely aligned with the original distribution.

• The dynamic ratio beats fixed alternatives. Among four neuron selection strategies (knowledge-general only, knowledge-specific only, both with fixed ratio, both with entropy-based dynamic ratio), the entropy-based dynamic ratio achieved the best balance, while knowledge-general-only editing excelled at locality preservation and retention of coding abilities.

• Overlapping neurons mediate an edit-success versus locality trade-off. Editing only overlapping neurons gave the strongest locality (0.84 at T=1000, 0.80 at T=2000) with lower reliability; editing only non-overlapping neurons raised reliability and generalization but weakened locality; NMKE's entropy-guided combination gave the best overall balance.

• Binary masking beats soft masking over time. A soft-mask variant matched NMKE at T=1000 (0.96 / 0.87) but degraded sharply by T=2000 (0.72 / 0.61 / 0.19) versus NMKE's 0.94 / 0.86 / 0.74, because soft masks update densely and indiscriminately.

• Runtime is competitive but attribution adds cost. Per-edit step times were 16.83s (MEMIT), 26.49s (WISE), 22.16s (AlphaEdit), 22.25s (NMKE with MPC), 29.67s (NMKE with PSA), and 30.42s (NMKE with LPS). NMKE (MPC) added roughly 0.2s of per-edit overhead, while the best-performing variant, NMKE (LPS), added roughly 8s per edit.

• Hyperparameter effects. Higher bias terms improved editing accuracy; larger constant scalers broadened neuron activation and reduced locality preservation. Amplification factors of λ=10 and λ=30 were analyzed, with a moderate increase helping prioritize the most relevant neurons.

Methodology in Plain English

Transformers store factual associations in their feedforward layers, where each neuron acts like a key-value pair: an input pattern matches a key, and the neuron writes its value into the output. The authors' starting point is that layer-level editing rewrites too much of this machinery at once.

Their method has two parts. First, attribution: for each editing prompt, they perturb each neuron's contribution to the output, project both the perturbed and unperturbed representations into the vocabulary space, and measure the gain in log-probability for the target answer. This gives an importance score per neuron per prompt, producing a matrix of attribution scores. Second, masking: they compute two scores per neuron — a "knowledge-general" score counting how many prompts give it a positive attribution, and a "knowledge-specific" score taking its maximum attribution across prompts. Instead of choosing a fixed fraction of neurons to keep, they estimate the fraction from how spread out the attribution distribution is, using normalized entropy: uniform, high-entropy attribution suggests more knowledge-general neurons, and a heavy-tailed maximum-attribution distribution suggests more knowledge-specific neurons. Constant scalers and bias terms adjust these ratios. Neurons above either threshold are kept in a binary mask, and that mask limits the weight update matrix during editing. The optimization pipeline itself follows AlphaEdit.

Why This Matters

Impact on research. The paper reframes lifelong editing degradation as a granularity problem rather than a formulation problem: the fix is not a better constraint on layer-level updates but restricting updates to the right neurons. It also provides a functional taxonomy of knowledge neurons with ablation evidence linking neuron type to capability loss.

Real-world applications:

• Keeping deployed assistants factually current without retraining, e.g., correcting outdated facts such as the year of the next Olympic Games or changed organizational facts.
• Error correction in production models, where a model repeatedly produces a wrong but confidently stated answer.
• Sensitive content removal, one of the use cases the paper lists for model editing, allowing targeted deletion of specific knowledge without sacrificing general ability.
• Long-lived domain assistants in environments where facts change frequently, such as operations or customer-facing systems requiring thousands of small updates over time.

Industry relevance. The runtime numbers matter: NMKE matches AlphaEdit's per-edit cost in its cheapest configuration (22.25s versus 22.16s step time), so the stability gains do not require a heavier editing pipeline. The result that AlphaEdit loses all GSM8K and HumanEval accuracy at T=1500 is a direct warning about deploying layer-level editors in long-running systems. The paper reports 8 NVIDIA A100 GPUs for experiments, and code is released at https://github.com/LiuJinzhe-Keepgoing/NMKE.

Future Directions

• Reducing attribution cost at scale. The best-performing variant (LPS) adds roughly 8s per edit, while the cheapest (MPC) adds roughly 0.2s. The paper presents NMKE as a flexible framework supporting attribution methods of varying complexity, but narrowing this quality-versus-cost gap remains open.

• Pushing beyond the evaluated edit horizon. The paper reports results for T up to 5000 and shows MMLU at 0.59 at that point. Whether NMKE's stability holds at even larger edit budgets is not established.

• Understanding overlapping neuron behavior. Table 3 shows overlapping neurons hold locality while non-overlapping neurons drive edit success and generalization; a principled way to allocate edits between the two sets could improve the trade-off further.

• Extending the neuron taxonomy. The functional analysis in the provided content covers MMLU college and high school biology and chemistry on LLaMA2-7B. The paper states that broader generality verification across domains is presented in Appendix B.1, but that appendix is truncated here. The paper also defers limitations and discussion to Appendix C, which is not included in the provided content, so the authors' own stated limitations are not reported in this material.

Target Audience

Researchers and engineers working on model editing, knowledge injection, and continual learning in LLMs; interpretability researchers studying where and how transformers store factual knowledge; and practitioners who need to keep deployed models factually current over long service lifetimes without retraining or losing general capabilities. Readers without background in transformer internals and editing evaluation metrics will find the notation-heavy method and appendix material demanding.

Authors’ abstract

Lifelong knowledge editing enables continuous, precise updates to outdated knowledge in large language models (LLMs) without computationally expensive full retraining. However, existing methods often accumulate errors throughout the editing process, causing a gradual decline in both editing accuracy and generalization. To tackle this problem, we propose Neuron-Specific Masked Knowledge Editing (NMKE), a novel fine-grained editing framework that combines neuron-level attribution with dynamic sparse masking. Leveraging neuron functional attribution, we identify two key types of knowledge neurons, with knowledge-general neurons activating consistently across prompts and knowledge-specific neurons activating to specific prompts. NMKE further introduces an entropy-guided dynamic sparse mask, locating relevant neurons to the target knowledge. This strategy enables precise neuron-level knowledge editing with fewer parameter modifications. Experimental results from thousands of sequential edits demonstrate that NMKE outperforms existing methods in maintaining high editing success rates and preserving model general capabilities in lifelong editing.