Massive Editing for Large Language Models Based on Dynamic Weight Generation

AI summary

Overview

Research area: Knowledge Editing (KE) for Large Language Models — specifically large-scale (batch) editing of LLM parameters.

Technical level: Intermediate. The paper assumes familiarity with LLM internals (feed-forward layers, weights), diffusion models, and standard KE evaluation metrics, but its core idea is describable without heavy mathematics.

Scope: The paper proposes MeG, a method that trains a diffusion model to generate the weights of a single dynamically attached neuron so that tens of thousands of factual edits can be injected into an LLM with high Reliability, Generality, and Locality.

What This Paper Is About

Most existing large-scale knowledge editing methods work by directly modifying subsets of weights inside an LLM, which caps how much knowledge can be edited and progressively degrades the model's unrelated behavior as the number of edits grows. MeG instead attaches one extra neuron to selected feed-forward layers and uses a diffusion model to generate that neuron's weights on demand, conditioned on the query being edited. The goal is to let an LLM absorb very large batches of new facts (from 1024 up to 10000 edits) without cumulative interference to the knowledge it already holds.

Key Contributions

1. A weight-generation-based KE framework (MeG). Instead of editing existing LLM weights, MeG adds a single dynamic-weight neuron to a chosen FFN layer and generates its weights with a Diffusion Transformer (DiT) conditioned on the query. Because only one neuron is added regardless of edit scale, the paper argues knowledge capacity is no longer bounded by the size of the LLM being edited and interference does not accumulate.

2. An InfoNCE-tuned text encoder for Generality. A BERT-based text encoder is fine-tuned with an InfoNCE contrastive loss on original/paraphrased pseudo-knowledge pairs, so that equivalent phrasings of the same edit map to similar conditioning representations.

3. An entropy-based Familiarity Network for Locality. A K-classifier (K=10) with 5 FFN blocks plus a classification head is trained on edited-knowledge queries; output entropy is used to route queries as either edited knowledge (low entropy) or irrelevant knowledge (high entropy), in which case the generated neuron weight is set to zero.

4. A diffusion step-compression and efficiency analysis. Using DDIM sampling, generation is compressed from 1000 denoising steps to 50 while largely preserving all KE metrics, reducing diffusion overhead to a small fraction of total generation time.

Main Findings

• MeG leads on the combined Score metric at 10000 edits across all tested models and datasets. On ZsRE with 10000 edits, Score was 82.80 (Phi-2), 83.20 (GPT-J), and 83.90 (Llama-3); the best baseline scores were 45.64 (MALMEN) on Phi-2, 52.70 (MEMIT) on GPT-J, and 70.03 (MALMEN) on Llama-3. Score is defined as the harmonic mean of the six Reliability, Generality, and Locality results under the TF and AG generation settings.

• Locality is the standout improvement. On ZsRE with GPT-J at 10000 edits, MeG's Locality exceeded the second-best method by +63.69 percentage points (AG) and +12.79 (TF); on COUNTERFACT the margins were +35.28 (AG) and +16.17 (TF). All baselines showed notably poor Locality, especially under AG generation.

• Performance degrades more slowly as edits scale up. Across Phi-2, GPT-J, and Llama-3 on ZsRE and COUNTERFACT, MeG outperformed baselines at 1024, 2048, 4096, and 10000 edits, and the gap widened with scale. SCEN could only be evaluated up to 2048 edits because of its computational and memory overhead.

• Reliability and Generality are not always the single best on every model. On GPT-J at 10000 edits on ZsRE, MeG's Reliability/Generality were marginally lower than some baselines (e.g., Reliability AG 99.11 and TF 99.16, Generality AG 61.69 and TF 75.68), while still achieving a substantial Locality advantage and the highest Score.

• General capabilities are largely preserved after 10000 edits. On Phi-2 after 10000 ZsRE edits, MeG scored BBH 40.67, GSM8K 61.18, MMLU 57.00, RTE 60.47, and SST2 89.68, compared with pre-edit values of 40.58, 42.84, 56.98, 60.79, and 89.56 respectively. Direct fine-tuning (FT) fell to BBH 20.12 and RTE 1.53.

• Contrastive fine-tuning of the text encoder drives Generality. On COUNTERFACT with GPT-J at 1024 edits, Generality was 18.46 (AG and TF) with frozen BERT, 56.54 (AG) / 56.93 (TF) with MSE-tuned BERT, and 84.96 (AG) / 86.33 (TF) with the InfoNCE-tuned BERT.

• The Familiarity Network drives Locality. On ZsRE with Phi-2 at 1024 edits, removing it dropped Score from 91.30 to 81.32, with Locality contributions of 34.86 percentage points (AG) and 12.21 (TF).

• Diffusion generation is necessary for Generality. Replacing DiT with an MLP generator on Llama-3/ZsRE at 10000 edits dropped the Score from 83.90 to 76.21, mainly through lower Generality (DiT 61.33 AG / 78.95 TF vs MLP 45.49 AG / 70.82 TF), although the MLP variant cut training time by about 90%.

• Inference overhead is small. On ZsRE with Phi-2 at 10000 edits, 50 denoising steps gave an average additional time of 0.03s per edit and accounted for only 5.58% of total time (versus 0.304s and 32.12% at 1000 steps), with Score changing from 82.80 to 82.51. At 10 steps the Score dropped to 77.83.

• Without the Familiarity Network, interference from the added neuron does not grow monotonically. Locality showed an initial rise followed by a mild decline, with only 4.89 percentage points (AG) and 3.06 (TF) degradation at 10000 edits compared with 1024 edits.

Methodology in Plain English

The pipeline has four parts:

1. Text encoding. A BERT encoder is fine-tuned with an InfoNCE contrastive loss so that a query and its paraphrases land close together in representation space. The [CLS] embedding of a query becomes the conditioning signal for everything downstream. Because real paraphrase pairs are unavailable, the authors collected pseudo-knowledge with equivalent expressions for this training.

2. Routing. A small classifier (the Familiarity Network) is trained to assign edited queries to one of 10 random categories. The intuition is that a trained network produces low-entropy outputs for data resembling its training set and high-entropy outputs for unfamiliar data. At inference, the entropy of the network's output distribution is compared against a threshold: low entropy means the query concerns edited knowledge and a weight should be generated; high entropy means the query is irrelevant and a zero weight is used, leaving the model unchanged.

3. Collecting training data for weight generation. The authors first pick which FFN layer to attach the neuron to by exploration (diverging from T-Patcher, which uses the last FFN). For each edit, they freeze the LLM's original weights and optimize the single new neuron's weight until the model produces the target answer. This yields N knowledge–weight pairs.

4. Generating weights with diffusion. A Diffusion Transformer is trained on those pairs, treating each weight vector like an image: weights are split into non-overlapping patches of size 100 to form tokens, and the query's CLS embedding conditions generation. A velocity-prediction (v-prediction) objective is used instead of predicting noise or clean data directly. The generated weight dimensions are 25602 for Phi-2 and 40962 for GPT-J and Llama-3. At inference, DDIM fast sampling runs the reverse denoising process, compressed from 1000 to 50 steps, and the resulting weight is loaded into the neuron.

Experiments used Phi-2 (2.7B), GPT-J (6B), and Llama-3 (8B) in Float16, with baselines FT, MEMIT, PMET, MALMEN, and SCEN. Locality is measured as whether the model's response to irrelevant queries is unchanged rather than as accuracy on a dataset before/after editing — a deliberate change the authors argue better matches the metric's intent. All experiments were averaged over five independent runs with fixed random seeds.

Why This Matters

This paper argues that the ceiling on large-scale knowledge editing is set by the editing mechanism itself: modifying internal weights accumulates interference, while adding neurons exhausts static storage. Showing that a generated, query-conditioned single neuron can absorb 10000 edits without eroding general capabilities points to a different scaling strategy for keeping deployed models current.

Real-world applications:

• Keeping deployed assistants factually current — updating product details, policies, or personnel facts without retraining or degrading unrelated behavior.
• Correcting systematic model errors — error-correction style edits, which is what the ZsRE experiments model.
• Counterfactual and compliance-driven updates — rewriting specific beliefs or facts about an entity that the model learned incorrectly, which is what COUNTERFACT models.
• Preserving general reasoning during heavy updates — relevant wherever a model must remain capable on math, reasoning, and classification tasks (GSM8K, BBH, MMLU, RTE, SST2) while its facts are being refreshed.

Industry relevance: the method's key operational claim is low overhead — roughly 0.03 seconds of extra inference time per edit and 5.58% of total generation cost at 50 denoising steps, measured on a single NVIDIA RTX 4090D GPU. The paper also notes a flexibility trade-off: an MLP generator is about 90% faster to train but loses Generality, while DiT is needed for the reported Generality results. Code is released at a public GitHub repository.

Future Directions

• Closing the remaining Generality gap. MeG does not lead on Reliability or Generality in every configuration (for example, Generality on GPT-J/ZsRE at 10000 edits), and the MLP-versus-DiT ablation shows Generality is where the weight generation module matters most.
• Scaling beyond 10000 edits. The paper tests up to 10000 edits and reports slow degradation; whether the design holds at substantially larger scales is not established in the content presented, and SCEN could not be evaluated beyond 2048 edits for comparison.
• Layer selection and architecture transfer. The authors explore and select specific FFN layers per LLM rather than defaulting to the last FFN; how this selection generalizes across architectures is left open.
• Cost of the training pipeline. Weight generation requires first collecting N knowledge–weight pairs by optimizing each neuron individually; the practical cost of that collection stage at very large scales is not reported in the presented content.

Target Audience

Researchers and engineers working on knowledge editing, model updating, and parameter-efficient adaptation of LLMs; practitioners who need to inject large batches of factual updates into deployed models without retraining; and readers interested in diffusion-based weight or parameter generation as an alternative to static memory banks and direct weight modification. Familiarity with LLM feed-forward layers, diffusion sampling, and standard KE metrics helps, but the central idea — generate one neuron's weights per query instead of editing the model — is accessible to anyone who understands how LLMs store and retrieve factual knowledge.

Authors’ abstract

Knowledge Editing (KE) is a field that studies how to modify some knowledge in Large Language Models (LLMs) at a low cost (compared to pre-training). Currently, performing large-scale edits on LLMs while ensuring the Reliability, Generality, and Locality metrics of the edits remain a challenge. This paper proposes a Massive editing approach for LLMs based on dynamic weight Generation (MeG). Our MeG involves attaching a dynamic weight neuron to specific layers of the LLMs and using a diffusion model to conditionally generate the weights of this neuron based on the input query required for the knowledge. This allows the use of adding a single dynamic weight neuron to achieve the goal of large-scale knowledge editing. Experiments show that our MeG can significantly improve the performance of large-scale KE in terms of Reliability, Generality, and Locality metrics compared to existing knowledge editing methods, particularly with a high percentage point increase in the absolute value index for the Locality metric, demonstrating the advantages of our proposed method. Code is available at https://github.com/RodeWayne/MeG-for-Knowledge-Editing.