Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions

AI summary

Overview

Research area: Multi-agent LLM systems, agent memory/context compression, and group decision-making (hidden-profile tasks from social psychology).

Technical level: Intermediate. The paper is readable with a basic grasp of LLM prompting and benchmarking, but it relies on a crossed experimental design and bootstrap/Holm statistics that assume some familiarity with empirical NLP methodology.

Scope (one sentence): A diagnostic benchmark, CrossProfile, that holds fact wording fixed while varying who holds which fact, in order to test whether the two natural memory-selection heuristics for LLM groups — keep the rarest claims, or keep the claims relevant to the decision criteria — actually help a downstream decision maker.

What This Paper Is About

When an LLM agent group debates a decision and that discussion is compressed into a short "memory" for a final decision call, someone has to choose what the memory keeps. Two heuristics look obviously sensible: keep the facts few agents mentioned (rarity), or keep the facts that bear on the stated decision criteria (relevance). The paper shows that standard hidden-profile benchmarks cannot tell these heuristics apart from luck, because on those tasks "rare" and "useful" happen to coincide. The authors build a benchmark where rarity and usefulness are deliberately crossed, and find that both heuristics fail in specific, predictable cells — and that the decision model itself adds a third failure by treating a fact stated by many agents as more credible.

Key Contributions

1. CrossProfile, a diagnostic benchmark of 60 main templates (plus 10 pilot templates excluded from main estimates) across 12 decision domains, each with four agents, three options, and 12 facts (five support, three decoy, four irrelevant). Each role group is independently shared or private, giving 2^3 ownership variants and 480 derived tasks, with automatically verified answer keys (all 70 templates pass the verifier).
2. Evidence across four models (Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, DeepSeek-V4.1-Flash, DeepSeek-V4-Pro) with random-reference, fixed-item-count, and oracle-relevance controls, showing that rarity and relevance each fail in the ownership cells the design predicts because neither tracks the balance of evidence.
3. Isolation of a social-proof effect in the decision model: the model gives extra weight to facts attributed to many agents, demonstrated by removing holder tags and by randomly reassigning the same tag sets with line order held fixed.
4. Natural-discussion diagnostics showing that in unscripted discussions the bottleneck sits upstream of memory: the full transcript scores below an empty memory on all three models tested.

Main Findings

• HiddenBench conflates rarity with usefulness. On controlled HiddenBench at r = .6, rarest-first selection (R1) reaches 88.2% on Qwen and 87.7% on Llama versus 51.3% and 59.0% for the evidence-preserving summary (B3). Averaged over budgets, R1 minus B3 is +29.2 points [22.9, 35.4] and +26.7 [20.2, 33.2]. But a memory of only the private facts reaches 91.3% and 90.3%, while a memory of only the shared facts reaches 5.6% and 4.6% — far below an empty memory (35.4% and 36.4%). Rarity wins there because private facts are the correct evidence, not because rarity is a good principle.

• Rarest-first is no better than random on CrossProfile averages. R1 scores 64.0% versus 64.5% for Random on Qwen, and 63.0% versus 63.4% on Llama. This average hides a full reversal: when support is private and decoys shared, R1 reaches 90 to 97% while first-mention selection (B4) falls to 11 to 31%; when support is shared and decoys private, B4 sits at 92 to 96% and R1 at 18 to 40%.

• The rarity advantage depends on who holds the decoys (H1). The pre-specified interaction is +64.6 points [60.3, 69.0] for Qwen and +62.6 [56.8, 68.2] for Llama (Holm p < .001). Against Random the interaction is +40.7 [34.6, 47.1] and +34.2 [27.8, 41.0]; when both policies retain exactly seven facts it is +66.8 [61.9, 71.7] and +72.6 [67.2, 78.1]. DeepSeek models show the same pattern (Flash H1 +62.6 [56.7, 68.6]; V4-Pro interaction against Random +30.6 [22.5, 38.6] on 30 templates). On a common 30-template subset, R1 minus Random averages −1.7, +1.7, −1.3, and −4.7 points across the four models.

• Relevance readmits on-topic decoys. Relevance-first (R1-rel) has the highest mean of the budgeted policies (72.6% Qwen, 74.6% Llama; H3 = +8.5 [3.5, 13.6] and +15.6 [11.1, 19.9]), but in the classic cell (private support, shared decoys, private irrelevant) it loses 22.2 points [−28.9, −16.1] on Qwen and 20.0 [−26.1, −13.9] on Llama relative to R1, which scores 96.7% and 97.2% there. A perfect oracle relevance filter loses just as much: −23.3 [−30.6, −16.1] and −31.1 [−38.9, −23.9]. DeepSeek-V4.1-Flash, whose relevance filter is nearly perfect (99.4% precision, 98.9% recall, versus 66.0% recall on Qwen and 85.6% on Llama), still loses 32.2 points.

• Accuracy tracks the balance of evidence, not the policy. Pooling the four copy policies, accuracy rises from 13.5% and 10.3% at a net evidence lead of −2, to 76.0% and 75.6% at +2, to 96.2% and 96.1% at +4 or more. In a logistic model with this lead and the tone of kept irrelevant facts, the policy terms are no longer distinguishable from zero. Wrong answers are mostly the decoy: B4 picks it in 71.1% and 63.3% of classic-cell decisions.

• The decision model follows social proof from holder tags. Although the all-facts memory gives the correct option a two-point lead in every variant, accuracy is 21.0 points [16.5, 25.6] higher on Qwen and 13.8 [9.7, 17.9] higher on Llama when support facts are shared rather than private. Removing holder tags raises classic-cell accuracy by +14.4 [7.8, 21.1] on Qwen, +13.3 [6.7, 20.0] on Llama, and +24.4 [17.2, 32.2] on Flash (72.8% to 97.2%); random reassignment with order fixed raises it by +12.8 on both open models. The effect is not purely an error: where support is shared and decoys private, removing tags lowers Qwen's accuracy by −10.6 [−16.7, −4.4].

• In natural discussions, evidence never arrives. No memory policy is detectably better than another (R1 minus B3: −1.4 [−5.0, 2.4] on Qwen, −3.1 [−9.2, 2.7] on Llama). The full transcript scores 22.1, 33.8, and 25.6 versus 33.8, 36.9, and 35.9 for an empty memory on Qwen, Llama, and Flash; the group's own final vote is correct in only 24 to 32% of discussions, while the same models reach 62 to 90% given every fact. String matching finds private facts stated in full in just 1.7% [0.6, 2.9] of Flash discussions.

• A prompt fix helps one model only. Asking agents to state evidence before recommending raises Llama's full-transcript accuracy by +30.8 [21.5, 39.5] (+25.1 [15.9, 34.4] in a paired ablation with fresh seeds), with no detectable change for Qwen. Under that prompt, memory policy matters again for Llama: R1 57.1% versus B4 22.9% and B3 53.2%.

Methodology in Plain English

The authors build synthetic decision scenarios with a known answer. Each template names three options and 12 sentences of evidence, split into three roles fixed by the experimenters: support (favors the right answer), decoy (true, on-topic, but favors the wrong answer), and irrelevant (about other attributes). Scoring is mechanical — favorable facts add one, unfavorable subtract one, irrelevant count zero — so the correct option wins by +2 and each competitor scores 0. Because the sentences never change, only who holds them, the experimenter can make the support facts rare or common independently of the decoy facts.

Agents then "discuss" in a scripted way that guarantees every assigned fact is stated, so the memory policy is choosing from a known inventory rather than from whatever a free-form model happened to say. Competing memory policies build the memory under a fixed token budget (a fraction r of the full fact text, with r = .6 primary and .4/.8 as checks): rarest-first, first-mention, random, a summary, and relevance-first (an LLM filter for on-topic statements). A separate call with the same model weights then answers using only the task description, the options, and the memory. Controls strip or randomly reassign the [A1,A2] holder tags, hold the number of retained items constant, and swap in an oracle relevance filter. Statistics use 95% percentile bootstrap intervals from 10,000 resamples with Holm correction on the three pre-specified tests, resampling templates rather than individual items. A separate natural-discussion condition uses the benchmark's own two-round exchange with initial and final votes instead of scripting.

Why This Matters

Impact on research: The paper undercuts a widely applicable shortcut. "Keep rare information" and "keep relevant information" are standard instincts in summarization, retrieval, and multi-agent memory design, and this work shows both can be measured as wins on a confounded benchmark while being no better than random — or actively harmful — once ownership is varied. It also relocates the failure: the memory policy is not the only suspect, and evaluations that report only end-task accuracy cannot separate disclosure failures, compression failures, and weighting failures.

Real-world applications:

• Enterprise decision-support agents where a committee of assistants summarizes evidence for an executive: the summary should be selected for the balance of evidence on each option, not for how many agents mentioned a claim.
• Clinical or procurement review panels using LLM scribes, where a true but partial fact (a failed inspection, a slow repair record) can flip a recommendation without any falsehood being introduced.
• Multi-agent research assistants that aggregate distributed sources, where the same fact reaching many agents is a routing artifact rather than independent corroboration.
• Meeting-transcript summarization tools that must decide which lines survive a context budget before a downstream decision is made.

Industry relevance: The social-proof result is directly actionable and cheap: when sources are not independent, presenting "how many agents said this" as a credibility signal is a bug, and simply removing or randomizing those tags lifted classic-cell accuracy by 13 to 24 points. The upstream finding is equally practical — if private evidence never enters the discussion, no memory architecture fixes the decision, and a single prompt change (evidence before recommendation) moved one model's transcript accuracy by +30.8 points.

Future Directions

• Memory selection by evidence balance. Since accuracy is predicted by the net lead of the correct option among kept facts, and policy identity adds little once that lead is known, can a policy be built that explicitly audits and balances kept evidence per option rather than ranking individual claims?
• Attribution without false authority. The paper shows holder tags mislead when facts were assigned rather than independently observed; open questions are how to represent genuine corroboration, and whether provenance should be shown to the decision model at all.
• Closing the upstream disclosure gap. Only one of the models responded to the evidence-first prompt. What makes a model actually state its private evidence, and can disclosure be raised for weaker models without hurting those already disclosing?
• Beyond the fixed design. The paper studies four agents, three options, and a fixed group size; it explicitly does not vary group size, and its natural-discussion results leave open whether other intervention styles (the calibrated-controller line of work it cites) compose with better memory selection.

Target Audience

Researchers and practitioners working on multi-agent LLM systems, agent memory and context compression, and benchmark design for collective decision-making. It is most valuable to anyone who has measured a "rare information" or "relevance filtering" gain on a hidden-profile-style benchmark and wants to know whether that gain survives a design in which rarity and usefulness are decoupled — and to engineers shipping summarization or memory layers in front of a downstream decision call.

Authors’ abstract

When an LLM group's discussion is compressed into a short memory for a final decision maker, what should the memory keep? Two natural heuristics keep the rarest claims or the claims relevant to the decision criteria. On hidden-profile tasks both look attractive, because how widely a fact is held is confounded with whether it helps. We introduce CrossProfile, a diagnostic benchmark of 60 templates with automatically verified answer keys, in which the ownership of supporting, decoy, and irrelevant facts is crossed while their text is held fixed (480 tasks). Across Qwen2.5-7B, Llama-3.1-8B, and two DeepSeek models, both heuristics fail in predictable places. Selecting by relevance readmits decoys, because decoys are on topic: even a perfect relevance filter loses 23 to 31 points to rarest-first selection in the classic hidden-profile configuration. Rarest-first selection is no better than random selection on average and wins only where the rare facts happen to be the useful ones. In both cases accuracy follows the balance of evidence the memory keeps, not the policy that kept it. The decision model adds a third failure, a form of social proof: it gives extra weight to facts attributed to many agents. Removing the attribution tags, or reassigning them at random with line order fixed, raises accuracy in the classic configuration by 13 to 24 points. In natural discussions the evidence is lost before any memory acts: the complete transcript yields lower accuracy than no notes at all.