What You Can't See Is What You Learn: Restricted Evidence Visibility Favors Compositional Generalization in Shared-Genome Language-Model Societies

AI summary

Overview

• Research area: Multi-module neural systems, emergent communication, compositional generalization, and mechanistic interpretability of language-model societies.
• Technical level: Advanced. The paper assumes familiarity with LoRA adapters, attention masking, emergent communication channels, causal abstraction, and interchange interventions.
• Scope: A preregistered, matched causal comparison of slot-selective evidence masking in a four-cell society built from one frozen Qwen2.5-0.5B-Instruct backbone and one shared rank-8 LoRA, tested on natural-language function composition over Z_17.

What This Paper Is About

When a trainable module can read an entire input program, it can simply memorize a whole-program lookup instead of learning to compose smaller operations. The authors ask whether hiding most of the input from each module — forcing each cell to see only its own assigned evidence slot and receive everything else through a learned inter-cell communication channel — changes which solution gradient descent discovers. To answer this, they build four-cell "societies" that share one frozen language model and one adapter, and train matched pairs that differ only in their attention mask.

Key Contributions

1. A tightly matched causal comparison of slot-selective evidence masking. Restricted and global twins share architecture, token layout, positional geometry, trainable parameterization, nominal token count, Transformer calls, training budget, initialization bytes, and ordered training-example streams. Only the attention mask differs — though that mask also implicitly identifies each cell's assigned slot, so the treatment bundles foreign-evidence access, role disambiguation, and active-context load.

2. A large paired effect that survives a collision-stratified check. Median paired advantages are 0.7648 at depth two and 0.6050 at depth three. A post hoc collision-stratified analysis over nine complete pairs finds a median depth-three advantage of 0.558 on programs whose complete affine map never appeared in training, arguing against whole-function lookup as the explanation.

3. Mechanism tests using value-indexed packet transplants. Across six audited restricted societies, packets transplanted from other episodes that denote the same mathematical intermediate preserve behavior at 0.94–1.00 across all tested interfaces, while counterfactual-value packets steer outputs toward the mathematically predicted answer. The sole high-performing global model does not show this interchangeability.

4. Complete outcome reporting, including two formal preregistered failures. An earlier qualification cohort passed 0/10 complete gates (one model met every task-performance gate but all ten failed ordinary-language preservation, with roughly 50–61 percentage points of top-1 regression), and the present battery fails its absolute floor by 0.0012. Final checkpoints, evaluations, audit scripts, preregistration materials, and incident records are released, with one unavailable restricted checkpoint disclosed.

Main Findings

• Restricted visibility beat global visibility in 9 of 10 matched pairs. Restricted societies exceeded their global twins by at least 20 percentage points at both depths in 9 of 10 pairs, with median paired advantages of 0.7648 (depth two) and 0.6050 (depth three).

• The communication channel is necessary for the restricted societies. Severing all packets yields exactly chance performance, 1/17, in all ten restricted models.

• Most global twins memorized without generalizing. All ten global twins fit the primitive-operation training set (single-operation held-out-phrasing accuracy 0.869–0.991) and fit the depth-two training bank exactly, but depth-three training-bank exact accuracy on a fixed 150-program subset ranged from 0.41 to 1.00 while held-out composition stayed near chance at 0.05–0.11.

• One global model is a constructive counterexample. The global twin at initialization 204, order 954 reached 0.843 at depth three — above its restricted sibling, which scored 0.6922 — and also collapsed under packet deletion. Global visibility is therefore not incompatible with a high-performing, communication-dependent solution.

• Restricted societies ranged widely but all performed substantially. All ten restricted twins achieved held-out depth-three accuracy between 0.582 and 0.933. Across an earlier restricted-only world and the paired world, all twenty restricted-visibility societies exceeded 0.58 at depth three.

• The preregistered battery formally fails by a hair. Every preregistered component passed except the absolute restricted-arm depth-three floor: the observed median was 0.6988 against a required 0.70. Median depth-two restricted accuracy was 0.8339.

• Post hoc inferential analysis, clustered by initialization. All five initialization-level mean differences were positive at both depths (depth two: 0.7715, 0.7996, 0.7555, 0.7672, 0.5350; depth three: 0.6367, 0.7176, 0.6020, 0.5774, 0.2563). Cluster-bootstrap intervals over the five initialization strata were [0.628, 0.784] at depth two and [0.397, 0.669] at depth three, and the exact two-sided five-stratum sign-flip p-value was 0.0625 at each depth — the smallest attainable with five strata.

• Composite-function collisions do not explain the effect. Collision-stratified results were available for 19 of 20 checkpoints (nine complete pairs). On map-novel depth-three programs the median paired advantage remained +0.558, versus +0.634 on the map-redundant stratum. Restricted models scored 0.553–0.913 on map-novel programs, ordinary global models stayed near chance at 0.049–0.094, and the 204/954 global model reached 0.828. Map-novel programs comprised 13 of 60 depth-three programs, giving 884 scored episodes per checkpoint.

• Audited restricted societies use an approximately value-indexed relay. Among correctly answered held-out episodes, same-value packet transplants preserved behavior at 0.94–1.00 across all tested interfaces; deletion or approximately norm-matched noise reduced performance to at most 0.11; cross-example shuffling scored 0.07–0.11; counterfactual transplants redirected outputs with fidelity 0.74–1.00. Only the strongest model passed every absolute preregistered transplant gate.

• The high-performing global model uses a different code. Same-value transplants succeeded on only 0.12–0.25 of cases for the 204/954 global twin, indicating an episode- or context-dependent rather than value-indexed communication state. This is a mechanistic case study, not a frequency estimate.

• Both arms learned mask-native procedures. In a post hoc train-mask × evaluation-mask crossover, all ten globally trained models stayed at chance under the restricted evaluation mask at depth three (median 0.0583, range 0.0544–0.0669); the best global model fell from 0.8431 to 0.0581. All nine recoverable restricted-trained checkpoints fell to chance under the global evaluation mask (median 0.0615, range 0.0539–0.0694). No checkpoint retained competence under the opposite mask.

• Centralized comparators were highly variable and not compute-matched. Three staged centralized models scored 1.000, 0.993, and 0.098 at depth three. On an earlier qualification-world split, three atom-trained flat one-call comparators scored 0.117–0.200 at depth two and 0.100–0.267 at depth three, while all three staged controls failed. The authors treat these as contextual baselines rather than causal identification.

Methodology in Plain English

The authors built a small "society" of four processing cells. All four cells are the same frozen Qwen2.5-0.5B-Instruct model plus one shared rank-8 LoRA adapter, totaling approximately 4.32M trainable parameters. There are no cell-specific parameters and no learned cell identities.

Each cell sees an identical token layout containing the shared question (T_q = 54 tokens) plus four evidence slots (T_s = 33 tokens each), with two packet pseudo-token slots — 132 evidence-slot tokens and 188 total tokens per adapter-active cell call. Execution is staged in a fixed relay: cell 1 publishes from the start-value span without incoming mail, and cells 2–4 each consume the previous packet once while applying their assigned operation or structural forwarder. Communication happens only through packets of two 896-dimensional vectors, which update as RMSNorm(P_in + 0.05 ΔP). A frozen LM head plus a learned mouth projection produces the answer, with h_final = h_base + β·W_mouth(P_4) and β frozen at 0.003. For the 17 task labels, evaluation uses contextually residualized logits; ordinary-language preservation is checked separately on uncorrected full-vocabulary logits.

The only difference between the two arms is the attention mask. In the restricted arm, a cell may attend only to its own assigned evidence slot and the shared question, leaving foreign spans with exactly zero direct attention influence. In the global arm, all four slots are readable. In the restricted arm, the mask simultaneously blocks foreign evidence and uniquely identifies the cell's own slot.

Ten twin pairs (five initializations × two data-order streams) trained for exactly 20,000 updates on a task world generated and sealed before any training or outcome inspection. Seeds and hashes were fixed in advance: operator seed 6011, split seed 2203 (split hash 94e506d408b6def1), and phrasing-grammar seed 7717 (grammar hash def14eb4182e2949). Machines had to reproduce two bit-exact golden fingerprints (restricted c7fcb21aad26951b, global b7c6157b1b3b79af) before contributing runs.

The task is ordered natural-language function composition over Z_17, using 12 affine bijections and programs of length 0–3. Training sampled identity episodes at probability 0.10, single-operation episodes at 0.25, and two- and three-operation episodes at 0.325 each. Evaluation used held-out ordered programs and held-out phrasings: the depth-two held-out set contains 12 programs and the depth-three set 60 programs, each evaluated over all 17 start values and 4 phrasing rotations, giving 816 and 4,080 scored episodes per checkpoint. Chance accuracy is 1/17 ≈ 0.0588.

Beyond simple deletion tests, the authors performed causal interchange interventions indexed by the semantic intermediate value: a donor packet from a different episode replaced the recipient's packet, with the donor chosen so the running value either matched or differed by a known target.

Why This Matters

The paper argues that information availability itself can redirect which solution training discovers — restriction acts as a basin-shifting inductive bias rather than a necessity claim. It also demonstrates that aggregate performance gains and destructive channel ablations are insufficient evidence that a communication channel carries semantic content; natural same-value and counterfactual-value transplants are needed to establish what a relay state actually encodes. The admitted confound is important: because the mask also identifies each cell's slot, attribution to evidence visibility alone awaits a role-marked control.

Potential real-world applications (these are implications the reader might draw, not demonstrations in the paper):

• Multi-agent LLM pipelines where modules currently share full context, offering a candidate design lever for encouraging reusable intermediate representations rather than whole-task memorization.
• Distributed or partitioned data settings in which each agent legitimately has access to only part of the input and must exchange compact state to complete a task.
• Auditing learned communication channels to check whether exchanged states carry the semantic variable they are supposed to carry, using value-indexed transplants as a diagnostic.
• Budget-constrained adapter training, where restricting what each module reads may shift optimization toward compositional strategies without adding parameters.

Industry relevance: the practical claim is narrow but pointed. Adding more context to every module is not automatically better; the shape of what each module can see may matter as much as how much it can see. The results also give teams a concrete protocol for testing whether a latent message channel is doing real semantic work or merely acting as an episode-fingerprint. The authors caution that the system is an explicitly task-gated specialist, not an always-on zero-regression language-model retrofit, and that the centralized controls were neither pair-matched nor compute-matched.

Future Directions

• A role-marked control. Run a fresh comparison in which both arms receive an explicit current-cell or own-slot marker, isolating foreign-evidence access from role disambiguation. Until then, the treatment confounds three coupled factors: foreign-evidence access, role information, and active-context load.

• Collision-free task worlds. The current benchmark contains composite-map collisions, which reduce its discriminative purity. The authors recommend collision-free constructions and report a small residual redundant-versus-novel advantage separately.

• Quantifying code types rather than case-studying them. Only one high-performing global model was available for the code-type comparison, and four restricted checkpoints remain unaudited. A larger sample would estimate how frequently each visibility regime produces a value-indexed versus episode-entangled interface.

• Testing whether role cues, evidence access, or context load drives the code-form difference, and whether the observed effect and packet protocol transfer to other architectures, task families, budgets, or cross-family packet interfaces.

Target Audience

Researchers working on multi-agent and multi-module language-model systems, emergent communication, compositional generalization, and mechanistic interpretability will get the most from this paper. It is also relevant to practitioners designing latent-relay or shared-context LLM pipelines who need to know why destructive channel ablations are not sufficient evidence of semantic communication. The paper requires comfort with attention masking, LoRA fine-tuning, interchange interventions, and preregistered experimental design, but its central comparison — same system, same seeds, only the mask differs — is stated clearly enough for an intermediate reader to follow.

Authors’ abstract

Multi-module systems often expose every module to the full input. We test whether restricting evidence visibility changes which solutions gradient-based training discovers. Four-cell societies share one frozen pretrained language model and one low-rank adapter, communicating only through two model-width continuous vectors in a fixed relay. On a prospectively sealed natural-language function-composition task, we train ten matched restricted/global pairs sharing initialization bytes, training order, token layout, parameters, and computation; only the attention mask differs. Restricted societies outperform their globally visible twins by at least 20 points at both depths in 9 of 10 pairs, with median paired advantages of 0.7648 and 0.6050. Cutting communication reduces every restricted society to chance, and the depth-three advantage remains 0.558 on programs whose composite function never appeared in training. Across six audited restricted societies, same-value packet transplants preserve behavior at 0.94-1.00 across all tested interfaces; destructive interventions collapse performance; and counterfactual packets redirect outputs toward the mathematically predicted answer. The sole high-performing global model also requires communication, but its same-value packets are not interchangeable across episodes. Restricted visibility is thus not necessary for composition; under this protocol it substantially increases the probability of a generalizing relay and favors a reusable, value-indexed interface. The complete preregistered battery nevertheless formally fails because restricted-arm median depth-three accuracy is 0.6988, below the 0.70 floor. An earlier qualification cohort likewise yielded 0/10 complete passes: one model met every task-performance gate, but all ten failed ordinary-language preservation, confining the system to explicitly task-gated use.