Skip to content
AI.info

AI agents

Context Engineering and Attention Budgets

Design context assembly as a budgeted retrieval, ordering, and authority process.

By the end you can

Example

Legal tools that retrieve the authority and still hallucinate between 17% and 33% of the time

Three commercial legal research products retrieve the controlling authority into the model's context before they answer. They still hallucinate between 17% and 33% of the time. Retrieval is the architecture that is supposed to make a missed controlling clause impossible. The products are the ones a firm actually buys: Lexis+ AI, from LexisNexis, and Westlaw AI-Assisted Research and Ask Practical Law AI, both from Thomson Reuters. The number comes from the first preregistered empirical evaluation of retrieval-augmented commercial legal research tools, published in 2025 in the Journal of Empirical Legal Studies.

The finding is one sentence in the abstract: “we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time” — Magesh and colleagues, 2025.

The vendors had claimed the opposite: that retrieval 'eliminates' or 'avoids' hallucinations and guarantees 'hallucination-free' legal citations. One in three to one in six answers says otherwise. Putting the governing document inside the window is not the same as the answer being governed by it. What enters the context, where it sits, and what authority it carries has to be engineered rather than assumed.

  • Decision at stake: Design context assembly as a budgeted retrieval, ordering, and authority process — the step these three products already perform, and still get wrong between 17% and 33% of the time.
  • Hidden assumption: That retrieval into a large window guarantees use. That is the assumption sold as 'hallucination-free', and the preregistered measurement of it returned 17% to 33%.
  • Primary control question: Context can contain contradictory instructions with different authority — which retrieved passage governs the answer, and can the runtime say why it was the one that governed?
  • Evidence to collect: The context contains enough evidence for the next decision without duplicating the full project — measured, as Magesh and colleagues measured it, on tasks with a known correct citation rather than on how fluent the answer reads.

Visual

Four layers in a single call, budgeted against the usable window and not the advertised one

What reaches the model in one call arrives in four layers: Authority layer, Current state, Selected evidence, and Action space. The Action space and Compression should not share an owner, and should not share a test.

The budget those layers are cut against is not the number on the model card. NVIDIA built a benchmark, RULER, to ask the real size directly. Eight authors, 17 long-context models, 13 tasks across four categories: retrieval, multi-hop tracing, aggregation, question answering. Each was run at 4K, 8K, 16K, 32K, 64K and 128K tokens. What they found: “While these models all claim context sizes of 32K tokens or greater, only half of them can maintain satisfactory performance at the length of 32K.” — Hsieh and colleagues at NVIDIA, 2024.

So the advertised window and the usable window are two different numbers. For half the models tested they diverged at 32K — a quarter of a 128K claim. Size the five layers against the length at which the model you are actually calling still passes retrieval, tracing, aggregation and question answering. Everything beyond that is space you are billed for, not space you can spend.

FigureProcess · 5 steps
  1. 1

    Authority layer

    System policy, task contract, and user instructions with clear precedence.

  2. 2

    Current state

    Verified facts, pending actions, budgets, and open questions.

  3. 3

    Selected evidence

    Retrieved passages, tool results, and artifacts relevant to the next decision.

  4. 4

    Action space

    Only tools and transitions valid for the current state.

  5. 5

    Compression

    Summaries and references that preserve critical evidence within budget.

More context costs more and can be used less

Agent context is the working set supplied to one model call: instructions, task state, observations, memories, tools, and artifacts. More context increases cost. It can also reduce effective use through distraction, conflict, and position effects.

Position has been measured. A 2024 paper in TACL, Lost in the Middle, moved a single answer-bearing document through a multi-document context and traced a U-shaped curve. Accuracy was highest when the document sat at the beginning or the end, lowest when it sat in the middle. “For example, GPT-3.5-Turbo’s multi-document QA performance can drop by more than 20%—in the worst case, performance in 20- and 30-document settings is lower than performance without any input documents (i.e., closed-book performance; 56.1%).” — Liu and colleagues, TACL 2024. Read that last clause twice. A context containing the answer scored below the same model given nothing at all.

Padding alone is enough, with the task held fixed. Same Task, More Tokens, an ACL 2024 Outstanding Paper, changed only the quantity of irrelevant text around an unchanged question and its unchanged evidence. “We show that LLMs quickly degrade in their reasoning capabilities, even on input length of 3000 tokens, which is much shorter than their technical maximum (on average over all tested models, a drop in accuracy from 0.92 to 0.68).” — Levy and colleagues, ACL 2024. Twenty-four accuracy points, bought with nothing but volume, at 3,000 tokens.

Context engineering decides what enters, where it appears, how it is labeled, and which source has authority. The canonical task record should remain outside the prompt.

Padding the window is paid for twice: once at the billing line, and again in the twenty-four points between 0.92 and 0.68.

Analogy

A Surgical Tray Prepared for One Procedure

A surgical tray holds only the instruments and records the current operation needs, while the hospital keeps the full archive elsewhere. Bringing every available item into the room can hinder work.

How much it hinders has a number. One irrelevant sentence added to a grade-school maths problem is enough to break models that were already solving it. That is the GSM-IC benchmark, built by Shi and colleagues and reported at ICML 2023. “We use this benchmark to measure the distractibility of cutting-edge prompting techniques for large language models, and find that the model performance is dramatically decreased when irrelevant information is included.” — Shi and colleagues, ICML 2023. Every prompting technique they tested took a significant performance drop. Fewer than 30% of the base problems were still consistently solved once the distractors were there. The mitigation that made the difference was not a bigger window. It was an instruction: 'Feel free to ignore irrelevant information given in the questions.'

One extra sentence on the tray, and under a third of the operations that used to go right still do. A surgeon reaches for the scalpel that is on the tray. A model can leave a critical item unread even when the context around it is small.

Context quality depends on selection, authority, and ordering—not raw volume.

Key idea

Context can contain contradictory instructions with different authority

Tool outputs, web pages, old plans, and memories may include imperative language. If the prompt does not preserve origin and precedence, untrusted content can look like new policy.

This is not a stylistic worry. It is a catalogued attack class in two independent security taxonomies. NIST's adversarial machine learning taxonomy, AI 100-2e2025, indexes it as NISTAML.015 and names the mechanism: “Because GenAI models combine the data and instruction channels, attackers can leverage the data channel to affect system operations by manipulating resources with which the system interacts.” — NIST AI 100-2e2025, March 2025. The OWASP Gen AI Security Project ranks the same class first on the OWASP Top 10 for LLM Applications 2025, as LLM01:2025 Prompt Injection. It defines 'Indirect Prompt Injections' as those arriving from external sources such as websites or files. That is exactly the material a retrieval step puts into the window.

It has already happened in a shipped product. On 11 June 2025 Microsoft's Security Response Center published an advisory for CVE-2025-32711, known as 'EchoLeak' and titled 'M365 Copilot Information Disclosure Vulnerability'. Microsoft rated it Critical, with a CVSS 3.1 base score of 9.3 (AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:L/A:N), and classified it as CWE-74 injection. The advisory records it as not publicly disclosed and not exploited, and states the service had already been fully mitigated with no customer action required. The description is one sentence, carried verbatim as the NVD description: “Ai command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network.” — Microsoft Security Response Center, 11 June 2025.

So: label source and authority, keep policy outside retrieved content, and expose untrusted text as data rather than instructions.

The budget this is spent inside is measurable too. Anthropic instrumented its own research system and reported on 13 June 2025 that “agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens as chats.” Any framework has to fit those calls into a fixed window, and LangChain's short-term memory documentation states the consequence: “a full history may not fit inside an LLM’s context window, resulting in a context loss or errors.” Choosing what to keep, and what authority to give it, is what stands between those two facts.

Once retrieved text can issue orders, whoever wrote that text is directing the agent — NISTAML.015, LLM01:2025, CVSS 9.3.

Steps

Build a context budget for one decision

One decision, not one project, is the unit a context budget is built for. Name the next decision — the action the model must take now. Reserve authority space: stable policy and task constraints go in a protected layer that retrieved text cannot occupy. Select evidence: only items that can change the current decision. Remove stale material — superseded plans, duplicate results, irrelevant tool descriptions. Then test recall under load.

That last step has a published protocol rather than a gesture. NoLiMa removed the lexical overlap that lets needle-in-a-haystack tests be passed by string matching, so a system has to associate rather than match. It was run against 13 LLMs that claim at least 128K-token contexts. “At 32K, for instance, 11 models drop below 50% of their strong short-length baselines. Even GPT-4o, one of the top-performing exceptions, experiences a reduction from an almost-perfect baseline of 99.3% to 69.7%.” — Modarressi and colleagues, ICML 2025.

Copy the construction, not the conclusion. Build distractor sets where the critical evidence does not share wording with the query. Insert them at the length you actually ship. Require that the decision still changes when the evidence changes. A budget that works exposes “Context can contain contradictory instructions with different authority”. What it leaves behind is evidence for “The context contains enough evidence for the next decision without duplicating the full project”. The number 11 of 13 is what happens when nobody runs the test.

FigureProcess · 5 steps
  1. 1

    Name the next decision

    Choose the action the model must make now, not the entire project.

  2. 2

    Reserve authority space

    Place stable policy and task constraints in a protected layer.

  3. 3

    Select evidence

    Retrieve only items that can change the current decision.

  4. 4

    Remove stale material

    Drop superseded plans, duplicate results, and irrelevant tool descriptions.

  5. 5

    Test recall under load

    Insert distractors and verify that critical evidence still affects the choice.

One compaction destroys a prompt that holds the only copy

Treat context as a compiled view of durable state. It can be regenerated, tested, and compared rather than becoming the only place where task knowledge exists. That is what makes the measurements above actionable instead of depressing. If the window is a build output, you can rebuild it: the answer-bearing document moved out of the middle, the padding that cost 0.92 to 0.68 removed, the retrieved text demoted from instruction to data.

Whoever assembles the context owns “Context can contain contradictory instructions with different authority” — the class NIST indexes as NISTAML.015 and OWASP ranks as LLM01:2025. The evidence that agent context engineering is working is “The context contains enough evidence for the next decision without duplicating the full project”, demonstrated on a NoLiMa test at shipping length. It is not inferred from a context window that was large enough to hold everything.

If the prompt is the only copy of what the task knows, one compaction destroys it.

Key takeaways