Context Engineering: Selection, Ordering, and Compression

A crafted email was enough. No user interaction at all, and Microsoft 365 Copilot disclosed information over a network. Microsoft published the flaw on 11 June 2025 as CVE-2025-32711, "M365 Copilot Information Disclosure Vulnerability"; it was reported publicly as EchoLeak. The official description is one sentence: "Ai command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network." Microsoft classified it CWE-74 and scored it 9.3 CRITICAL on CVSS 3.1. The National Vulnerability Database scored the same identifier 7.5 HIGH.

That is the shape of the problem this lesson is about. Nothing was missing from the assistant's context. It retrieved what it was designed to retrieve. The failure was in how the package was assembled: an attacker's words arrived inside it, carrying no marker to separate a message the user had received from an instruction the developer had written. A complete context can still be badly assembled. The five failures below are the ways that happens.

Every item should have a reason for inclusion and a declared role. Two of the steps below — filtering and ordering — are measured accuracy variables with published numbers, not matters of house style.

Ordering first. Where a document sits in the context changes how well the model uses it. Accuracy on multi-document question answering follows a U-shaped curve by position: strong at the start, strong at the end, worst in the middle. Liu and six co-authors measured that in 2024, in "Lost in the Middle: How Language Models Use Long Contexts". GPT-3.5-Turbo scored 56.1% closed-book, with no documents at all, and 88.3% in the oracle setting, handed only the single answer-bearing document. Between those two ends the paper reports: "For example, GPT-3.5-Turbo's multi-document QA performance can drop by more than 20%—in the worst case, performance in 20- and 30-document settings is lower than performance without any input documents (i.e., closed-book performance; 56.1%)." The answer-bearing document was present in every one of those runs. Buried in the middle of 20 or 30 others, it left the model below the score it reached with nothing to read.

Filtering next, and the counter-intuitive part is which items do the damage. "The Power of Noise: Redefining Retrieval for RAG Systems", published in 2024, reports: "One counter-intuitive finding of this work is that the retriever's highest-scoring documents that are not directly relevant to the query (e.g., do not contain the answer) negatively impact the effectiveness of the LLM." The same work found that inserting random documents into the prompt improved accuracy by up to 35%. Random text helped. Near misses — high similarity, no answer — hurt. So filtering has to be about relevance to the decision, not about retriever score.

The pipeline below is therefore not a formatting convention. Interpret the request. Collect candidates. Filter by authority rather than by score. Order by function, separating instructions, evidence, examples and untrusted content. Compress selectively, preserving decisive spans, provenance, uncertainty and contradictions. Then emit a manifest recording item IDs, versions, scores, transformations and token allocation.

Choose the method according to the downstream claim. Compression can be measured rather than assumed, and the published numbers are large. LLMLingua reported up to 20x prompt compression with little performance loss in 2023, evaluated on GSM8K, BBH, ShareGPT and Arxiv-March23.

The extractive column in the table below has numbers of its own. LLMLingua-2, published in 2024, treats prompt compression as token classification over an extractive dataset: it decides, token by token, what to keep. Its abstract reports: "Additionally, our model is 3x-6x faster than existing prompt compression methods, while accelerating the end-to-end latency by 1.6x-2.9x with compression ratios of 2x-5x." The evaluation spans MeetingBank, LongBench, ZeroScrolls, GSM8K and BBH. Keeping the original spans is not the slow, cautious alternative to summarising. It has its own compression ratio, its own end-to-end speedup, and it keeps the wording a citation can be checked against.

Length itself is a cost even when nothing is dropped. Levy and two co-authors built FLenQA, a dataset in which only the input length varies, and reported "a notable degradation in LLMs’ reasoning performance at much shorter input lengths than their technical maximum". A context that fits inside the documented window is not thereby a context the model reasons well over.

The three families preserve different things. Extractive selection keeps original spans and removes the surrounding material: strong provenance, exact wording, good for citations and legal clauses, and liable to miss context spread across passages. An abstractive summary generates a shorter representation of several passages: it merges redundant information and suits conversational state, but it can introduce or omit claims, so it needs source-linked validation. Structured state stores decisions, entities, dates, constraints and unresolved fields: compact, machine-checkable, useful for workflows and memory, dependent on a stable schema — and silent about anything its fields do not have a slot for.

The manifest should let another reviewer reconstruct what the model received. For high-risk systems placed on the European market, that has stopped being only good engineering practice. Article 12(1) of the EU Artificial Intelligence Act, Regulation (EU) 2024/1689, provides: "High-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system." Article 19(1) then requires providers to keep those automatically generated logs, to the extent they are under their control, for a period appropriate to the intended purpose and in any case at least six months, unless Union or national law provides otherwise. Six months is a retention floor. A design decision follows from it.

The six recording steps are short. Store each item's source ID, version, access decision and retrieval time. State its role: instruction, evidence, example, user claim, tool result or memory. Capture the transformations applied — chunking, filtering, reranking, extraction, summarization. Track how much of the token budget each role consumed. Preserve contradictions rather than silently merging incompatible sources. And link the output back to the context items that support it.

That last step is the one that has to be measured rather than trusted. ALCE was the first benchmark for automatic citation evaluation, scoring fluency, correctness and citation quality together; Gao and three co-authors introduced it in 2023. Their result on one of its datasets: "For example, on the ELI5 dataset, even the best models lack complete citation support 50% of the time." A citation that names a context item is itself a claim about that item. On ELI5 the best systems could not fully support half of their statements with the passages they pointed at. The manifest exists because that link is a measurable failure mode, not a formality.

Good context packaging preserves the evidence the decision needs while cutting distractors, cost, privacy exposure and ambiguity. Every claim in this lesson points the same way. The answer-bearing document buried among 20 or 30 others left GPT-3.5-Turbo below its own 56.1% closed-book score; supplied alone, it reached 88.3%. The retriever's highest-scoring passages that did not contain the answer hurt accuracy, while random documents helped it by up to 35%. Compression ratios of 2x–5x bought 1.6x–2.9x end-to-end latency without abandoning the original wording. And the strongest systems on ELI5 still failed to fully support half their statements with the passages they cited. Selecting and compressing decide how good the answer is, not only how fast it arrives.

The next lesson turns to output constraints. Once the input interface is built, the application has to define how generated results enter deterministic software.