Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models

AI summary

Overview

Research area: Natural language processing — prompt engineering, instruction following, and long-context reliability/hallucination evaluation for large language models.

Technical level: Intermediate. The paper is written for readers who already know what system prompts, context windows, retrieval-augmented generation, and benchmark harnesses are, though its conclusions are stated in plain language.

Scope: Two controlled experiments that cross four prompt rendering formats (markdown, plain text, prose, and markdown table) against two scale axes — number of simultaneous instructions (10 to 160) and injected context length (2,000 to 512,000 tokens) — on one deterministic, contamination-free synthetic corpus, across five models from three families.

What This Paper Is About

Practitioners routinely decide how to format prompts and injected documents, how many instructions a system prompt can carry, and how much context a model can hold, but the paper argues there is almost no controlled evidence crossing these decisions against each other. The core problem is that format is normally tested at a fixed scale, and scale is normally tested at a fixed format, so nobody knows whether markdown, plain text, prose, or tables behave the same at ten instructions as at a hundred and sixty, or at two thousand tokens as at five hundred thousand. The goal is to run that joint test on a single held corpus so that any measured difference can be attributed to format and scale rather than to differences in content, and then to release everything for exact reproduction.

Key Contributions

1. A joint decay-and-placement analysis of instruction following. Adherence-versus-instruction-count decay curves are compared across four formats and two placements (system prompt versus user turn) on one instruction pool of programmatically verifiable rules. The authors state that prior work establishes threshold decay at larger instruction counts but varies neither format nor placement.

2. A four-format, six-rung factorial for long-context hallucination. Recall error, false-premise sycophancy, and fabrication of never-stated facts are measured jointly across a 2k-to-512k-token ladder in four formats on a single contamination-free corpus. The paper states it is not aware of a prior study crossing format with context length for all three failure modes simultaneously.

3. A structure-versus-syntax decomposition and token-overhead-adjusted accuracy. Holding content byte-identical across list-structured markdown, a table, unstructured plain text, and unstructured prose separates whether any structure helps from whether tabular structure specifically helps. Real subword token counts are then used to ask whether a format's accuracy advantage survives its token cost.

4. VeyraBench. The full harness, the Book of Veyra corpus generator, and all raw model outputs are released for byte-identical reproduction from a fixed seed. The corpus is described as a positive construction argument against pretraining contamination rather than a de-duplication filter.

Main Findings

• Instruction following collapses to a hard floor by N = 80, regardless of format. Perfect-response rate (all rules satisfied) reaches zero by 80 simultaneous instructions for every model, every format, and both placements, and stays at zero through N = 160. At N = 10 the rates were 0.938 (Sonnet 5), 0.919 (Gemini Flash), 0.850 (Claude Haiku), 0.725 (Qwen 35B), and 0.588 (Qwen 27B); by N = 80 all were 0.000.

• No model shows a reliable markdown advantage. Four of five models show small markdown-minus-plain deltas (at most 2.1 percentage points in the direction that recurs most often) that are not reliably signed across N. The clearest directional signal runs the other way: Qwen 35B favors plain text at five of six instruction levels, widening to 4.8 percentage points at N = 160.

• Placement effects rival or exceed format effects, and their direction is model-specific. At N = 160, user-turn placement helps Claude Haiku (+6.6 percentage points) and Qwen 35B (+5.1 pp), hurts Gemini Flash (−8.7 pp) and Qwen 27B (−1.8 pp), and is statistically indistinguishable from zero for Sonnet 5. Logistic fits shift Claude Haiku's knee from N ≈ 98 (system) to N ≈ 104 (user), while Gemini Flash's knee moves from N ≈ 82 (system) to N ≈ 46 (user).

• One model shows a large, isolated format collapse. Gemini Flash's prose and table formats fall sharply at N ≥ 40: prose-minus-plain is −18.1 pp and table-minus-plain is −13.7 pp at N = 40, and −10.7 pp and −15.0 pp at N = 80. Every other model keeps prose and table within about 2.5 pp of plain. Manual inspection found the affected responses often contained no essay text but a tail fragment of an internal rule-verification pass — a hidden-reasoning-leakage failure rather than a scoring artifact.

• Long-context recall is flat, then degrades in a format-dependent way. At 2k, 16k, and 64k tokens every model and format sits at 0.98–1.00 recall accuracy, with no format discriminating. Separation starts at 128k: Claude Haiku at 128k scores 0.383 on plain text versus 0.817–0.867 on the other three formats, a 48.4-percentage-point spread. Sonnet 5's spread nearly doubles from 11.7 pp at 256k to 20.0 pp at 512k, while Gemini Flash's stays comparatively flat (5.0 pp to 6.7 pp), so spread tracks proximity to each model's own effective ceiling rather than absolute token count.

• Neither pre-registered format ordering holds. Prose — the format predicted to perform worst under both orderings — is best or tied-best in three of the five illustrative cells, while plain is simultaneously the worst performer in one cell and the best in another. The authors attribute part of Sonnet 5's 512k pattern to a rising refusal rate: three in four of its incorrect markdown/512k recall responses are a literal "insufficient information" non-answer rather than a wrong guess.

• Fabrication is exactly zero. Across all 5,760 main-set absent-fact probe responses, spanning every model, every reachable rung, and every format, no model invents a specific answer to a question whose fact was never stated.

• Sycophancy stays negligible. The highest observed false-premise agreement rate in any (model, format, rung) cell is 8.3% (5 of 60 responses, Qwen 35B at 128k/markdown), and most cells are at or under 3%. All 19 sycophantic responses in the dataset were manually reviewed and confirmed as unambiguous agreement with the stated incorrect value.

• What actually rises is outright refusal. Refusal on the false-premise probe climbs toward each model's own ceiling: Claude Haiku rises from 0% at 2k to 89.6% at 128k, and Sonnet 5 climbs to 78.8% by 512k. The paper frames refusal as a distinct failure mode from both sycophancy and fabrication, and notes that which format triggers the most refusal is itself model-dependent.

• Format carries a substantial token cost. Measured with the o200k_base tokenizer, markdown costs 1.258× plain text, prose 1.221×, and table 1.367× — a +22% to +37% range. The paper reports that adjusting for this changes which format is preferable in the cases with genuine accuracy spread to adjust.

• Cross-rung comparisons required an anchor set. Gemini Flash's main-set markdown recall appears to improve from 0.900 at 256k to 1.000 at 512k, but the unchanged anchor-set questions show both rungs already near ceiling, identifying the movement as a question-composition artifact. Every cross-rung claim in the paper is checked against its anchor-set counterpart.

Methodology in Plain English

The authors built one fictional universe, the Book of Veyra: 8,780 uniquely named solar systems, guilds, and creatures, each with a fixed set of attributes. Names are assembled from hand-curated syllable tables and numbers are drawn from fixed ranges, all seeded with the fixed seed 42 and generated with no language model involved, so the corpus — and every downstream file — regenerates byte-identically (verified by matching MD5 checksums across repeated runs). Because every entity name is a novel combination, no model could have memorized it, which makes any correct answer evidence of retrieval rather than parametric recall. Only unguessable facts are scored, keeping the chance baseline near zero, while attribute values intentionally overlap across entities to create genuine interference.

That same content is rendered four ways — markdown subsections with bold key-value bullets, the same lines with markup stripped, flowing prose, and a markdown key-value table — with identical facts, units, and wording, so differences can be attributed to presentation alone. The corpus is then sliced into nested prefixes at six nominal rungs (2k, 16k, 64k, 128k, 256k, 512k tokens), with actual token counts measured rather than estimated.

The first experiment hands each model a block of 10 to 160 simultaneous rules — five fixed structural rules plus word-level forbid/include constraints — and checks compliance with regex or word-boundary matching rather than an LLM judge, with 20 trials per cell. The second poses three probe types at each rung and at three needle depths (10%, 50%, and 90%): a recall question, a false-premise question, and a question about a fact never stated, with 20 questions per probe type per rung and three repeats per question. A fixed 20-question anchor set drawn once from the 2k rung is re-asked unchanged at every larger rung to support valid cross-rung comparison. Models whose context windows cannot fit a rung, after a measured real-token-to-estimate multiplier of 1.1–1.6×, are skipped rather than truncated. This produces 960 calls per model in Experiment 1 (4,800 scored trials total) and 5,520 calls per full-context model in Experiment 2 (30,480 scored responses total).

Why This Matters

Impact on research. The paper's central methodological claim is that format cannot be evaluated independently of scale, and its results support that: several format effects that look like noise at one instruction count or one context length become large and directional at another. It also offers a reusable asset for the field — a deterministically regenerable, contamination-free corpus rendered in four content-identical formats, with the harness and raw outputs released. Two findings push back on active research directions: the paper reports that tables did not separate from unstructured formats on this corpus, contrary to results motivating table-specific hypotheses elsewhere, and that long-context degradation under its false-premise probe takes the form of refusal rather than sycophancy, which changes how a rising "failure" rate should be read.

Real-world applications:

• System prompt design. The finding that user-turn placement helps two models, hurts two, and does nothing for one argues against portable rules of thumb and in favor of per-model testing at the instruction counts actually used. The floor at N = 80 argues against accumulating large rule blocks in a single call.

• Retrieval-augmented pipelines. The 48.4-percentage-point plain-text collapse for Claude Haiku at 128k, and the observation that spread tracks a model's own effective ceiling, means ingestion-script formatting choices should be validated near the context lengths a deployment actually uses.

• Context budgeting. With table costing 1.367× plain text in tokens and markdown 1.258×, format decisions are also capacity and cost decisions, and the paper reports that cost adjustment can reverse which format appears preferable in cases with real accuracy spread.

• Reliability monitoring. The refusal climb (0% to 79–90% across models) identifies a failure mode that a dashboard tracking only "correct" versus "incorrect" would misattribute to comprehension loss, and the reasoning-leakage cases behind Gemini Flash's collapse identify a response-integrity check worth adding.

Industry relevance. The paper was authored at the Machine Human Intelligence Lab (MHIL) and reports using production API endpoints (claude-sonnet-5, claude-haiku-4-5, gemini-3.5-flash) alongside self-hosted open-weight models served through Ollama with extended thinking disabled. It is a preprint that has not undergone peer review.

Future Directions

• Determine whether refusal and comprehension are separable. The paper notes that the absent-fact probe's null cannot distinguish a model that correctly verified a fact's absence from one that reflexively declines everything at that rung, and that refusal is genuinely present at extreme context lengths for at least one model. Isolating these would clarify whether the 0/5,760 fabrication result reflects verification or blanket abstention.

• Explain why format rankings flip. The paper observes that the same format can be best for one model/rung and worst for another and offers two mechanisms — refusal confounding at Sonnet 5's 512k rung and a rung-dependent effect for Gemini Flash — but does not establish a general account of the flip.

• Investigate the reasoning-leakage failure mode. Gemini Flash's model-specific collapse at N ≥ 40, where the visible response field contained a tail fragment of an internal rule-verification pass rather than essay text, is reported as a distinct behavior worth targeted study rather than a scoring artifact.

• Test whether the N = 80 floor holds under other rule types and larger instruction counts. The paper's instruction pool is dominated by word-level forbid/include constraints at high N, and the authors note their range (10–160) is narrower than prior work. Whether structurally different rules, or instruction counts above 160, move the floor is an open question.

Target Audience

This paper is most useful to applied LLM engineers and prompt/pipeline designers who make format, instruction-count, and context-length decisions in production systems; to evaluation researchers and benchmark builders interested in contamination-free synthetic corpora and factorial experiment design; and to model-provider teams investigating long-context reliability, refusal behavior, and model-specific format sensitivity. Readers who want a defensible answer to "should I use markdown or plain text, and how much can I put in one prompt?" will find the paper's main message — that the answer is model-specific and scale-dependent rather than universal — directly applicable, provided they note that the detailed analyses in Sections 7 through 11 are not included in the available text.

Authors’ abstract

Practitioners make three prompt-design decisions with almost no controlled evidence behind them: how to format instructions and context (markdown, plain text, prose, or tabular), how many simultaneous instructions a system prompt can carry before compliance degrades, and how much context a model can hold before recall and honesty degrade. We report two controlled experiments crossing all three factors on one held, contamination-free synthetic corpus (the "Book of Veyra," 8,780 uniquely-named entities, deterministically regenerable from a fixed seed), evaluated across five models. Experiment 1 (960 calls/model) measures instruction-following decay as rule count N grows from 10 to 160, crossed with four formats and system-prompt vs. user-turn placement. Perfect-response rate collapses to zero by N=80 for every model, format, and placement. Placement produces effects at least as large as format at N=160 in most models, but the direction is model-specific. No model shows a reliable markdown advantage; one 35B model favors plain text instead. Experiment 2 (5,520 calls/model) measures recall accuracy, false-premise sycophancy, and absent-fact fabrication across a 2k-to-512k-token context ladder in the same four formats. Recall stays near ceiling through 64-128k tokens, then degrades sharply and format-dependently: one model's accuracy spread reaches 48 points at 128k tokens. Fabrication never occurs (0/5,760 probes), and sycophancy stays negligible (<=8.3%). What rises sharply near each model's context ceiling is outright refusal to answer (0% to 79-90%), distinct from sycophancy or fabrication. Neither pre-registered format ordering holds, and token overhead (+22% to +37% over plain text) further changes which format is preferable where accuracy spread is genuine. We release the full harness, corpus generator, and raw results (VeyraBench): https://github.com/iNetanel/veyrabench