LOB-Impact Benchmark: Emergent Market Impact in Generative Limit Order Book Models

AI summary

Overview

Research area: Quantitative finance / market microstructure, specifically the evaluation of generative limit order book (LOB) simulators, with connections to reinforcement learning for optimal execution and to the empirical market-impact literature.

Technical level: Advanced. The paper assumes familiarity with limit order book message data, autoregressive state-space sequence models (S5, Mamba-3, Gated-DeltaNet), the square-root law of metaorder impact, and Hurst/autocorrelation diagnostics.

Scope (one sentence): The paper introduces a three-phase benchmark that injects synthetic metaorders into the generated message streams of five neural LOB generators across five NASDAQ stocks and measures whether a realistic market-impact law emerges, using a blind control to separate the model's reaction from the mechanical impact of the book.

What This Paper Is About

Generative models of limit order books are currently judged by how closely their output looks like historical data — matching spread, volume, inter-arrival and imbalance distributions. But such models are increasingly proposed as "world models" in which reinforcement-learning trading agents are trained, and an agent trained in a simulator will learn a wrong policy if the simulator misprices how the market responds to the agent's own orders. Distributional metrics are blind to this response. The paper builds a benchmark that asks a counterfactual question — what happens to the generated market when a large execution order is pushed through it? — and applies it to 13 models on five stocks.

Key Contributions

1. A three-phase, counterfactual impact benchmark for message-level generators. Phase 1 (free generation) checks whether order-flow persistence is compensated by price diffusivity; Phase 2 (metaorder injection) measures whether an impact law exists and extracts its exponent δ and amplitude Y; Phase 3 (post-execution silence) measures relaxation. The benchmark is released as open source.

2. A blind control that isolates indirect impact. Every build-up injection is run twice on the same windows and seeds: once visible, and once blind, where the orders fill in the simulator but neither model input (message stream, book state) carries them. The visible-over-blind ratio attributes the response to the model's learned reaction rather than to standard book mechanics.

3. An automatic per-stock calibration and an existence-before-magnitude estimator. Child size is the stock's median market-order size, spacing follows the participation identity η = 1/(1 + Δ_N·φ) with η = 10%, giving Δ_N = 9/φ, and the child count is fixed a priori at N = 100. A law is reported only where a least-squares fit over a δ grid identifies one; the quoted number uses 18 quantile bins with signed conditional means and OLS through positive means.

4. The empirical result that impact emerges but is not the square-root law, and does not relax. Across 25 model–stock pairs (25 identifications), neural generators trained only for next-token prediction produce impact exponents from 0.63 to 1.21 — slightly above the traditional 0.5 — with amplitudes far above the empirical Y ≈ 0.5, and no observed decay in the post-execution window.

Main Findings

• Impact emerges from next-token prediction alone. All five neural generators produce an identifiable impact law on every one of the five stocks: 25 model–stock pairs, 25 identifications. These models were trained solely on next-token prediction, with no impact-related objective.

• The exponent is too steep. Measured exponents δ span 0.63 to 1.21 (e.g., S5-120M on MSFT at 0.63 ± 0.11; Mamba3-4k on GOOG at 1.21 ± 0.11), slightly above the 0.5 of the square-root law. On AMD the values are 0.66 ± .05 (S5-120M), 0.79 ± .06 (S5-4k), 0.87 ± .06 (Mamba3), 0.90 ± .05 (GDN) and 0.93 ± .05 (Mamba3-4k).

• The amplitude is far too large. Neural amplitudes Y range from 1.70 to 49.52 across the table, against the measured empirical value of Y ≈ 0.5 (Tóth et al. 2011; Zarinelli et al. 2015; Bucci et al. 2019a). On AMD alone Y runs from 2.24 (S5-120M) to 8.57 (GDN).

• The response is the model's, not the book's. Visible-over-blind peak ratios (v/b) range from 3.4 to 38, all above the 3× threshold. This matters because the kernel-based replays, which contain no model, are themselves identified on AMD at δ = 0.73–0.81 — inside the neural range — so the exponent alone cannot attribute causality.

• No relaxation is observed. In the post-execution phase, end-over-peak ratios on AMD run from 0.88 to 1.36, where real metaorders relax to one-half to two-thirds of their peak within ten execution durations (Moro et al. 2009; Bucci et al. 2019b). The neural decay kernel comes out permanent (γ ≈ 0) by two independent methods: schedule ordering and an impulse fit. The authors tentatively attribute this to the still-short context windows of the models; only GDN's drift, and the kernel baselines with resilience typed in, bring the curves down.

• Every neural generator fails the flow-balance claim (C1). On AMD the real stream has persistent trade signs (γ_flow = 1.66) with a diffusive mid-price (H = 0.54). The Mamba family carries longer flow memory (γ_flow = 1.20 and 1.06) with a too-diffusive price (H = 0.58 for both); the S5 family carries shorter memory (1.90, 1.83) with near-real diffusivity (H = 0.55, 0.54); GDN has the panel's longest memory (0.79) with H = 0.49. Every neural generator also drifts when left alone, from +9 (S5-4k) to −251 (Mamba3-4k) ticks per rollout.

• Distributional accuracy does not imply interactive fidelity. On AMD the Historic replay has the lowest LOB-Bench scores (KS .060, L1 .162, WS .096) of any row, yet it has no impact at all and fails the existence claim C2. Conversely the neural models score worse distributionally while being the only rows that generate a law of their own.

• The negative and positive controls behave as designed. The Historic replay passes Phase 1 perfectly but yields no law; the Heuristic, Propagator and OW replays are identified (AMD: 0.81, 0.76, 0.73) but fail the square-root and relaxation claims; the parametric models (CST, NMZI, Hawkes, QR) mostly fail to produce a law, with QR identified on some stocks.

• Estimator validation is reported. Planted laws of known exponent are recovered across δ ∈ [0.5, 1.0] with a small downward bias (at most 0.11 at neural amplitudes), so reported exponents are conservative; in 500 null trials with nothing planted, the identifiability test returned not one false law; and the kernel baselines' built-in resilience is recovered in the correct order on all five stocks.

Methodology in Plain English

The team takes a generative model that has learned to emit exchange messages one token at a time, and treats it as a small synthetic market. They first let it run undisturbed and check whether it respects a basic accounting identity of real markets: trade signs are highly persistent, so prices can only stay diffusive if the impact of each trade dies away. A model with persistent flow and a trending price has a free lunch built in.

They then perform the intervention that matters. After conditioning the model on a window of real market data, they insert one aggressive "child" order of median trade size, then let the model generate a fixed number of messages before inserting the next child, and so on for 100 children. The spacing is calibrated per stock so the metaorder is 10% of market-order flow during execution — the same convention used in broker data. Because a simulator, unlike history, can be re-run, they also run the identical experiment blind: the orders fill in the matching engine but are hidden from the model, so the model behaves as if nothing happened. The difference between the two runs is the model's genuine reaction to being traded against.

The measured quantity is the mid-price displacement after each child, plotted against cumulative participation as a power law I = a·q^δ. The estimated δ is the impact exponent and Y the dimensionless amplitude. The buy and sell experiments are combined by antisymmetrisation, I = ½(I_buy − I_sell), which also cancels direction-independent drift. Existence of a law is tested before its magnitude, and a number is printed only where the fit is sharp, the amplitude is above a floor, and the peak clears three standard errors.

Finally, for relaxation, they inject a short 10-child metaorder and then stay silent for 100 blocks — ten execution durations — and ask whether the price drifts back toward where it started. Sample sizes are part of the protocol: 1,024 rollouts per direction per phase over 20 trading days, with the trading day, not the rollout, as the independent unit for bootstrap intervals.

Why This Matters

Impact on research. The paper shifts the evaluation question for generative market models from "does the stream look real?" to "does the stream respond to me like a real market?" It provides a reusable instrument, with per-stock calibration and validated estimators, and shows that a model can score well on existing distributional benchmarks such as LOB-Bench while having no counterfactual response at all. It also links two literatures that rarely meet: empirical metaorder-impact estimation and generative sequence modelling.

Real-world applications:

• Training RL execution agents. An agent learning to schedule a parent order inside a simulator will exploit any mispricing of its own impact; a benchmark that measures the impact law is a prerequisite for trusting such training.
• Simulator procurement and model risk. A firm buying or building a synthetic market simulator can run this protocol on its own names instead of relying on distributional fit.
• Pre-trade cost model development. Synthetic markets that reproduce a plausible impact law, and eventually relaxation, offer a counterfactual test bed for execution algorithms where historical data cannot answer what would have happened.
• Market design and stress exploration. Because the generator can be re-run, one can probe responses to participation levels or order sizes that do not appear in the historical record.

Industry relevance. The calibration is deliberately taken from practitioner conventions — a 10% participation target inside the 3–40% range retained in broker databases, child size at the median market order, a 100-child metaorder matching the 73–100 child orders and 96–98 trades per hidden order reported empirically. That makes the pass criteria interpretable to execution desks and to anyone validating a simulator before deployment.

Future Directions

• Longer context windows. The authors tentatively attribute the absence of relaxation to the models' still-short context windows (500 to 4,000 messages against build-up rollouts of roughly 40k messages on AMD), so extending context is the obvious test of whether resilience can emerge at all.
• Inducing resilience rather than waiting for it. Only the kernel baselines with resilience typed in produced post-execution decay, which raises the question of what training signal or architectural inductive bias would make a generative model absorb and then release liquidity.
• Correcting the exponent and amplitude. The measured laws are systematically steeper (0.63–1.21) and far larger in amplitude (Y from 1.70 to 49.52 against ≈ 0.5) than reality, so a natural next step is an objective or fine-tuning procedure that targets the impact surface directly.
• Broadening the panel. The reported evidence covers five NASDAQ stocks and 13 models, with EA out of distribution for the neural corpus; extending to more names, other venues, and interactive agent-in-the-loop training is left open by the released benchmark.

Target Audience

Researchers and practitioners in market microstructure and machine learning for finance who build, evaluate, or procure generative limit order book simulators; reinforcement-learning researchers using simulators as world models for trading and execution; quantitative execution researchers familiar with the square-root law and metaorder impact estimation; and model-validation teams who need an out-of-distribution test that goes beyond matching historical distributions. Readers without a background in order book mechanics or sequence models will find the paper demanding.

Authors’ abstract

Realistic market impact is essential for generative limit order book (LOB) models to serve as world models for trading agents. If a simulator misprices this mechanism, agents learn policies that are optimal in the simulator but wrong in the real market. Existing metrics assess generated streams purely through distributional comparison with historical data, leaving counterfactual response to an agent's own orders largely unexplored. Here, we show that market impact emerges naturally in neural LOB generators trained solely for next-token prediction. We inject execution-style metaorders into the generated message flow of five autoregressive models across five NASDAQ stocks and measure the response using estimators from the empirical impact literature, validated on baselines whose physics is known in advance. Our benchmark evaluates any message-level generator in three phases: free generation (reading order-flow persistence against price diffusivity); metaorder injection (yielding the impact law's exponent and amplitude); and a post-execution window (measuring relaxation). In addition, comparing each visible injection to a blind, simulator-only control isolates the model's learned response from standard book mechanics. For every model on every stock, we measure how the impact responds as a function of cumulative metaorder participation. Interestingly, we observe emergent impact in generative models of market data, with exponents in the range of 0.63 to 1.21, slightly above the traditional 0.5 of the square-root law of impact. In the family of models tested, we are so far unable to observe impact decay during relaxation. We tentatively attribute this to the still-short context windows of the models. We release the benchmark as open source to validate future generative models for market data.