Certified Access: Test-Time Compute on Conservative Physics-Structured AI Memory

AI summary

Overview

Research area: Artificial intelligence — specifically test-time compute (adaptive inference) on a physics-structured latent memory, combining symplectic/Hamiltonian dynamics, certified guarantees, and empirical evaluation on learned associative memories.

Technical level: Advanced. The paper leans on symplectic integrators, Jacobian determinants, Hamiltonian mechanics and conformal symplecticity; the empirical sections are more accessible but presuppose comfort with latent-variable models and risk calibration.

Scope in one sentence: The paper formalizes test-time compute on a Latent Dynamics Model memory as "certified access" — three conditional certificates (phase-volume Jacobian, bounded energy change, latch-transport law) verified on designed analytic testbeds and then mapped onto trained memories.

What This Paper Is About

Adaptive inference mechanisms — retries, confidence-based escalation, non-local routing — are usually bolted onto trained models as black boxes that add forward passes or halting heads, with no structural account of what the extra compute buys or what it costs the model's stability guarantees. This paper instead treats the model's memory as a conservative phase-space state (q, p) advanced by a damped symplectic (velocity-Verlet) integrator of a learned Hamiltonian, and asks what each access mechanism provably does to phase volume, energy, and stored content. The goal is a certificate stack — stated theorems under explicit assumptions — verified numerically on a designed testbed and then examined on trained memories, with the authors explicitly stating that the main contribution is the formalization and verification, not benchmark performance.

Key Contributions

1. A taxonomy of reachability failures. Failures are split into kinematic reach failure, where the target lies outside a causal box C_T with half-widths L_i = T ε c / sqrt(M_i), and energetic escape failure, where the target is inside C_T but shielded by a potential barrier ΔV_b. Different mechanisms with different certificates resolve each.

2. Verification of the certificate stack. On an analytic testbed with dimensions 2 and 4, 5 seeds, and oracle placement, the exactness claims (Jacobian determinant, energy change, latch-transport law) hold to machine precision, while re-absorption and channel-hop thresholds hold to leading order. Reported checks include relative error 2.0 × 10^-12 for the causal cap, 2.2 × 10^-16 for state-dependent Jacobian deviations, and latch transit error below 1.8 × 10^-15.

3. Empirical measurement of latch transport (state erasure vs. transport). An untrained state-replacing map has det J = 0 exactly and erases the stored Goldstone charge; the certified gated channel transports it with preserved spread. Phase volume alone is shown to be insufficient — a random shift also has det J = 1 but scrambles the spread.

4. Calibrated compute rationing and its scaling limits. A self-calibrating rationing gate on a trained CLU memory reduces relaxation steps by 4.81 ± 0.44× at capacity kv16 under a 400-epoch budget, and the paper maps this across capacity and epoch budget, including the finding that a parameter-matched physics-free router beats the energy-gated non-local edge in both FLOPs and accuracy.

Main Findings

• Volume certificate is conditional. For damping frozen within a step, the update is conformally symplectic with det J = (1 − γ)^d in d dimensions. The state-dependent governor γ_n = s·tanh(max(0, H − E*)) adds derivative terms, so the certificate does not cover the governed map.

• A causal box bounds reach, but only in relativistic mode. The drift sub-step moves each coordinate by strictly less than ε c / sqrt(M_i) for every momentum and every potential, giving Q_T(z_0) ⊆ C_T(q_0). The bound is energy-blind — it involves only (ε, c, M_i), not V_θ or |p|. In Newtonian mode no such box exists, which is why the Newtonian control crosses it.

• The Lorentz squeeze cures escape, not reach. The mass-weighted squeeze is exactly symplectic (det = 1) and injects energy bounded by e^{2|ζ|} H, re-absorbed by the governor in about 2ζ/γ_c steps. Measured det S^(M) = 1.000 (± 4 × 10^-6). Validated ratios against the bound: (ζ, ratio, bound) = (0.25, 1.13, 1.65), (0.5, 1.55, 2.72), (1.0, 3.79, 7.39), (2.0, 27.5, 54.6); on a quartic well the raw ratio can exceed the matched-quadratic bound.

• Squeeze reach cost scales badly with distance. Reaching just beyond C_T requires energy growing quadratically in the excess distance (exponential in rapidity ζ, not exponential in distance itself). At p_0 = 1.2, landing at d = 4.0 needs ζ ≥ 2.0105 and d = 5.0 needs ζ ≥ 2.64, both exceeding the swept budget ζ ≤ 2.0; the squeeze landing rate drops to zero at d ≥ 4.0 while d ≤ 3.2 remains covered.

• The gated channel reaches globally at det J = 1. The canonical translation q ↦ q + Δ, p ↦ p has Jacobian I_{2d} exactly, with energy cost equal to the endpoint potential difference V(b) − V(a) — measured as exactly 0.0 on the symmetric matched channel, so it does not grow with distance in this testbed. The gate must stay frozen: if the gate varies during the jump, det J = 1 + ∇g(q)^T Δ ≠ 1 (unit-test value 2.05).

• Latch transport law. For the Goldstone charge Q = p^T X q, the channel shifts it by exactly Q' − Q = p^T X Δ, preserved iff Δ ⟂ X^T p. On 16 states from the capture ball (std(Q_in) = 0.0803): the coset-tangent channel gives ΔQ = 0.0 (predicted precision 7.5 × 10^-10) with max error ≤ 1.2 × 10^-7 and std(Q_out) = 0.0803; the across-coset channel gives ΔQ = 0.2500. The state-replacing map gives std(Q_out) = 0 (all states to one bit-identical charge); the random shift has det J = 1 but std(Q_out) = 0.2793.

• det J = 1 with zero energy change is not sufficient for stability. On a non-coercive well V = ½kq_0² − εq_0⁴ with connected component |q_0| < x_b = 3.536, an exit at b = 5.0 has ΔH = 0.0 exactly and still escapes; an energy-only sub-level test wrongly admits it. Escaping arms grow linearly (r*(2T)/r*(T) ≈ 1.9); the screened gated channel holds at r* = 0.09 because it refuses the jump. The state-replacing map coincides exactly with the screen-ignored ablation on all failures, so the screen — not the jump — provides boundedness. An admissible exit at b = 3.0 costs ΔH = 2.88 and stays bounded.

• Rationing helps most at low capacity and under-training. At a 400-epoch budget: 4.81 ± 0.44× step reduction at kv16, 1.57 ± 0.07× at kv24, 1.14 ± 0.06× at kv32. At full convergence (2000 epochs) the gate matches rather than exceeds the full-budget baseline. The Learn-then-Test wrapper held on all 30 cells with coverage 0.647 ± 0.063 at target risk ε = 0.05, resting on an exchangeability assumption the jittered protocol only approximates. The residual adds at most +0.024 AUROC over a standard readout margin.

• Capacity is bounded by epoch budget, not a hard ceiling. Across a 198-job grid, intra-CLU step reductions are 9.9×, 9.5×, 6.2× at kv32, kv64, kv96. kv32 saturates by 1000 epochs; kv96 needs 4000 epochs to reach accuracy 0.975; at 4000 epochs smaller cells over-train and accuracy falls from 1.00 to 0.89.

• A noise wall. Under evaluation cue noise σ ∈ {0.3, 0.6}, gate accuracy at σ = 0.6 on kv32 falls to 0.36 while CLU storage fidelity stays at 1.0 — the governed relaxation over-commits to the corrupted cue.

• A physics-free router wins on routing. On the lattice task, the one-hop non-local edge keeps a flat FLOP count as distance grows (unlike multi-hop diffusion), but the parameter-matched physics-free router outperforms the energy-gated edge: at N = 4, 1.000 ± 0.000 accuracy at 8.81e7 FLOPs versus 0.887 ± 0.139 at 1.18e8; at N = 8, 0.948 ± 0.070 at 8.81e7 versus 0.715 ± 0.172 at 1.18e8.

Methodology in Plain English

The authors build their argument in two layers.

The first layer is analytical and self-contained. They take a memory whose state is a position–momentum pair, evolved by a standard physics integrator (velocity-Verlet, a kick–drift–kick scheme) with a learned potential and a relativistic kinetic term that caps speed. From this structure they derive three properties for each access mechanism: the Jacobian determinant (does the mechanism preserve phase volume?), a bound on injected energy, and a rule for how protected "latch" content is carried along. Each property is stated together with the assumptions under which it holds.

The second layer is numerical verification on a deliberately simple, hand-designed testbed — a symmetric double well with a barrier, plus a rotational sector for the latch — where ground truth is known and channel placement is set by the experimenter rather than learned. They sweep distances inside and beyond the causal box, sweep rapidities for the squeeze, and run a "BIBO battery" that pushes exits past the edge of the potential's coercive component to see whether bounded trajectories survive. They also run an ablation-style comparison against a naive state-overwriting map and against a random shift that happens to preserve volume.

The third layer moves to trained memories: an MQAR key–value recall memory (vocab 256, capacities kv16/24/32), a self-calibrating exit gate whose thresholds are set by a Learn-then-Test wrapper, a 198-job sweep over capacity and epoch budget, and a lattice routing task comparing the gated non-local edge to local-only, dense, diffusion-chain and physics-free router baselines. All compute was run on standard laptop CPUs with JAX 0.9.0.

Why This Matters

The paper's real contribution is a shift in how adaptive inference is justified: instead of reporting that extra compute improves accuracy, it asks what the extra compute provably does to the model's internal state, and it is candid about where those proofs stop (fixed damping within a step, stated testbed potentials, jump-only latch law, per-branch injectivity). Its negative results — the zero-energy escape at b = 5.0, the noise wall, the physics-free router winning — are unusually explicit and give other researchers falsifiable targets.

Real-world applications that follow from the framing:

• Long-context memory systems where retrieval is iterative relaxation: knowing whether an escalation step preserves or destroys stored content is a correctness question, not just a latency question.
• Risk-controlled inference pipelines: the Learn-then-Test wrapper with a target risk ε = 0.05 and measured coverage is directly applicable to settings where a system must certify its own error rate at deployment.
• Compute budgeting on constrained hardware: step reductions of 4.81× to 9.9× against an always-full budget translate into serving cost, and the paper maps where that gain disappears with scale.
• Simulation and scientific computing adjacent to physics: the symplectic-integrator and phase-volume vocabulary, along with the CERN-affiliated author list, points to transfer of structure-preserving numerical methods into learned systems.

Industry relevance: the rationing and routing results are the practical core — the honest finding that a simple physics-free router is cheaper and more accurate than the certified-gated one tells practitioners where not to spend engineering effort, while the certificate stack tells them which guarantees they can and cannot claim.

Future Directions

• Learned entrance-steering for gated channels. Channel placement is oracle-set in the analytic testbed and explicitly out of scope for learning; the paper names learning where to place entrances as future work.
• Beating the noise wall. Gate accuracy collapses to 0.36 at σ = 0.6 on kv32 while storage fidelity stays at 1.0; noise-aware thresholds, longer relaxation budgets and denoising initialization are all listed as open.
• Retry acceptance as a certified Markov kernel. Appendix E sketches four operating constraints derived from the symmetric squeeze being exactly volume-preserving, but the full stationarity certificate is left as a design program.
• Scaling and tightness of the certificates. The energy bound is proved for a quadratic isotropic Hamiltonian, while the causal box uses the relativistic kinetic energy — the paper notes these rest on different kinetic energies and are not a single joint statement; extending certificates to the state-dependent governor and the full routing policy is also open.

Target Audience

Researchers and engineers working on adaptive inference, latent-dynamics memory architectures, or structure-preserving machine learning who want guarantees rather than benchmark curves. It is most useful to readers comfortable with Hamiltonian mechanics and symplectic integration who are evaluating whether physics structure buys anything measurable — the paper's answer is nuanced: it buys exact, checkable certificates at the jump, but not BIBO stability, not noise robustness, and not a routing advantage. Practitioners interested purely in accuracy-per-FLOP leaderboards will find the paper's own conclusion (that a physics-free baseline wins) more informative than its mechanisms.

Authors’ abstract

Test-time compute mechanisms, such as retries, confidence-based escalation and non-local routing, are conventionally deployed as black-box additions to trained models. They typically add forward passes or halting heads, but give no structural account of what the extra compute buys or what it costs against the model's stability guarantees. We therefore propose a theoretical and empirical framework for test-time compute on a Latent Dynamics Model used as the memory. When an encoder's latent state $(q,p)$ is advanced by a damped symplectic (velocity-Verlet) step of a learned Hamiltonian, every mechanism that expands access has an exact phase-volume Jacobian, a bounded energy change and an exact latch-transport law. We split reachability failures into escape failures and reach failures. A Lorentz squeeze resolves escape failures by injecting bounded, re-absorbable energy. A gated canonical translation (a one-step channel across $C_T$) resolves reach failures at $\detJ=1$ exactly. We verify this certificate stack on a designed analytic testbed. Reaching just beyond $C_T$ with the squeeze requires energy that grows quadratically in the excess distance. The gated channel's energy cost equals the potential difference between its endpoints, which is zero on the matched channel measured here, and so does not grow with distance in this testbed. An untrained state-replacing map annihilates phase volume and erases stored charge, while the gated channel transports it. On a learned physics-structured memory, we show calibrated compute rationing and evaluate the scaling limits of non-local routing against a physics-free baseline. The main contribution is the formalization and verification of the certificate stack, not benchmark performance.