Jeva: System One Models from First Principles

AI summary

Overview

Research area: Architecture design for "system one" language models, which produce a single typed decision (choose one, answer absent/present, or emit a score) rather than a chain of generated reasoning.

Technical level: Advanced. The paper pairs a concrete neural architecture with formal propositions and proofs about set functions, permutation equivariance, and caching, though the prose explanation of the architecture is readable without following every derivation.

Scope: The paper proposes Jeva, a decision-native architecture that separates reusable option encoding from set-conditioned request refinement, and evaluates it on typed decisions, null detection, candidate scaling to 2,048 options, and cross-request caching.

What This Paper Is About

Current open-source system one models typically serialize a request and its candidate answers into one joint text sequence and read a decision off answer-token logits or a prediction head. The authors argue this design couples semantic representation, candidate interaction, and answer availability into a single prediction process, which makes decisions sensitive to option ordering, ties output space to a fixed answer symbol set, gives no principled way to say "none of these apply," and forces recomputation of unchanged candidate content on every request.

Jeva's goal is to make four properties explicit in the computation itself: null awareness, permutation invariance, scalability to variable candidate counts, and reuse of option-side computation across requests. It unifies three output types — Choice, Noul, and Score — in one architecture.

Key Contributions

1. A decision-native architecture. A shared language encoder independently maps the request and each option into compact vectors; a compact decision stack refines only the request state against an immutable option memory, so option representations carry no request information and remain reusable.

2. A theoretical account of why state refinement is necessary. Proposition 1 shows that with a single decision block and fixed parameters, the same two options A and B can be preferred differently depending on whether a third option is C₊ or C₋, a reversal that no independent scorer with set-independent tie-breaking can reproduce.

3. Coverage separated from competition. Proposition 2 shows that Ja: a coverage gate ρ = σ((max_i s_i − b_cov)/τ_cov) is unchanged when options scoring below the maximum are added, whereas a flat softmax with a null score contributes a log K term to the null decision. This makes answer absence distinct from uncertainty among candidates and supports an explicit null outcome.

4. Candidate symmetry and exact reuse. Proposition 3 states that permuting candidates and their metadata permutes candidate probabilities while leaving the final state, null probability, and typed Noul/Score outputs unchanged, and that substituting identical cached entries leaves all outputs unchanged — with no order-consistency loss required. Prepared-memory decision cost is O(L(d² + Kd + df)), linear in the candidate count K, and a bank of M options stores O(M(L+1)d) values.

Main Findings

• Typed decision quality. With a Qwen3.5-4B backbone and one decision block, Jeva-4B leads seven of eight slices in full-coverage accuracy (Table 1), exceeding 90% on BANKING77 and MASSIVE at both candidate counts (BANKING77 K=8: 92.05; K=16: 91.62; MASSIVE K=8: 91.09; K=16: 90.22). On Mind2Web it reaches 79.99%, 62.25%, and 66.23% on cross-task, cross-website, and cross-domain selection. On Typed Decisions it reaches 78.30%, compared with InternDecision-4B's 78.70%.

• Order stability. Across 600 Choice examples evaluated in original, reversed, and up to three seeded random orders, Jeva records zero flips, mean total variation of 3.17×10⁻⁸, and identical original and all-order accuracy of 76.67%. Jev-1.13.0, InternDecision-4B, and JevK5 show flip rates of 17.00%, 22.67%, and 26.17%, with all-order accuracies of 66.67%, 64.50%, and 47.83%.

• Null detection. On 3,080 matched BANKING77 requests with 16 candidates (588 answer-absent), Jeva reaches 98.66% null-detection AUROC, versus 93.61% for KEV-BF16, 84.27% for InternDecision-4B, and 79.44% for JevK5. It detects 93.88% of absent answers at a 5.02% false-positive rate.

• Typed fidelity. On 600 Noul and 800 Score examples (Table 2), Jeva achieves the lowest Noul Brier (0.0184) and MAE (0.0939), and the lowest Score MAE (0.2315). Its Score RMSE of 0.3222 is close to InternDecision-4B's 0.3208.

• Scaling to thousands of candidates. On 500 ToolRet requests across K ∈ {32, 128, 256, 512, 1024, 2048}, Jeva covers all requests with 78.40% aggregate accuracy versus 64.00% for recursive JevK5, scoring each complete set in one decision-head pass. Both InternDecision variants support only 65/84 requests at K=32.

• Batching and latency. With length-aware batching at 256 rows/131,072 padded tokens, p50 latency falls from 31.730 to 7.519 s at K=1024 and from 94.459 to 12.040 s at K=2048 (7.85×). At K=2048, Jeva achieves 69.88% accuracy versus JevK5's 67.47% at 28 s.

• Robustness to set expansion. On the same 83 queries as nested candidate sets expand from 32 to 2,048 options, Jeva remains correct at every size on 58 queries, versus 48 for JevK5 and 7 for CLM. Its all-size accuracy is 69.88%, exceeding JevK5 by 12.05 points.

• Candidate reuse. Over 50 matched MLDR queries at 0%, roughly 50%, and 100% option reuse, hybrid caching achieves 1.95× and 27.98× speedups in p50 latency at partial and full reuse, while exact-set caching remains near 2.1 s. At full reuse p50 falls from 2,102.0 ms to 75.1 ms, with a median paired speedup of 27.86×.

• Shared-bank reuse. On Typed Decisions, p50 falls from 93.1 ms to 46.6 ms (set cache) and 46.7 ms (hybrid cache), avoiding 99.2% of candidate-row encodings. Primed MLDR p50 falls from 2,097.3 ms to 73.9/74.2 ms. No decisions change across conditions; the maximum absolute probability difference is 0.00736.

Methodology in Plain English

The authors start from a simple complaint: if you paste a request and a list of options into one text and read the answer from the model's output tokens, then the options are being processed as part of the question. That means reordering them can change the answer, adding options can require new answer symbols, and you cannot tell "none of these is right" from "I'm unsure which of these is right."

Jeva splits the problem in two. Each option is encoded on its own with a shared language encoder, using a template that contains only the decision type and the option text — no request, no option index, no other option. The request is encoded separately. The option vectors are then frozen for the forward pass and act as a memory that the request state reads from, block by block, using attention. Because the read is a weighted sum over options and the option side never changes, the output is a permutation-equivariant set of candidate probabilities: reorder the options and the probabilities reorder with them. The attention weights have a clean interpretation as the solution to an entropy-regularized allocation problem over candidate evidence, balancing compatibility against spread of evidence.

The authors prove that a single such block is already more expressive than independent scoring: because the request state is built from the whole set, a candidate's score can shift based on which other candidates are present, reversing a pairwise preference that a fixed independent scorer could never reverse.

Answer availability is handled by a separate gate. A single scalar ρ estimates the probability that at least one acceptable option exists, computed from the maximum candidate score, a learned threshold, and a learned temperature. That ρ splits the final distribution into null mass and candidate mass, and because it depends only on the maximum, adding low-scoring options does not move it. The same machinery produces Choice (pick a candidate), Noul (read the positive candidate's probability), and Score (average rubric values weighted by candidate probabilities), with rubric values and the positive designation changing interpretation rather than adding encoder inputs.

Training is direct supervised learning. For open-set targets, the loss is a binary cross-entropy on the coverage logit plus a weighted cross-entropy over candidate probabilities, so all-null examples supervise availability without imposing a spurious ranking; closed sets use candidate cross-entropy. Jeva-4B uses Qwen3.5-4B with pooled width 2560, decision width 768, one block, 8 heads, and FFN width 3072, trained with rank-64 LoRA over both projections and the decision head (13.37M parameters), with separate AdamW parameter groups and global gradient clipping. Training follows a curriculum covering typed decisions, intent selection, verification, reading/reasoning, and large-candidate retrieval, and LoRA is merged for serving.

Evaluation uses Typed Decisions (2,000 test decisions), BANKING77 and MASSIVE with 8 or 16 candidates, three Mind2Web slices selecting from 16 supplied candidates, ToolRet cohorts, and MLDR. Baselines keep their documented interfaces and native runtimes; local latency uses synchronized H200 timing with the checkpoint and query stream fixed for within-model comparisons.

Why This Matters

The paper's contribution is structural rather than a pure accuracy win. It argues that four properties — null awareness, permutation invariance, option scalability, and cacheability — should be properties of the computation rather than things hoped for from training data, and it shows concrete mechanisms and proofs for each. If the argument holds, it reframes the design of decision-making models away from "serialize everything and read the logits" toward separating reusable content from set-dependent judgment, in the same way dense retrieval separated question encoding from corpus encoding.

Real-world applications:

• Tool selection at scale. A model that picks one tool from a registry of thousands of candidates, where the registry persists across requests and only the request changes.
• Long-document identification. Selecting among document candidates where option-side computation can be cached and reused, as measured on MLDR.
• Abstention-sensitive deployment. Systems where "none of the available labels apply" must be a first-class outcome, such as intent routing on user requests outside the supported label set.
• Rubric-based scoring. Producing a numeric score by weighting candidate rubric values, relevant to evaluation and grading pipelines.

Industry relevance: The latency numbers matter for serving. Reusing prepared option memory avoids re-encoding unchanged candidates, and the paper reports avoiding 99.2% of candidate-row encodings on shared-bank Typed Decisions. The linear-in-K decision cost means a single pass over a large candidate set can replace recursive selection, which is directly relevant to retrieval and routing systems that currently chain candidate filtering steps.

Future Directions

• Accuracy at very large candidate sets. At K=2048, Jeva reaches 69.88% accuracy, leaving headroom even as latency and coverage improve. Whether the architecture can push accuracy higher at thousands of options without recursive fallback is open.

• Joint argmax behavior under set growth. The paper notes that set refinement can still change scores and that the joint argmax over the full distribution need not remain fixed when candidates are added (Appendix A.3), so the interaction between coverage and competition under expanding sets needs further characterization.

• Ties and finite precision. The symmetry and reuse propositions are stated under deterministic exact arithmetic; the paper defers discussion of ties and finite-precision effects to Appendix B, which suggests a practical reliability question for deployed systems.

• Extension beyond the tested output types. The architecture unifies Choice, Noul, and Score, but the evaluated suite covers typed decisions, intent selection, verification, reading/reasoning, and large-candidate retrieval. Other decision shapes are untested.

Target Audience

Researchers and engineers working on efficient language model inference, retrieval-augmented routing, and structured prediction, particularly those building systems that must pick one item from a large, changing or persistent candidate pool. It is also relevant to readers interested in how permutation-invariant set architectures (Deep Sets, Set Transformer, Perceiver, ColBERT-style late interaction) transfer to decision tasks. Readers wanting only the practical takeaway can read the introduction, Section 3.3, and Section 4; the propositions and their proofs are for those evaluating the theoretical claims.

Authors’ abstract

System one models enable direct, low-latency decision-making. Yet recent practice of predicting an answer from serialized text does not, by itself, respect the structure of a decision problem. We argue that current designs underemphasize four important structural requirements. (i) null-aware: Models should distinguish answer absence from uncertainty among candidates; (ii) permutation-invariant: Reordering alternatives should not change the decision; (iii) option-scalable: Growing candidate sets should not require redefining the output space; (iv) cacheable: Request-independent computation over recurring alternatives should be reusable across requests. We introduce Jeva, a decision-native architecture that addresses these requirements jointly. A shared encoder independently represents the request and each option, while a compact decision stack refines only the request state against immutable option memory. Symmetric aggregation yields permutation-equivariant probabilities; shared scoring accommodates variable-size candidate sets. A separate coverage estimate distinguishes answer absence from candidate competition, enabling explicit null outcomes. Request-independent option representations and layer projections support reuse across newly assembled sets. These mechanisms unify Choice, Noul, and Score outputs. Across evaluations, Jeva combines strong typed-decision performance with near-perfect null detection and no decision flips under any tested permutations. It scales to thousands of candidates and substantially reduces latency through cross-request reuse without changing decisions.