The Pulse
Nokia Presents AnyJev to Stabilize LLM Decisions Without Training
Nokia Applied Research’s AnyJev is an open-source decision layer that uses an LLM’s next-token distribution to produce typed choices and thresholdable probabilities without additional model training.
AI.info Team ·
Nokia Applied Research’s AnyJev is an open-source software layer designed to make large language models more dependable at bounded decisions such as routing, yes-or-no checks and ordinal scoring.
The project reads an LLM’s next-token distribution without generating an answer or parsing free-form text. It then addresses two sources of instability that the repository identifies in ordinary logit-based classification: sensitivity to the order of the options and confidence scores that do not reliably reflect the chance of being correct.
AnyJev is published under the Apache-2.0 license. The repository lists Jiamu Zhang, Tianze Yang and Liang Wu from Nokia in Sunnyvale, California, along with Yucheng Shi from Tencent Hunyuan, as its authors.
Why option order matters to AnyJev
Many LLM applications reduce a model’s job to selecting one item from a fixed set. A developer may restrict the next token to labels such as “billing,” “technical” or “sales,” then treat the resulting softmax values as probabilities. Nokia’s repository says the approach can produce different answers when the same options appear in a different order.
AnyJev’s L0 method addresses that problem by presenting the options in K rotations, so each candidate occupies each position once. It averages the position bias over those rotations and divides out the label prior, reducing the effect of ordering and of a model’s tendency to favor a particular answer independently of the input.
The library supports three typed decision primitives: choice for selecting from up to 26 options, noul for a yes-or-no decision with a probability for the true value, and score for an ordinal result. The README lists transformers support, while vLLM and SGLang serving are on the roadmap rather than part of this release.
Qwen3-8B benchmark shows a wider automation threshold
Nokia reports its clearest benchmark on Qwen3-8B using 300 test items from the 20-class BANKING77 task. With raw logits, reversing the options changed the answer on 23% of items. AnyJev L0 reduced that rate to 7.3%, while L1, which adds calibration from labelled examples, recorded 7.7%.
Accuracy rose from 0.747 with raw logits to 0.803 with L0 and 0.807 with L1. Expected calibration error, a measure of the gap between stated confidence and observed correctness, fell from 0.240 to 0.184 with L0 and 0.095 with L1.
The largest difference appeared in the repository’s coverage-risk measure. At a threshold of no more than 5% error, raw logits classified only 7.7% of the test traffic as safe to decide automatically. L0 reached 46.3%, while L1 reached 52.0%. Nokia describes the result as a point estimate from 300 items and warns that the interval is wide.
The reported coverage figure is not a universal automation rate. AnyJev’s documentation says teams should measure it on their own task, model and traffic distribution.
L1 adds calibration, not new model knowledge
AnyJev’s levels separate bias correction from probability calibration. Raw mode performs a restricted softmax over the permitted label tokens. L0 needs no labels and removes position and label-prior bias, but it does not guarantee that the model’s uncertainty is calibrated.
L1 applies temperature scaling after L0 and requires between 100 and 500 labelled examples for each question. It reshapes confidence without changing the ranking of the choices.
The additional stability costs computation. A K-option choice at L0 requires K prefills. Nokia reports about 0.25 seconds per decision at batch size 32 for 20 options on one H100. The package can be installed with pip install "anyjev[hf]".
The repository’s own limits
AnyJev does not improve a model’s underlying ability to answer a question. The documentation says calibration cannot repair a model that lacks the required knowledge, and reports cases in which no readout beats the trivial baseline on maze edges and Minesweeper.
The label-free prior can also reduce accuracy when one answer dominates the data. Nokia says the current implementation is limited to 26 options in its letter-based readout, with span readout planned to raise that ceiling. The benchmark tables are based on Qwen models; Llama and Gemma rows are listed on the project roadmap rather than presented as completed evaluations.
The repository presents AnyJev as a decision layer around an existing open model rather than as a replacement model. Its practical value depends on whether a team needs a threshold it can inspect and test before allowing an automated route, approval, escalation or tool call to proceed.
Nokia’s AnyJev repository contains the implementation, benchmark files, roadmap and the project’s stated limitations.