The Pulse
Apple Paper Uses Hidden-State Probes to Guide Flow Models
A paper posted by Apple researchers on September 16 introduces probe guidance, which uses a small network trained on a flow model’s hidden states. Tests report gains on text generation and multiple-choice benchmarks, including a 16.8-point
AI.info Team ·
Probe guidance raised a 1.7-billion-parameter flow model’s BoolQ score by 16.8 points, according to a paper submitted September 16. The method, introduced by authors whose paper appears on Apple’s machine-learning research site, uses a small network trained on a larger model’s frozen internal states to steer text generation. The authors also report a 1.4-point improvement on the more challenging SIQA benchmark. The results are research evaluations, not a new Apple product announcement.
A 16.8-point rise on BoolQ
The researchers tested probe guidance on a continuous diffusion language model, a type of model that generates text by iteratively refining a noisy representation rather than producing one token at a time. On BoolQ, the model’s score rose from 42.0 without guidance to 58.8 with guidance at the stronger of the paper’s two tested settings. On SIQA, the score increased from 38.4 to 39.8. The paper also evaluates ARC-E, OpenBookQA, PIQA and RACE, reporting gains or parity across its multiple-choice suite.
For readers trying to interpret the numbers, the comparison is between guided and unguided versions of the same model setup—not a claim that the system outperforms leading language models overall. In the paper’s table, the guided model’s mean score reaches 44.5, below the 47.1 reported for Duo, a discrete diffusion model included as a comparison. The authors say guidance brings the continuous model close to parity with discrete approaches in most categories.
A probe reuses states the model already computes
Probe guidance works by training a lightweight multilayer perceptron, or MLP, on hidden states from a frozen model. The probe acts as a weaker version of the main model; at inference, the difference between its output and the main model’s output supplies a direction for adjusting generation. Because the main model has already calculated those hidden states, the guidance step adds an MLP evaluation rather than a second full pass through a separate model.
The paper estimates that probes require about 2% of model compute, and says inference uses 1.4 to 2.0 times fewer floating-point operations than autoguidance, which runs a weaker model alongside the main one. The authors estimate probe training adds about 1% in training compute. Those figures describe the tested approach and comparisons in the paper; they do not establish speed or cost savings for every model or deployment.
Generation gains depend on the evaluation
The team also tested unconditional text generation using two model families: ELF, which generates in a learned latent space, and FLM, which operates on token representations. On the larger ELF model, the authors report that probe guidance lowered generative perplexity from 38 to 23 at an entropy level of 5.40. In this evaluation, lower perplexity indicates higher measured text quality, while entropy tracks diversity. The paper says the guided model improved the quality-diversity tradeoff in its tests.
That result comes from experiments on OpenWebText, where the researchers generated 1,000 samples for evaluation. It should be read as a benchmark result, not evidence that guided outputs are uniformly better for every writing task. The paper’s tests focus on selected model families and scoring methods, rather than real-world use by people.
The authors identify limits in language models
The work also examines why autoguidance—an existing technique that steers generation using a weaker model—can fail on language tasks. In experiments with models trained from scratch, the authors found that guidance depended on using a weak checkpoint from a low-entropy stage, before the model’s entropy rose sharply during training. Later checkpoints failed to provide a useful signal in their tests.
The researchers caution that continuous diffusion language models still underperform standard autoregressive language models, and say the reasons guidance works are not fully understood. Their paper on arXiv presents probe guidance as a way to study and improve these models, while leaving open whether the measured gains will carry over to broader benchmarks and practical systems.