Every Pixel Is an Option: A System One Model for Medical Ultrasound Biometry

AI summary

Overview

• Research area: Medical computer vision — ultrasound image analysis, anatomical landmark detection, and vision-language models for clinical biometry.
• Technical level: Intermediate. Readers should be comfortable with vision transformers, LoRA fine-tuning, and the idea of dense feature-to-query matching, but the paper's core idea is conceptually simple.
• Scope: The paper introduces Kev-Bio, a single vision-language framework that converts clinical measurement questions into spatial queries to localize caliper endpoints across nine ultrasound views, evaluated on the FU/GU Biometry 2026 benchmark.

What This Paper Is About

Ultrasound biometry requires a clinician to place calipers on precise anatomical landmarks so that distances, angles, or circumferences can be computed — a step that varies between operators and is hard to automate because speckle noise, shadowing, and probe orientation obscure structures. Existing automatic systems rely on task-specific prediction heads, so each new measurement or anatomical view needs its own decoder or parameter set. The authors ask whether a single model can take a plain clinical measurement question as input and directly point to the relevant pixels, producing both landmark coordinates and the resulting clinical measurement.

Key Contributions

1. A question-conditioned localization framework (Kev-Bio). Clinical measurement questions are encoded into "decision states" at special <decide> tokens inside a language model, and each state is turned into queries for the individual caliper endpoints of that measurement — replacing fixed, task-specific output heads with a text-driven interface.

2. A shared dense feature map fused from three encoders. Visual features from Qwen3.5, an ultrasound foundation model (USFM), and DINOv2 ViT-L/14 are aligned, normalized, concatenated, and upsampled by a convolutional "Pixel Neck" into one feature map shared across all measurement tasks.

3. A language-conditioned pixel pointer with spatial probability outputs. Endpoint queries are dot-producted against the dense feature map, passed through a spatial softmax, and decoded to coordinates via differentiable soft-argmax — so the model yields an explicit spatial likelihood map per landmark rather than only a coordinate.

4. Joint supervision over points, heatmaps, and measurements. Training combines an L1 pointer loss normalized by image diagonal, a cross-entropy heatmap loss against a Gaussian centered on ground truth, a geometric loss on the derived clinical value normalized by the training-set median, and an auxiliary Huber-loss regression head used only during training.

Main Findings

• Best average accuracy among reported methods. Kev-Bio reaches 22.50 pixels mean radial error (MRE) and 15.38 mean absolute error (MAE) on clinical measurements, versus 23.88 MRE for the strongest baseline (CAMS) and 17.47 MAE for the strongest baseline (FMUB), averaged over five folds on the nine-view FU/GU Biometry 2026 training set.

• Best landmark localization on eight of nine views and lowest measurement error on six. These gains hold with a comparable parameter count of 35.0 million trainable parameters.

• Large improvement in fetal head biometry. MRE on the HC view drops from 17.03 pixels (IMPACT-L) to 13.76 pixels.

• Baselines win on three individual measurement tasks. Competing methods achieve slightly lower measurement errors on femur length, cervical length, and angle of progression, though Kev-Bio retains the lowest average error on both metrics.

• Results without extra unlabelled data. The reported numbers are obtained without the additional 191,170 unlabelled challenge frames that CAMS and FMUB leverage.

• The pixel pointer is the most important component. Replacing spatial matching with direct coordinate regression from decision states degrades MRE from 22.50 to 35.50 pixels and MAE from 15.38 to 23.64, the largest drop in the ablation.

• DINOv2 and the language model both contribute. Removing DINOv2 gives 24.85 MRE / 16.94 MAE; replacing language-model decision states with learned image-conditioned queries gives 23.66 MRE / 16.77 MAE.

• Probability maps serve as a usable uncertainty signal. Treating predictions more than 5% of the image diagonal from ground truth as errors, inspecting the 10% most uncertain predictions captures 40% of errors (versus 10% under random selection), with error-detection AUROCs of 0.70 for cardiac, 0.76 for prenatal, and 0.86 for intrapartum imaging.

• Different questions produce different spatial maps. On the same image, queries for ventricular length versus atrial width in the A4C view highlight different endpoints, while related fetal head measurements respond consistently at shared landmarks.

Methodology in Plain English

The model receives four things at once: the ultrasound image encoded by Qwen3.5's vision encoder, the same image encoded by an ultrasound-specific foundation model, a short text description of the anatomical view, and a list of clinical measurement questions. Each question is wrapped between a <q> token and a <decide> token, and the whole sequence is processed in a single forward pass of a Qwen3.5-0.8B language model initialized with the pretrained Kev adapter.

The trick is what happens at each <decide> token. Its hidden state — the model's internal representation of "I have now read this measurement question" — is converted by a small network into several query vectors, one per caliper endpoint (each measurement needs between 2 and 4 landmarks). A learned embedding is added to distinguish, say, the first endpoint from the second, and the queries are normalized.

Separately, a dense feature map is built for the image by combining features from three encoders chosen for complementary strengths, resizing them to a common resolution, and fusing them through the Pixel Neck while re-injecting shallow convolutional features for fine detail. Each endpoint query is compared to every location in that map by dot product, scaled by a learnable temperature, and turned into a probability distribution over pixels with a spatial softmax. Coordinates come out as the expected position under that distribution (soft-argmax), which keeps the whole thing differentiable.

Because the coordinates are differentiable, the model can compute the actual clinical value — a distance, an angle, or an ellipse circumference — and compare it directly against the ground-truth measurement. Training therefore penalizes three things simultaneously: how far the predicted points are from annotated points, how far the probability map is from a Gaussian bump at the true landmark, and how wrong the derived measurement is. An auxiliary head that regresses coordinates straight from the decision state adds extra supervision during training only; at inference, everything comes from the pointer and the geometry functions.

The setup uses LoRA adapters throughout (rank 16 for the language model and the Qwen/USFM encoders, rank 32 for DINOv2), trains for 24 epochs with AdamW at weight decay 0.05 and effective batch size 8, and runs on NVIDIA H100 GPUs with 80 GB memory. Loss weights are 0.1 for heatmap, 0.2 for geometric, and 1 for auxiliary.

Why This Matters

Impact on research. The paper demonstrates that the "System One" formulation — scoring candidate locations rather than generating text — transfers from decision-support tasks to spatial localization. It also shows that a single shared localization head conditioned on language can substitute for the per-task decoders that dominate ultrasound biometry pipelines, and that probability maps provide a free calibration signal that coordinate-only regressors cannot offer.

Real-world applications:

• Prenatal screening, where head circumference, abdominal circumference, and femur length are routine measurements currently dependent on operator caliper placement.
• Echocardiography, where views such as apical four-chamber and parasternal long-axis require many simultaneous distance measurements per image (8 and 11 respectively in this dataset).
• Intrapartum assessment, where the angle of progression is measured transperineally and is the largest single view in the benchmark at 4,000 labelled images.
• Point-of-care and low-resource settings, where less-experienced operators could use uncertainty maps to know which automated caliper placements need manual review.

Industry relevance. A 35.0M-parameter trainable footprint built on an 0.8B backbone is small enough for practical deployment, and the natural-language interface means new measurement protocols could be added through instructions rather than retraining new heads — attractive for ultrasound vendors who support many probe types and clinical workflows.

Future Directions

• Generalization beyond the benchmark. The authors state that generalization to unseen measurement instructions, unseen anatomical structures, and different imaging conditions remains unexplored, since evaluation covers only predefined measurements on a single benchmark.
• Uncertainty calibration. The entropy-based error detector is analyzed with AUROC and capture curves, but the authors list uncertainty calibration as explicit future work rather than a solved problem.
• Computational efficiency. Efficiency is named as a future direction, which matters given that three separate visual encoders feed the dense feature map.
• Exploiting the unlabelled data. The 191,170 unlabelled frames shipped with the challenge were not used here, while two baselines did use them — leaving open how much further the approach could go with semi-supervised or pretraining use of that pool.

Target Audience

Researchers and graduate students working on medical image landmark detection, vision-language models, or ultrasound AI who want a concrete architecture for instruction-driven spatial localization. It is also relevant to clinical AI engineers and ultrasound product teams evaluating whether a single shared model can replace task-specific biometry pipelines, and to clinicians interested in how predicted spatial uncertainty might flag caliper placements for review. Readers without background in vision transformers or parameter-efficient fine-tuning will find the method section dense, though the central idea is accessible.

Authors’ abstract

**Purpose:** Ultrasound biometry is widely used across various clinical applications, including prenatal and cardiac examinations, to obtain quantitative measurements by placing calipers at anatomical landmarks. However, these measurements remain subject to interoperator variability as they rely largely on manual caliper placement. Conventional methods often use specialized prediction heads for different ultrasound tasks, whereas vision-language models offer flexibility through text instructions. However, direct coordinate prediction does not explicitly model spatial likelihood distributions, limiting the representation of localization ambiguity. **Methods:** We propose Kev-Bio, a vision-language model that uses clinical measurement questions to guide ultrasound landmark detection. Inspired by System One decision models, Kev-Bio uses decision representations of the language model to generate queries for each caliper endpoint. These queries are matched against shared dense ultrasound features to produce spatial probability maps, from which landmark coordinates and clinical measurements are derived. The model is trained end-to-end using both landmark localization and clinical measurement losses. **Results:** On the nine-view FU/GU Biometry 2026 dataset, Kev-Bio achieves a mean radial error (MRE) of 22.50 pixels and a clinical measurement mean absolute error (MAE) of 15.38, outperforming the compared baselines in overall performance. **Conclusion:** These results demonstrate the effectiveness of Kev-Bio for ultrasound landmark detection and clinical measurement across different anatomical views. Our code and models are available at https://github.com/moeinheidari7829/Kev-Bio.