SIMIT: Self-Improving Vision-Language Models via Imagination at Test-Time

AI summary

Overview

Research area: Computer vision and multimodal machine learning, specifically test-time adaptation and self-improvement for vision-language models (VLMs).

Technical level: Intermediate. The paper assumes familiarity with VLMs, in-context learning, and fine-tuning, but its central idea (a model generating its own practice problems) is explained in an accessible way.

Scope: The paper introduces SIMIT, a framework in which a single vision-language model generates its own labeled practice data for an unlabeled test query and then improves itself either in-context or by fine-tuning, evaluated across 17 multimodal benchmarks using BAGEL-7B.

What This Paper Is About

Vision-language models are trained on expensive human-labeled data, but they cannot anticipate every task or failure mode they will meet after deployment. The authors study test-time self-improvement: can a model get better on an unlabeled test set using only itself, with no ground-truth answers, no teacher model, and no external reward model? SIMIT answers this by having the model imagine similar problems — writing new questions and committing to their answers first, then generating or rendering images that support those answers — so the model never needs to correctly solve the target query in order to build useful training signal.

Key Contributions

1. Imagination-based self-supervision. The same model creates its own query-conditioned training samples by inventing similar problems, avoiding the need to predict labels for the original test inputs. Answers are fixed before images are constructed, which turns label acquisition into conditional construction rather than reusing possibly wrong predictions.

2. General multimodal realization pipeline. The method combines the model's native image generation with an extensible library of structured rendering skills (documents, charts, diagrams), plus multi-stage verification, difficulty filtering, and confidence-adaptive budget allocation, to produce synthetic data spanning natural and structured visual tasks.

3. Flexible adaptation modes with broad evaluation. The self-generated samples can be used for in-context learning (SIMIT-ICL) or fine-tuning, either per-query or over the whole test set, and the two can be combined (SIMIT-FT→ICL). Across 17 benchmarks with BAGEL-7B, mean relative gains over zero-shot are 6.98% for SIMIT-ICL, 1.69% for single-query SIMIT-FT, 6.39% for test-set SIMIT-FT, and 7.20% for SIMIT-FT→ICL.

4. An analysis of why consensus-style self-supervision underperforms. The paper shows that majority-vote and self-reward based label-free reinforcement learning mainly sharpen predictions the model already gets right, while SIMIT corrects more initial errors and produces supervision with a lower error rate.

Main Findings

• Best overall configuration: SIMIT-FT→ICL achieves a +7.20% mean relative gain over the BAGEL-7B baseline across 17 benchmarks at 55× runtime, and the training-free SIMIT-ICL reaches +6.98% at 25× runtime. Domain-level results for SIMIT-FT→ICL are +6.9% (Image), +7.7% (VQA), +1.6% (Math & Science), +1.6% (Document & Infographic), and +3.2% (Multi-disciplinary).

• Beats prior test-time methods: Label-free reinforcement learning baselines reach +0.59% (TTRL, K=32, 75×), +1.82% (EMPO, K=32, 58×), +2.07% (TTRV, K=32, 41×), and +0.87% (SRLM, K=4, 251×). Test-time scaling baselines are weaker still: Self-Consistency peaks at +0.41% (K=32, 19×), Self-Refine is negative at every iteration count tested (−2.90%, −2.87%, −2.84%), and BAGEL's thinking mode degrades scores (−21.59% at 1,000 tokens, −21.28% at 10,000 tokens).

• Large gains on individual benchmarks: SIMIT-ICL raises NoCaps CIDEr from 0.755 to 1.162 (+53.9%), while test-set SIMIT-FT raises OK-VQA accuracy from 37.6 to 54.2 (+44.0%). SIMIT-ICL improves 16 of the 17 benchmarks and ties one.

• Computation alone does not explain the gains: At 25× the zero-shot runtime, SIMIT-ICL gains +6.98%, versus at most +2.07% for LFRL baselines that run longer than any SIMIT variant.

• Consensus rewards reinforce systematic errors: The base model often wrongly answers "unanswerable"; for VizWiz, "unanswerable" is the majority vote in 99.2% of TTRL training steps, and 79.3% of the few steps with nonzero group-relative advantage still reinforce it.

• Error correction versus disruption: Against a Self-Consistency (K=32) partition of queries, SIMIT corrects 14.4% of initially wrong predictions while disrupting only 2.0% of initially correct ones (Δ = +12.4pp). The strongest LFRL baseline, TTRV, corrects 9.3% at the same disruption rate, and the others have Δ ≤ +5.3pp.

• Cleaner supervision: A blind external evaluator (Claude Sonnet 5) judged SIMIT's positively-encouraged supervision to have a roughly one-third error rate relative to baselines (12% vs. 33–38%). Its error-concentration measure δ is small and negative (−6.5pp), whereas all LFRL baselines are large and positive (+22 to +79pp). SIMIT is the only method with more correct than incorrect supervision on queries the base model answers incorrectly.

• Every pipeline component matters (ablation): Full test-set SIMIT-FT scores +6.39% with a data quality score of 86.0 at 253 minutes. Removing adaptive budget allocation drops it to +3.27%; removing budget allocation and difficulty filtering gives +4.35%; further removing diversity prompting gives +3.47%; removing the skill library gives +1.69%; removing verification gives +3.08%. The skill library matters most, because native BAGEL generation often fails to produce legible text or requested structure.

• Joint question and image synthesis wins: At matched sample counts, generating new images for original questions gives +1.57%, generating new questions for original images gives −2.14%, and generating both gives +4.35%.

• Synthetic data can rival real annotated data: Compared against RICES (retrieval of real neighbors with Qwen3-VL-Embedding-2B) and TTT-NN (single-query fine-tuning on retrieved neighbors), SIMIT's synthetic samples outperform the human-annotated counterpart for certain adaptation modes and domains.

• Gains transfer to other models but are strongest when targeted: SIMIT-ICL improves SenseNova-U1 by +2.21% and Lance by +3.97% over zero-shot. Cross-benchmark transfer yields mean relative gains of 4.3% within domains and 0.5% across domains.

• ICL and fine-tuning are complementary: ICL leads on 8 benchmarks and test-set fine-tuning on 9; their combination performs best overall.

Methodology in Plain English

The model is given a test image and question with no answer key. It follows a pipeline:

1. Decide how much practice to do. The model checks its confidence in its zero-shot answer (mean token probability) and allocates a synthesis budget K accordingly. Less confident queries get more synthetic samples. The rule is a log-odds evidence criterion that picks the smallest budget predicted to reach a desired correctness probability, capped at a maximum.

2. Invent practice problems. Conditioned on the test query, the model writes K triplets: a new question, the intended answer, and a textual description of an image that would support that answer. Diversity-encouraging prompts vary the examples while keeping the task and answer format. Crucially, the answer is fixed before any image exists.

3. Route each description to the right renderer. The model classifies whether a description needs a natural image or a structured visual (chart, document, diagram). Natural images go to the model's own image generation; structured ones go to a skill library.

4. Realize the images. Rather than writing Python/LaTeX/HTML directly, the model emits a compact specification (e.g., a circle's center and radius, or graph nodes and edges), and a deterministic backend computes geometry, quantities, and layout. This gives syntax checks and explicit error feedback, and skills can compose (e.g., HTML embedding a generated image).

5. Verify and repair. The model acts as critic, checking that each image actually supports its question-answer pair. Failures trigger repair or regeneration with error feedback and bounded retries.

6. Filter by difficulty. Examples whose intended answer is too confidently or too weakly predicted are discarded using a band-pass rule with thresholds T_low and T_high, removing likely-noisy and low-value samples.

7. Adapt. The surviving samples become few-shot demonstrations (SIMIT-ICL, inference only) or fine-tuning data, either per query with a reset between queries (SIMIT-FT Single-Query) or pooled across the test set (SIMIT-FT Test-Set), optionally followed by in-context prompting on the fine-tuned model (SIMIT-FT→ICL).

Evaluation uses 50 validation examples and 500 evaluation examples per benchmark (850 and 8,500 total), with hyperparameters such as (ε, A0, A1, B, K_max) and (T_low, T_high) chosen by Bayesian optimization on validation data before evaluation. Each benchmark also supplies a 2,000-example annotated pool (34,000 total) used only for the real-versus-synthetic comparisons.

Why This Matters

Impact on research. The paper reframes test-time compute: instead of spending more tokens on one answer, the model spends compute producing targeted practice data that persists as either context or weights. It also provides a diagnostic argument for why consensus-based self-training can lock in mistakes, supported by the 99.2% "unanswerable" statistic on VizWiz and the stratified correction/disruption analysis.

Real-world applications:

• Automatic annotation: improving model outputs on unlabeled image collections where no human labels exist.
• Personalized assistants: adapting to a specific user's stream of tasks without collecting a labeled dataset.
• Document, chart, and diagram understanding: structured visuals are common in enterprise workflows, and the skill library directly targets this weakness of unified multimodal models.
• Low-latency-restricted or privacy-sensitive deployments: the single-query variant can discard synthetic data after answering and does not retain parameter updates across queries.

Industry relevance. SIMIT-ICL needs only inference access, making it applicable to API-accessible models where fine-tuning is infeasible. The self-contained setting — no teacher, verifier, retriever, or external generator — matches deployment conditions where only the model itself is available.

Future Directions

• Extending and generalizing the skill library, since removing it caused the largest ablation drop (+6.39% to +1.69%) and native generation still fails at structured visuals.
• Closing the supervision-error gap, since synthesized triplets are not guaranteed clean (12% error) when generated images fail to align with their intended description.
• Improving single-query adaptation, whose gain of +1.69% is far below test-set fine-tuning's +6.39%, and understanding when cross-query transfer is unavailable or undesirable.
• Testing generalization more broadly, since model-transfer results cover only SenseNova-U1 (+2.21%) and Lance (+3.97%) beyond BAGEL-7B, and cross-domain transfer was weak (0.5%).
• Reducing runtime cost, given 25× to 55× baseline runtime for the reported configurations.

Target Audience

Researchers and practitioners working on vision-language models, test-time adaptation, self-training, or synthetic data generation. It is most useful to readers who already understand in-context learning and fine-tuning and want a concrete alternative to majority-vote pseudo-labeling, as well as engineers considering adaptation of API-only or proprietary models where no labeled data or external teacher is available.

Authors’ abstract

We introduce SIMIT, a test-time self-improvement framework in which a vision-language model creates its own query-specific training data before answering an unlabeled test query. Our goal is to improve performance on a given target test set without annotations or external models. To pursue this goal, some test-time methods improve answers through longer reasoning or repeated sampling but leave the model unchanged. Others update the model using rewards based on majority agreement among multiple candidate answers or the model's own judgments of correctness, but such rewards can reinforce existing mistakes. SIMIT instead imagines similar problems whose solutions it controls: a model need not correctly solve the target query to construct useful practice examples. It synthesizes diverse (question, answer, image) triplets by specifying each answer before realizing its image. To support diverse multimodal tasks, SIMIT combines native image generation with an extensible skill library for structured visuals, including documents, charts, and diagrams. Within this pipeline, multi-stage verification checks that images support their assigned answers, while difficulty filtering retains informative examples. Adaptive budgeting further targets this practice by allocating more samples to queries the model is less confident in answering. The resulting data can improve the model in-context (SIMIT-ICL) or in-weight (SIMIT-FT). Across 17 diverse benchmarks, SIMIT outperforms existing self-improvement methods. Its best configuration achieves a +7.20% mean relative gain over BAGEL-7B, while training-free SIMIT-ICL achieves +6.98%.