Specificity-aware reinforcement learning for fine-grained open-world classification

AI summary

Overview

• Research area: Computer vision, specifically open-world fine-grained image classification with reasoning Large Multimodal Models (LMMs), using reinforcement learning with verifiable rewards.
• Technical level: Advanced (assumes familiarity with reinforcement learning post-training, GRPO, LMM prompting, and LLM-as-a-judge evaluation).
• Scope: The paper diagnoses why reasoning LMMs give overly generic labels in fine-grained open-world classification and proposes a specificity-aware reinforcement learning framework, SpeciaRL, that improves prediction specificity without sacrificing correctness.

What This Paper Is About

In open-world image classification there is no fixed label set, so a model can answer at many levels of specificity (for example, "flower" versus "daisy"). Reasoning LMMs are usually accurate but tend to give correct-yet-generic answers, and simply pushing them to be more specific increases the number of wrong answers. The paper asks how to steer these models toward predictions that are as specific as possible while staying within what the model actually knows.

Key Contributions

1. The paper tackles the non-trivial and underexplored challenge of promoting specificity without compromising correctness in fine-grained open-world image classification.
2. It provides an analysis showing that LMMs are overly generic, and that this is not due to a lack of domain knowledge, offering insight into both the potential and the limitations of these models.
3. It introduces SpeciaRL, an online reinforcement learning method with a specificity-aware dynamic reward, where the reward target for each sample is set from the best prediction found among that sample's online rollouts.
4. It shows empirically that SpeciaRL achieves the best trade-off between specificity and correctness compared with existing zero-shot and fine-tuned methods, across fine-grained and very fine-grained benchmarks.

Main Findings

• Models are generic, not ignorant: The preliminary analysis on Qwen2.5VL-7B shows predictions are mostly correct but skewed toward generic answers; in the very fine-grained test set almost 75% of predictions are Generic. Taking the best of 64 rollouts (BoN-64) yields much higher specificity and correctness, indicating the model already possesses the prior knowledge to be more precise but fails to reliably express it in one attempt.
• Best-of-N is an upper bound, not a solution: Even at BoN-64 the model still produces a notable share of Generic or Less Specific predictions, particularly in very fine-grained cases, suggesting some samples lie outside the model's capabilities.
• Prompting and standard fine-tuning trade away correctness: Asking Qwen2.5VL-7B to "Be specific" raises specificity on the fine-grained set (specificity 0.816 versus 0.742 for the base model) but lowers correctness (0.832 versus 0.846). Supervised fine-tuning reaches the highest specificity reported (0.935 on fine-grained, 0.789 on very fine-grained) with correctness of 0.807 and 0.857 respectively, giving harmonic means of 0.866 and 0.814. Reinforcement fine-tuning (rft) with the standard binary reward reaches 0.875/0.785/0.825 and 0.825/0.833/0.821.
• SpeciaRL gives the best balance: On the fine-grained set, SpeciaRL-7B reaches specificity 0.920, correctness 0.848 and HM 0.883, improving both specificity and correctness over the base Qwen2.5VL-7B (0.742, 0.846, 0.790). On the very fine-grained set it reaches 0.818, 0.855 and 0.830, again the best harmonic mean among the evaluated methods, including the BoN-64 reference (0.77, 0.998, 0.868 on very fine-grained).
• State-of-the-art on a general-purpose protocol: Under the evaluation protocol of the benchmark from Conti et al., which reports Text Inclusion, Language Inclusion, Semantic Similarity and Concept Similarity, SpeciaRL achieves state-of-the-art performance on three of the four metrics on both the fine-grained and very fine-grained test groups (for example, Text Inclusion 62.7 and 24.9, Semantic Similarity 49.2 and 50.5).
• The dynamic reward beats static alternatives: Ablating the reward, the standard binary reward for Specific and More Specific predictions performs worst among the variants tested (HM 0.825), while progressively rewarding Less Specific (0.875), then Generic (0.878), then Abstain (0.871) helps; the dynamic reward is best at 0.883. This highlights the value of rewarding correct predictions that are less informative than the ground truth.
• Rollout count matters non-monotonically: In the ablation on the fine-grained set, N=5 gives specificity 0.925, correctness 0.844 and HM 0.883; N=10 gives 0.920, 0.848 and 0.883; N=15 drops to 0.848, 0.810 and 0.824. The authors note that similar behaviour, where smaller group sizes outperform larger ones, has been reported in a recent study on GRPO, possibly due to batch-based grouping strategies mixing unrelated episodes.
• Reasoning quality improves, not just the final label: Qualitative examples show SpeciaRL uses fine visual details in its reasoning traces to deduce a fine-grained class, whereas the base model captures the same details but does not convert them into a specific prediction.

Methodology in Plain English

The authors first define a way to grade predictions. They use a strong LLM as a judge that sorts each prediction into one of six mutually exclusive categories relative to the ground truth: Wrong (refers to a different concept), Abstain (refuses), Generic (correct but much broader), Less Specific (correct but a close parent category), Specific (exact match or synonym), and More Specific (a subtype or instance). These categories are ordered from least to most informative, and each non-Wrong category gets a score of 1, 2, 3 or 4 respectively. Correctness is the fraction of non-Wrong predictions; specificity is the average normalized score over non-Wrong predictions; and overall performance is the harmonic mean of the two.

With that measurement in place, they run a diagnostic: they sample many predictions per image to see how good the model could be. This shows the model's knowledge is present but its sampling is inefficient.

SpeciaRL then fine-tunes Qwen2.5VL-7B with GRPO. For each training image the model generates a group of rollouts. An LLM verifier categorizes each rollout, and the best category within the group defines a minimum specificity requirement for that sample, with special handling when the best rollout is More Specific (the requirement becomes Specific) or Wrong (the requirement becomes Abstain). Any rollout at or above that level earns reward 1, otherwise 0; Wrong predictions always earn 0. Because the requirement is computed from the model's own online rollouts, the reward never asks the model to exceed what it can already do, and it adds no extra generation cost beyond the rollouts GRPO already produces.

Training uses 3,000 randomly selected samples from the CUB bird dataset, batch size 256, learning rate 3e-5, 15 epochs, KL penalty coefficient 0.01, N=10 rollouts per sample, and Qwen3-30B-A3B-Instruct-2507-FP8 as the reward judge, implemented with the Verl framework. Evaluation uses a different judge, Llama3-72B, to avoid family-specific bias. All test domains (flowers, food, pets, cars, aircraft) are disjoint from the bird training domain, so every reported result is out-of-domain.

Why This Matters

Impact on research. The paper reframes "model is not specific enough" as a sampling and behavioural problem rather than a knowledge problem, and shows that a reward anchored to the model's own best-of-N can extract capability the model already has. It also offers a reusable way to measure graded correctness in unconstrained output spaces, which is a general problem for evaluating generative models.

Real-world applications (the paper does not report deployed systems; these follow from the task setting it studies):

• Biodiversity and citizen-science tools that must identify a species rather than just say "bird" or "insect".
• Inventory and cataloguing pipelines where a listing needs a precise product or model name, not a broad category.
• Automotive and equipment appraisal workflows that distinguish make, model and year.
• Food, agriculture and retail recognition, where generic answers such as "food" or "flower" are of limited use.

Industry relevance. Any production system built on a multimodal model that answers "what is this?" benefits from a post-training recipe that increases answer granularity without raising error rates, and from an evaluation protocol that penalizes both vague answers and confidently wrong ones.

Future Directions

• Extending the reward to other unconstrained tasks. The specificity-aware dynamic reward is designed for classification, but the same idea could apply to any open-ended task where correctness is graded rather than binary.
• Understanding the rollout-size effect. Increasing N from 10 to 15 reduced both specificity and correctness in the reported ablation, which the authors connect to grouping strategies that may mix unrelated episodes; clarifying this remains open.
• Pushing past the model's own ceiling. Because the reward is anchored to the best online rollout, samples that are generic even at BoN-64 stay outside the reward's reach; whether those cases need knowledge injection, retrieval or better sampling is unresolved.
• Reducing dependence on the LLM verifier. Reward computation relies on an external judge model; the paper notes additional studies on judge robustness and sensitivity in the supplementary material, and scaling this verification cheaply is a natural next step.

Target Audience

Researchers and practitioners working on multimodal foundation models, reinforcement learning post-training, or open-world and fine-grained visual recognition. It is most useful to readers who already understand LMM prompting and reinforcement learning basics, and to engineers who need a principled way to make a deployed vision-language model answer more precisely without becoming less accurate.

Authors’ abstract

Classifying fine-grained visual concepts under open-world settings, i.e., without a predefined label set, demands models to be both accurate and specific. Recent reasoning Large Multimodal Models (LMMs) exhibit strong visual understanding capability but tend to produce overly generic predictions when performing fine-grained image classification. Our preliminary analysis reveals that models do possess the intrinsic fine-grained domain knowledge. However, promoting more specific predictions (specificity) without compromising correct ones (correctness) remains a non-trivial and understudied challenge. In this work, we investigate how to steer reasoning LMMs toward predictions that are both correct and specific. We propose a novel specificity-aware reinforcement learning framework, SpeciaRL, to fine-tune reasoning LMMs on fine-grained image classification under the open-world setting. SpeciaRL introduces a dynamic, verifier-based reward signal anchored to the best predictions within online rollouts, promoting specificity while respecting the model's capabilities to prevent incorrect predictions. Our out-of-domain experiments show that SpeciaRL delivers the best trade-off between correctness and specificity across extensive fine-grained benchmarks, surpassing existing methods and advancing open-world fine-grained image classification. Code and model are publicly available at https://github.com/s-angheben/SpeciaRL.