Training Specialist Models without Reasoning Trajectories for Domain Expert Distillation

AI summary

Overview

• Research area: Machine learning — large language model specialist distillation, domain adaptation, and reasoning-trajectory supervision.
• Technical level: Advanced (the paper includes formal derivations of the distillation objective and a KL-constrained specialist objective).
• Scope: The paper argues and empirically demonstrates that when specialists are trained only on question–answer pairs, the specialist's optimization procedure—not the answer labels—selects the latent reasoning trajectories that downstream distilled students inherit.

What This Paper Is About

Specialist distillation usually works by first adapting a general model into a domain specialist, then having that specialist generate reasoning trajectories that supervise a student. But in most specialized domains, the specialist itself is never taught how to reason: datasets provide only question–answer (QA) pairs, with no gold trajectories. The paper asks what actually determines the reasoning paths a QA-only specialist generates, and shows that the answer lies in the specialist's training choices, which implicitly select one trajectory distribution out of many that are equally consistent with the correct answers.

Key Contributions

1. Identifies specialist optimization as the key design variable for distillation data. Under QA-only training, reasoning trajectories remain underdetermined by answer labels; the specialist's optimization procedure selects which trajectory distribution downstream supervision is drawn from.
2. Uses student distillation as an agnostic probe. Because students inherit neither the specialist's parameters nor its optimization constraints, distilled students reveal the supervision that specialists selected. The paper reports a strong correspondence between specialist and student specialization–generalization profiles across domains and model families (Spearman ρ = 0.9573, permutation test p-value = 0.0093, across nine specialist-student pairs covering 27 distinct evaluations).
3. Characterizes and controls the specialization–generalization trade-off. Through controlled distillation experiments, trajectory-quality analysis, and comparisons between unconstrained tuning, implicit drift control (Layer-Selective Tuning, LST), and explicit KL anchoring (Anchored Supervised Fine-Tuning, ASFT), the paper identifies distributional drift as a governing axis of latent supervision.
4. Releases code, models, and datasets at github.com/CONE-MT/DCO/tree/main/specialist_distillation and huggingface.co/collections/yileitu/qaonly-specialist-distillation.

Main Findings

• QA-only specialist distillation consistently improves target-domain performance. Across FFT, LoRA, and LST tuning strategies and three target domains (chemistry, physics, low-resource multilingualism), both specialists and distilled students achieve large in-task (It) gains over the origin Qwen3-8B and two self-training baselines.
• An 8B model can bridge a 4× parameter gap. The LST-distilled model scores 29.54 / 92.75 / 34.17 across all It splits, surpassing the zero-shot Qwen3-32B scores of 25.35 / 92.45 / 33.78.
• Post-hoc rationalization is toxic. The untrained Self-Rationalize baseline plummets on the multilingualism It split, falling from 29.48 to 6.02, consistent with claims that post-hoc reasoning on translation is often spurious and unreliable.
• Different QA-only methods select different trajectory distributions. LST consistently improves It over the origin, achieves modest Id gains, and stays on par on Ood. FFT yields impressive It gains in domains such as chemistry but severely sacrifices Id and Ood, suffering catastrophic forgetting. LoRA shows a trade-off pattern similar to LST but falls short of LST across all 9 splits.
• Distillation rank-preserves specialist quality. The specialist-to-student rank preservation is reported with Spearman's ρ = 0.9573 (Figure 2, each point being a (specialist, distilled) performance pair from Table 2).
• Transfer holds across divergent model families. Using Intern-S1-mini as the specialist (π_φ0-S1m) and distilling into Qwen3-8B (3-8B), chemistry It rises to 31.33 (FFT- Distill) and 36.91 (FFT- Distill), versus the Qwen3-8B origin at 19.34.
• Distributional drift is a controllable axis. Limiting how far a specialist drifts from its base model directly recalibrates its trade-off between domain mastery and general capabilities, and the student moves with it; LST provides an implicit anchor with lower behavioral KL divergence, while ASFT provides an explicit intervention whose anchoring strength (λ ∈ {0.05, 0.2, 0.5}) systematically moves both specialist and student along the same trade-off.
• Trajectory-level ambiguity is formalized. From the decomposition log π(a|q) = log Σ_{τ→a} π(τ|q), the paper shows that trajectories with larger posterior weight dominate the gradient update, so repeated optimization resolves the underdetermination by selecting a particular trajectory distribution.

Methodology in Plain English

The authors start from Qwen3-8B-Instruct as the "origin" model and train domain specialists on question–answer pairs only, using three different tuning strategies: full fine-tuning (FFT), LoRA (rank r = 64, α = 2r, dropout 0.05, target: all), and Layer-Selective Tuning (updating the bottom 4 and top 16 layers). They add ASFT as an explicit KL-based drift-control mechanism with anchoring strengths λ ∈ {0.05, 0.2, 0.5}.

Each specialist then generates candidate chain-of-thoughts and answers, which are filtered so that the predicted answer matches the ground-truth answer and the chain-of-thought is complete. Filtering criteria differ by domain: subtask-specific criteria for chemistry, rule-based symbolic and numerical verification for physics, and spBLEU ranking retaining the top 20% subset for multilingualism. Every distilled student is FFT-trained from the same Qwen3-8B initialization with identical configurations on an equal amount of subsampled data, so differences among students reflect the data they received rather than student-side variance. Two self-training baselines, Self-Distill (question-only) and Self-Rationalize (answer-conditioned), plus larger Qwen3-14B and Qwen3-32B models, serve as comparisons.

Evaluation spans three categories: in-task (same domain and formulation), in-domain (same domain, different task distribution), and out-of-domain (entirely different domains). Training data are SMol for chemistry (14 subtasks), the university-level physics subset of MegaScience, and OPUS for bidirectional English–8 low-resource language translation.

Why This Matters

This work reframes specialist training itself as the object of design. When gold reasoning is absent, tuning choices directly control the latent supervision passed downstream, so the standard "train a specialist, then distill" pipeline commits to one behavior among many without anyone explicitly choosing it.

Real-world applications implied by the paper's setup:

• Molecular and chemical discovery: domain models trained on SMol-style molecular understanding and generation tasks across 14 subtasks.
• Physics tutoring and problem solving: models trained on university-level MegaScience physics data and evaluated on PHYSICS and high-school physics benchmarks.
• Low-resource translation and multilingual assistants: bidirectional English–8 low-resource language translation trained on OPUS and evaluated on Flores-101 and general reasoning in those 8 languages.
• Cost-efficient domain deployment: replacing a 4× larger generalist with an 8B distilled specialist, avoiding the need for expensive and hard-to-verify expert rationales.

Industry relevance: the results suggest that model builders can tune a single knob—how far a specialist drifts from its base model—to trade domain precision against general-capability retention, and that these choices propagate predictably into deployed students. This makes rationale data selection an engineering decision rather than a byproduct of fine-tuning, and it shows that cross-family distillation is viable, since transfer held when the specialist came from a different model family (Intern-S1-mini) than the student (Qwen3-8B).

Future Directions

• Determining how the latent trajectory distribution selected by a specialist could be characterized directly rather than only through the distillation probe.
• Establishing how to choose the drift-control strength for a target mix of domain precision and general-capability retention, given that LST and ASFT move specialist and student along the same trade-off.
• Testing whether the rank-preservation relationship extends beyond the nine specialist-student pairs and 27 evaluations reported here, including the divergent model families mentioned but not fully tabulated in the truncated content.
• Understanding why forcing weak models to rationalize answers poisons distillation, given the collapse of the Self-Rationalize baseline on multilingualism (It from 29.48 to 6.02).

Target Audience

Researchers and practitioners working on LLM distillation, domain adaptation, and reasoning-model training, particularly those building specialist models from question–answer data without expert reasoning annotations. It is also relevant to engineers selecting fine-tuning strategies (FFT, LoRA, LST, ASFT) for domain deployment, and to readers interested in how training-time optimization choices propagate into downstream model behavior across model families.

Authors’ abstract

Specialist distillation effectively transfers domain expertise to student models via teacher-generated reasoning trajectories. However, when these specialists are trained solely on question--answer pairs without explicit reasoning supervision, what governs the trajectories they generate? In this work, we show that specialist optimization implicitly selects from this latent trajectory space. To isolate and observe this latent distribution, we leverage student distillation not as a downstream goal, but as an agnostic probe---since students inherit no parameterization or optimization constraints from the specialist, inheriting only the sampled trajectories themselves. Through this probe, our empirical analysis unveils a tight governing relationship: across 27 specialist--student pairings, their specialization--generalization profiles correlate exceptionally strongly. Crucially, explicitly controlling the specialist's distributional drift systematically shifts both the teacher and its distilled student along a controllable trade-off between domain precision and general-capability retention. Across chemistry, physics, and multilingual settings, distilled students systematically reflect these specialist-induced profiles, even across divergent model families. Our findings establish a new view of specialist training: when gold reasoning is absent, tuning choices directly control the latent supervision passed to downstream models.