Answering in the User's Variety: Calibrated Accommodation Pairs for Post-Training

AI summary

Overview

Research area: Large language model post-training and linguistic accommodation (dialect and register matching). Although the arXiv metadata lists this under Computer Vision (cs.CV), the content is an NLP/LLM training-dynamics study. Technical level: Intermediate — readers should be comfortable with SFT, DPO, RLVR, and likelihood ratios, though the core ideas are explained concretely. Scope (one sentence): The paper traces how each post-training stage of open model lineages changes both the replies models give to users of different English varieties and the preferences embedded in the training updates, then proposes a preference-data recipe (Calibrated Accommodation Pairs) to correct selective accommodation.

What This Paper Is About

Chat models tend to answer in Standard English even when the user writes in African American English (AAE), Singlish, Nigerian Pidgin, or other varieties, and looking only at outputs cannot reveal whether training treats those users differently. The authors ask two questions: which post-training stage installs the Standard-English default, and whether the training update itself prefers matching some varieties over others even when the visible replies look uniform. They then build and test preference data designed to make models match a user's dialect density without imposing dialect on Standard-English users.

Key Contributions

1. Standard replies, selective updates. Across 10 open model lineages followed through SFT, DPO, and reinforcement learning with verifiable rewards (RLVR), post-training standardizes replies to every informal cue, yet the DPO update still favors matching British and Australian spelling over AAE and Singlish on all 10 lineages after adjusting for baseline likelihood and length.
2. A controlled re-run of accommodation data. Adding accommodation-consistent preference pairs raises convergence only at the largest dose tested (60k pairs, 33% of the DPO data), and does so unevenly across runs while introducing dialect for Standard-English users.
3. Calibrated Accommodation Pairs (CAP). A preference-data recipe that penalizes under-accommodation, over-accommodation, and imposition; on OLMo-2-1B it raises held-out convergence by 0.173 over matched controls without a detectable increase in dialect for Standard-English users.
4. Diagnostics and benchmark. A copy-controlled convergence ratio (scoring only novel spans) and a matching bonus that separates output behavior from what a training update prefers, plus released code and data.

Main Findings

• Post-training flattens replies to every informal cue. On OLMo-2-7B, British-spelling reproduction falls from 0.171 at base (95% CI [0.137, 0.207]) to 0.120 after SFT, 0.058 after DPO ([0.040, 0.076]), and 0.060 after RLVR; informal-register convergence falls from 0.413 at base under the chat template to 0.009 after DPO. The preference stage lowers informal-register convergence on all 10 lineages and British-spelling convergence on 9 of 10.
• Formal register is the exception. Replies keep converging to formal requests at every stage (OLMo-2-7B after DPO: 0.141), so post-training does not simply reject formality.
• Dialect grammar was already weakly mirrored. Base OLMo-2-7B reproduces AAE features at 0.022 ([0.013, 0.031]) and Singlish lexis at 0.051; after SFT both are near zero, including on human-written AAE tweets.
• The prestige ordering predates post-training. Convergence follows the pre-registered prestige tiers at base (ρ = 0.778 on OLMo-2-7B) and after DPO (ρ = 0.845); post-training lowers the level but does not install the order.
• The pattern generalizes. From base to final release, informal-register convergence falls in all 12 further families, British-spelling convergence in 9, and AAE ends at most 0.001. Across 5 deployed chatbots, convergence is at most 0.053 for informal register and 0.009 for AAE, while British spelling is reproduced at 0.089–0.207 and Nigerian Pidgin at 0.116–0.502, above any post-trained open model (at most 0.046).
• Capability survives. Asked to reply in the user's variety, OLMo-2-7B after DPO realizes AAE features at 0.567 of their authentic rate versus 0.009 in default replies; Tülu-3-8B reaches 0.588 and OLMo-3-7B 0.411. Two of three families exceed half their authentic rate on request.
• The DPO update is selective. Pooled over lineages, the prestige-minus-stigmatized matching bonus is 0.020 nats per token (CI [0.014, 0.027]) and stays positive under leave-one-lineage-out. Per variety, the bonus is negative for AAE on 9 of 10 lineages (OLMo-2-7B: −0.018, CI [−0.024, −0.012]), for Singlish on 6, and for informal register on 6, and is not detectably negative for British and Australian spelling on 9. The pre-registered mixed model gives a positive tier effect on 9 of 10 lineages but a CI above zero only on 2.
• Popular families agree. Across 14 further updates, the adjusted contrast is positive in 12, reversed in 0, and inconclusive for SmolLM3's preference stage and Qwen3.5-4B. No update significantly favors stigmatized over prestige matching.
• One reversal depends on adjustment. OLMo-2-32B's raw contrast reverses (−0.031) while its adjusted contrast is positive (0.012); OLMo-3.1-32B does not reverse, so size alone does not explain it.
• Dialect-bearing preference data is rare. Only 0.81% of prompts in the OLMo-2-7B preference mix and 0.64% in Tülu-3's trigger the dialect recognizers.
• Dose response is not gradual. In a DPO re-run from OLMo-2-1B SFT, 1k (0.8%), 5k, and 20k (14%) added pairs leave held-out convergence at its dose-0 level (20k: 0.001, CI [−0.002, 0.004]). At 60k (33%, added after pre-registration), held-out convergence rises above control in all 4 runs by 0.090, 0.023, 0.030, and 0.027 (0.043 on average, CI [0.032, 0.058]), transferring to held-out varieties by 0.035 on average. IFEval prompt-level accuracy is 57.9% over the 4 runs, against 59.1% at dose 0 and 58.2% for matched controls.
• That re-run does not calibrate. At 60k, Standard-English replies gain 0.10–1.06 trained dialect features per 100 words (controls 0.00), Singlish replies carry 1.07–9.47 times the user's trained-feature rate, and AAE replies only 0.17–0.47 times.
• CAP raises convergence without insertion on OLMo-2-1B. Held-out convergence is 0.176 versus 0.2 for controls and 4.5 for one-sided pairs (×100 scale), a gain of 0.173 over controls (CI [0.138, 0.207]), which is 4.0 times the gain of the registered one-sided pairs (0.043); held-out varieties gain 0.150 versus 0.032. Insertion changes by 0.001 features per 100 words (CI [−0.008, 0.013]) where one-sided pairs insert 0.560, at no cost in instruction following (IFEval 2.1 points, CI [−0.7, 4.9]). Singlish replies sit at 0.79 of the user's density, against 2.59 for one-sided pairs.
• CAP meets only part of its pre-registered bar. M1 (held-out convergence) and M4 (IFEval) pass; calibration (M2), density tracking (M3), and compliance with explicit requests for Standard English (M5) fail. On Qwen3-8B, CAP gives 0.017 (CI [0.009, 0.026]), below the one-sided recipe's 0.154 and short of M1.
• Ablations isolate what each component buys. Without O pairs, held-out convergence rises to 0.583 but Standard-English replies gain 0.039 dialect features per 100 words; without A pairs, convergence is 0.008; fine-tuning on mirrored replies instead of preferring them leaves convergence unchanged (−0.001); a 1.5× target reaches 0.340 but with 1.61 dialect features on Standard-English requests; one-sided pairs rebuilt with CAP's builder reach 0.201 only by overshooting, with Singlish at 10.41 times the user's density (CAP: 0.79) and 0.378 inserted features per 100 words.
• Sensitivity checks hold. Changing one analysis choice at a time (estimator, bootstrap, reply length, sampling seeds, human-audit correction) leaves both main findings in place; CAP's held-out gain persists under a request-only bootstrap, without the false-positive exclusion, and on the earlier requests (0.137).
• Pre-registered predictions that failed include the claim that the prestige/stigmatized split grows at the preference stage and that convergence rises gradually with the dose of accommodation-consistent pairs.

Methodology in Plain English

The authors wrote 287 first-person requests in six task types and rendered each one in six English varieties — British, Australian, AAE, Colloquial Singapore English, Nigerian Pidgin, and Appalachian English — plus Standard American English and two register controls (informal and formal Standard English). This gives "twin" requests that differ only in variety features, so any difference in the model's reply can be attributed to the variety rather than the content.

They then took every publicly released checkpoint of 10 open lineages (OLMo-2 at 1B–32B, OLMo-3, Tülu-2, Tülu-3, Zephyr, plus OLMo-3.1-32B), sampled 2 replies per request over 3 seeds, and scored only novel spans — tokens outside any word 4-gram shared with the request and outside quotations — so a model that simply echoes "keep dragging" earns no credit. Feature detectors count how often a variety feature appears relative to the opportunities for it, giving a variety density; comparing a model's

Authors’ abstract

People adapt how they talk to the person in front of them. However, chat models tend to respond in Standard English regardless of the dialect used by the user. A uniform answer can hide whether training treats these users alike, and output audits cannot tell as well. We trace how each post-training stage, from supervised fine-tuning (SFT) to preference optimization (DPO) and reinforcement learning, changes a model's tendency to match the user, and read what each update prefers from two content-matched replies, one matching the user and one not. Across 10 open model lineages, post-training standardizes replies to every informal cue, yet after adjusting for how likely each reply already was, the DPO update still favours matching British and Australian spelling over AAE and Singlish on 10 lineages, and models still produce AAE features when asked. We then introduce calibrated accommodation pairs (CAP), preference data that reward matching users' dialect density while preserving Standard English for Standard-English users. On a fully open model, CAP raises convergence on features it was not trained on without a detectable increase in dialect for Standard-English users or a cost in instruction following. In contrast, one-sided pairs reach a similar gain only by overshooting the user and imposing dialect. Our findings show that uniform replies can hide selective updates and audits should read the update as well as the output. Code: https://github.com/kouzhizhuo/cap-accommodation.