Preference Optimization Improves Definition Generation: Semantic Change in EU Institutional Sustainable Finance Disclosures

AI summary

Overview

Research area: Natural Language Processing — lexical semantic change detection (LSCD) and definition generation, applied to EU sustainable finance disclosure documents.

Technical level: Intermediate. The paper assumes familiarity with supervised fine-tuning, LoRA, and preference optimization (DPO), but the core idea and the qualitative case study are readable without deep technical background.

Scope: The paper shows that Direct Preference Optimization applied to definition-generating language models reduces redundant per-sense definitions, and tests the resulting models both on the SemEval'20 Task 1 semantic change benchmark and on a corpus of European Investment Bank (EIB) Environmental and Social Data Sheets spanning 2014–2026.

What This Paper Is About

Definition generators are language models that output a dictionary-style definition for a target word as it appears in a specific sentence. They are useful for detecting semantic change, but they suffer from "noise senses": they produce many slightly different definitions (differing by a word or a punctuation mark) for what is really one sense, making the output unusable for human experts.

The paper's goal is to fix this by fine-tuning definition generators with Direct Preference Optimization (DPO), teaching the model to prefer one canonical definition per sense and reject its own near-duplicates. The authors then check whether this helps on an academic benchmark and whether it produces usable output for a real analytical task: auditing how sustainability terminology in EU development finance documents has shifted over time.

Key Contributions

1. DPO reduces sense granularity. The authors show that preference optimization for less diverse definitions per sense consistently lowers the number of predicted definitions per target word, moving model output closer to the human dictionary distribution.

2. Benchmark effect depends on the base model. On SemEval'20 Task 1 graded change detection, DPO improves performance when the underlying model is not already strong, but degrades it for stronger base models; the authors report that scores overlap in confidence intervals and do not permit firm conclusions about the DPO step's necessity.

3. A new family of open definition generators. The authors release Llama-DPO, OLMo-SFT, and OLMo-DPO, built on OLMo-3-7B (7 billion parameters) and LlamaDictionary-en_ML38BI (8 billion parameters), reporting that the OLMo-based models outperform prior work based on Llama3-8B on the benchmark's best (no-postprocessing) configuration.

4. A validated real-world case study. A domain expert qualitatively evaluated the pipeline on EIB documents, confirming that it surfaces policy-relevant shifts (financialisation, normative stretching, regulatory absorption) and identifying which model (LlamaDictionary-en_ML38BI DPO) preserves the domain nuance needed for institutional analysis.

Main Findings

• DPO shifts sense distributions toward the human gold standard. On the DBnary development set, the human gold has 7,645 total senses, 1.49 senses per word, 20.1% ambiguous words, and 6.76 senses per ambiguous word. Llama ML38B SFT produces 9,228 senses, 1.79 per word, 30.2% ambiguous, 5.50 per ambiguous word; after DPO these fall to 9,044, 1.76, 29.3%, and 5.58. OLMo-3 SFT produces 12,721 senses, 2.47 per word, 46.7% ambiguous, 4.68 per ambiguous word; OLMo-3 DPO reduces these to 12,143, 2.36, 45.3%, and 4.72.

• Definition generators overestimate ambiguity, and OLMo-3 does so far more than Llama. The OLMo-3 percentage of ambiguous words is more than twice the gold DBnary figure, even after DPO.

• The best benchmark scores come from no postprocessing at all. Spearman correlations on SemEval'20 Task 1 subtask 2 without merging: mT0-xl SFT 0.476 [0.18, 0.69], Llama3Dictionary SFT 0.429 [0.12, 0.66], LlamaDictionary-ML38B SFT 0.517 [0.23, 0.72], OLMo-3 SFT 0.586 [0.32, 0.76], LlamaDictionary-ML38B DPO 0.532 [0.25, 0.73], OLMo-3 DPO 0.546 [0.27, 0.74]. The authors note these confidence intervals overlap, so the models cannot be fairly ranked against each other.

• Merging and clustering do not consistently help. With full merging at the tuned threshold of 22, scores drop (e.g., OLMo-3 DPO falls to 0.380); with clustering using BAAI/bge-m3 embeddings, complete linkage and a cosine threshold of 0.2, results are also mixed (e.g., OLMo-3 SFT 0.557, Llama ML38B DPO 0.493).

• Definition generation does not beat sentence embeddings on the benchmark. The authors report that their approach does not outperform XL-Durel + APD; they reproduced the XLD-APD performance from prior work at 0.768.

• DBnary-trained models beat models trained on smaller professional dictionaries. Models fine-tuned on the crowd-sourced DBnary data score slightly better than those fine-tuned on the WordNet–Oxford–CoDWoE mixture.

• A 7B model performs on par with an 8B model. The authors state that OLMo-3 (7B) matches LlamaDictionary-en_ML38BI (8B) on the benchmark, but cannot attribute a cause from this study since pretraining data, instruction tuning, and tokenizers all differ.

• APD separates stable from contested vocabulary in the EIB corpus. Terms expected to be stable ("public procurement", "private company", "environmental NGO") receive low APD scores (e.g., < 0.09), while "environmental documents" and "new hydrogen" (APD 0.391) score high. The domain expert observed an empirical APD threshold of approximately 0.27 separating substantive policy shifts from baseline variation.

• Expert coding of 1,666 candidate pairs. Out of 170 candidate phrases evaluated, 53% of pairs show no meaningful change and 15% could not be compared due to mismatched phrase boundaries. Among the remaining pairs: 21% hallucinations, 20% reintroductions of a previously attested sense, 19% financialisation, 16% normative stretching, 12% regulatory absorption, and 12% combining multiple types.

• Llama SFT is far too granular for practical use. It produced 130 distinct senses for "quality management" and distinguished "protecting the environment" from "synonym of environmental organisation" for "environmental NGO", and hallucinated links such as tying "necessary activity" to the US Constitution.

• There is a trade-off between low granularity and domain sensitivity. OLMo-3 SFT had the lowest hallucination rate and good structural coherence, but OLMo-3 DPO collapsed specialised terms (hallucinating "new hydrogen" as a "Japanese fuel"), while LlamaDictionary-en_ML38BI DPO retained policy sensitivity and was the most useful model for the case study.

• Concrete semantic shifts were documented. "Additional CO2 emissions" moved from a physical sense in 2014 to a financial one by 2016; "average lifetime" moved from physical objects in 2019 to financial entities by 2022; "broad participation" moved from a quantitative baseline in 2017 to a political one by 2022; "ambitious target" moved from a generic sense in 2015 to an EU-specific regulatory one by 2020, later also acquiring a normative reading by 2023 without losing its original sense.

Methodology in Plain English

The authors start from existing definition generators and split the DBnary training data into two parts. One part is used for standard supervised fine-tuning (for the new OLMo-3 model only; the Llama model was reused from prior work). The second part is used to build preference data: the model generates definitions for many usages of the same word sense, and the definition closest by Levenshtein edit distance to the gold definition is marked as "chosen" while the others are marked "rejected". After this post-processing, only senses with at least two usages are kept, giving 20,016 training examples and 2,086 development examples. DPO is then applied so the model learns to prefer the canonical definition and suppress near-duplicates.

Training used LoRA and bfloat16 precision. SFT used LoRA rank 256, alpha 512, dropout 0.1, learning rate 2e-5, batch size 32, and a maximum of 20 epochs (early-stopped after epoch 2). DPO used rank 64, alpha 16, dropout 0.2, learning rate 1e-6, batch size 24, one epoch, and beta 0.5 (the authors found beta in the 0.4–0.1 range increased diversity rather than reducing it). DPO took no more than 2 hours per run on NVIDIA GH200 GPUs; SFT took 5 hours walltime on two GH200 GPUs.

For change detection they reuse an existing "definitions as senses" algorithm that compares sense distributions across time periods with Jensen-Shannon divergence, tuning the definition-merging threshold (found to be 22) on the DBnary development split. They also try agglomerative clustering of definition embeddings. For the case study, they first rank candidate terms by Average Pairwise Distance using XL-Durel embeddings, then examine the generated definitions for the highest-ranked terms; the EIB corpus itself consists of 4,201 documents (4,189 identified as English by OpenLID-v3, 12 in Albanian, Greek, and French), split into 151,041 sentences, with noun phrases extracted by flair/chunk-english and target phrases selected by a domain expert.

Why This Matters

Impact on research: The paper turns definition generation from a noisy, hard-to-read output format into something a non-NLP researcher can actually work with. It also provides an honest negative-ish result: postprocessing tricks (merging, clustering) do not rescue benchmark scores, and a 37-word benchmark is too small to rank close systems fairly. The released models, code, and predictions make the approach reusable.

Real-world applications:

• Auditing whether an institution's operational vocabulary has genuinely changed or whether existing commitments were simply relabelled under a new narrative — the paper explicitly frames this around the EU's 2021 Global Gateway strategy.
• Regulatory and compliance monitoring: tracking how terms like "environmental documents" shift after regulatory shocks such as the 2020 EU Taxonomy.
• ESG and sustainable finance analysis, where terms like "additional CO2 emissions" or "average lifetime" quietly acquire financial meanings.
• Building interpretable, human-readable semantic change reports for social scientists and political scientists, rather than opaque scalar change scores.

Industry relevance: Financial institutions, development banks, and ESG rating agencies all produce large volumes of disclosure text whose terminology drifts. A pipeline that flags which specific senses were gained or lost — rather than just a change score — supports due diligence, greenwashing detection, and regulatory reporting at corpus scale instead of through manual document review. The finding that a 7B open model matches an 8B model also matters for organisations that cannot use restrictively licensed models.

Future Directions

• Build larger, domain-specific benchmarks. The authors note that expert impressions did not align with performance on SemEval'20 Task 1's 37 basic-vocabulary English words, and that prior work has reported similar benchmark-versus-real-world gaps.
• Develop better definition generators rather than dataset-specific postprocessing. The authors explicitly recommend this, noting their clustering experiments covered only one embedding model and one algorithm.
• Add independent annotators and report inter-rater agreement. The qualitative categorisation in the case study was done by a single domain expert.
• Investigate why DPO helps some models and damages others. The excessive penalisation of domain-specific jargon in OLMo-3 DPO, and the unexplained parity between the 7B and 8B models, remain open questions.
• Improve hallucination control and phrase-boundary handling. Hallucinations accounted for 21% of the analysable pairs, and 15% of pairs could not be compared due to mismatched phrase boundaries.

Target Audience

NLP researchers working on lexical semantic change detection and definition generation will find the core contribution and benchmark tables most directly relevant. Computational social scientists, political scientists, and sustainability or ESG analysts will benefit from the case study and the released models, which require no NLP expertise to use. Practitioners building disclosure-monitoring or greenwashing-detection tools will find the APD pre-filtering pipeline and the ~0.27 threshold observation practically useful. The paper is less suited to beginners, since the methodology assumes working knowledge of LoRA, DPO, and Spearman-rank evaluation.

Authors’ abstract

Dictionary-like definition generators are known to be useful for lexical semantic change detection (LSCD). However, they suffer from overly granular sense distribution, producing excessive number of definitions for one and the same lexical sense. In this work, we show that fine-tuning with Direct Preference Optimization (DPO) helps to overcome this problem. We test our approach both quantitatively on a curated academic LSCD benchmark and qualitatively on real-world data for a practical task of auditing diachronic changes in sustainability terminology within project documentation of EU development finance. All models, code and predictions produced in this study are publicly available at https://github.com/ltgoslo/dpo-for-definition-generation.