ECO Decoding: Entropy-Based Control for Controllability and Fluency in Controllable Dialogue Generation

AI summary

Overview

Research area: Natural Language Processing — controllable dialogue generation (CDG), specifically inference-time weighted decoding for attribute control.

Technical level: Intermediate. The core idea is intuitive (adjust control strength by uncertainty), but the paper involves Bayesian factorization of the next-token distribution, entropy computation, and multiple evaluation protocols across three decoding baselines, two datasets, and three language models.

Scope: The paper proposes ECO decoding, an entropy-based method that dynamically sets attribute control strength at every decoding step, and evaluates it against FUDGE, Director, and DASC on DailyDialog and MultiWOZ using DialoGPT-small (176M), DialoGPT-large (1.1B), and Llama2-7B.

What This Paper Is About

Weighted decoding for controllable dialogue generation multiplies a language model's next-token distribution by an attribute classifier's probability, raised to a control strength (lambda). Prior methods hold that strength fixed for the whole response, which forces a trade-off: too little control and the attribute never appears, too much and the sentence becomes incoherent or ungrammatical. This paper asks whether the control strength should instead be decided token by token, based on how uncertain the model is at each step — applying stronger attribute pressure where the language model has many plausible options, and backing off where it is confident.

Key Contributions

1. The authors identify static control strength in existing weighted decoding methods as the source of the controllability–fluency trade-off, and propose a dynamic control strength that varies at every generation step based on entropy.
2. They show ECO decoding generalizes to multi-attribute scenarios and alleviates the probability interpolation problems that single-attribute weighted decoding methods commonly face.
3. They report that ECO decoding outperforms existing weighted decoding methods across all tested controllable generation models (FUDGE, Director, DASC), across model sizes from 176M to 7B parameters, and across two datasets.
4. ECO decoding requires no additional modules or training and can be applied on top of existing weighted decoding methods.

Main Findings

• Static control strength is the core problem: Because a fixed strength keeps applying the same attribute bias even when a sentence already has enough attribute signal or when specific tokens are needed for fluency, every tested method exhibits a trade-off between grammar and attribute accuracy as lambda changes.

• Single-attribute gains on DailyDialog (Table 1): With emotion control, FUDGE improves from 76.98 to 81.03 (+4.05), Director from 79.94 to 82.82 (+2.88), and DASC from 74.65 to 75.74 (+1.09). With dialog-act control, FUDGE improves from 41.07 to 46.42 (+5.35), Director from 70.96 to 71.56 (+0.60), and DASC from 42.59 to 47.17 (+4.58).

• Grammar is maintained rather than traded away: Grammar scores move only slightly in either direction (for example, Director on dialog-act goes from 91.18 to 91.15, a change of -0.03), and ROUGE/Dist scores are generally retained or marginally improved (FUDGE dialog-act Dist-1 rises from 0.59 to 0.62).

• Multi-attribute control improves (Table 2): Under simultaneous emotion and dialog-act control, FUDGE goes from 66.17/44.17 to 66.41/45.57, Director from 80.48/60.65 to 81.18/61.20, and DASC from 75.19/51.17 to 77.22/54.12. The paper notes DASC shows a ROUGE-1 drop of -0.62 and ROUGE-L drop of -0.52 in this setting.

• Gains hold as model scale grows (Table 3): With DialoGPT (1.1B), Director improves from 75.66 to 76.05 (+0.39) and with Llama2 (7B) from 75.43 to 75.66 (+0.23) on the emotion attribute, with grammar also rising slightly in both cases.

• Cross-domain generalization on MultiWOZ (Table 4): On the multi-attribute setting, Director with DialoGPT (176M) improves from 63.62/49.29 to 66.52/52.59, and with Llama2 (7B) from 59.57/23.90 to 65.80/28.87. The authors report no degradation in ROUGE, grammaticality, or diversity.

• Human evaluation agrees with automatic metrics (Table 5): For emotion, Director goes from 2.82 to 3.19 on Accuracy, 2.96 to 3.16 on Interest, and 2.75 to 3.15 on Sensible. For dialog-act, Accuracy goes from 3.04 to 3.42, Interest from 2.93 to 3.41, and Sensible from 2.78 to 3.36. The paper summarizes the average improvement as roughly +0.38 Accuracy, +0.34 Interest, and +0.49 Sensible.

• Entropy correlates with part of speech (Table 7): Function words such as ADP (3.04 for DialoGPT-S), CCONJ (3.68), and PUNCT (3.69) show low entropy, while content words such as NOUN (4.81), ADJ (5.00), and INTJ (5.47) show higher entropy, supporting the intuition that high-uncertainty positions are easier to steer without harming fluency.

• The choice of weighting function matters (Table 8): Using the language model distribution for entropy ("lm_entropy") scores 79.24/58.63, while the reciprocal function of attribute entropy scores 81.18/61.20 — the strongest row — and the exponential and linear/negative variants score 80.48/61.12 and 80.17/60.96 respectively. The linear variant degrades performance because its weights decrease without converging as entropy rises.

• Smoothing temperature is stable up to a point (Table 6): Varying tau from 0.1 to 5.0 produced only minor fluctuations (grammar 90.30 to 90.28, accuracy 82.74 to 83.28), while tau = 10.0 deviated noticeably (grammar 89.57, accuracy 85.69).

• Decoding overhead is small (Table 10): Average per-token latency rises from 3.087 ms to 3.114 ms for FUDGE (+0.9%), from 1.143 ms to 1.160 ms for Director (+1.5%), and from 1.116 ms to 1.123 ms for DASC (+0.6%).

• Zero-shot prompting is reported for comparison (Table 9): The study includes zero-shot prompting results with Llama2 on the DailyDialog test set (accuracy 76.28 for emotion, 40.12 for dialog-act, and 58.71/29.62 for the multi setting), though the truncated text does not discuss these numbers further.

Methodology in Plain English

The authors build on the standard weighted decoding recipe. At each step, a language model proposes a next-token distribution, and an attribute classifier scores candidate tokens by how well they fit the desired attribute (emotion or dialog-act). The final distribution is the language model probability raised to one weight, multiplied by the attribute probability raised to the control strength lambda. Every prior method fixes lambda.

ECO decoding replaces the fixed lambda's exponent with a step-specific factor. At each step the method keeps the top-k candidate tokens and computes two entropies: one over the language model's renormalized top-k distribution, and one over the attribute classifier's probabilities for those same candidates. Both distributions are renormalized with a softmax using temperature parameters (tau_lm and tau_c).

The insight is directional: when the language model is confident (low entropy), its preferred token is likely needed for fluent, grammatically constrained output, so the attribute weight should be weaker. When the language model is uncertain (high entropy), many candidates are plausible and there is room to push toward the desired attribute, so the attribute weight should be stronger. The control factor is therefore defined as an inverse, converging function of entropy — the reciprocal form performed best in the ablation — so low-uncertainty distributions get more weight and high-uncertainty distributions get proportionally less. If all control factors were fixed at 1, the method reduces exactly to conventional weighted decoding.

For multiple attributes, the formula takes the product of attribute classifier terms, each raised to its own entropy-adjusted weight, and each attribute's entropy is computed separately.

Experiments train each method's attribute classifiers on the DailyDialog training data while keeping the language model frozen. FUDGE was trained for 30 epochs with batch size 8 and learning rate 2e-5; Director for 20 epochs with batch size 32 and learning rate 1e-5; DASC for 30 epochs with batch size 4 and learning rate 1e-5. All runs used greedy search, a maximum sequence length of 128, and a single NVIDIA GeForce RTX 3090. Controllability was judged by two RoBERTa-based evaluators trained on DailyDialog that reach 89.66% and 80.60% test accuracy and are independent from each method's own attribute classifier; quality used ROUGE-1/ROUGE-L and Dist-1/Dist-2, and grammar used a RoBERTa-based CoLA model. Human evaluation used three expert raters scoring Accuracy, Interest, and Sensible on a 1–5 scale over 10 sampled contexts per attribute value from the Director model's outputs.

Why This Matters

Impact on research: ECO decoding argues that the controllability–fluency tension in weighted decoding is not a property of the method class but of a single hyperparameter, and it removes that hyperparameter's fixed nature without retraining. Because it plugs into FUDGE, Director, and DASC unmodified, it offers a general-purpose add-on for decoding-time control experiments, and its multi-attribute results address a known weakness of probability interpolation when several classifiers are multiplied together.

Real-world applications:

• Emotion-aware conversational agents that can shift tone (the paper's attributes include anger, disgust, fear, happiness, sadness, and surprise) while still producing natural replies.
• Task-oriented assistants where dialog-act control (inform, question, directive, commissive) determines whether the system asks, tells, or requests — relevant to the seven MultiWOZ domains: restaurants, trains, attractions, hotels, taxis, hospitals, and police.
• Customer-support and booking systems that need to hit a conversational intent reliably without sounding robotic or ungrammatical.
• Deployment on constrained hardware: since gains were observed on DialoGPT-small (176M) and the measured latency overhead is under 2%, the method is practical where re-training large models is not.

Industry relevance: The method requires no additional modules or training, adds roughly 0.6% to 1.5% per-token decoding latency, and works on top of already-deployed decoding stacks, which lowers the cost of adopting controllable generation in production chatbots. The paper also flags an abuse risk: if malicious attributes are used as control targets, the model could be induced to generate inappropriate responses, and the authors state that generating controlled responses with malicious attributes should be restricted.

Future Directions

• Building new corpora with additional attributes beyond emotion and dialog-act, since the paper states validation covered only dialogues with two attributes and that the datasets used are limited in both quantity and diversity.
• Developing a refined normalization process that accounts for the number of classes within each attribute as the number of attribute classifiers grows, which the authors name as a required step for multi-attribute settings.
• Testing combination with methods the paper mentions but does not evaluate, such as GeDi and DExperts, which require a separate language model per attribute value.
• Investigating effective probability control strategies under richer multi-attribute conditions, and extending evaluation to larger models and further dialogue domains.

Target Audience

Researchers and practitioners working on controllable text generation, dialogue systems, and inference-time decoding strategies — particularly those already using FUDGE, Director, or DASC who want attribute control without retraining. It is also useful for engineers deploying conversational agents who need a low-overhead method that scales from 176M-parameter models to 7B models, and for readers interested in entropy as a signal for when to intervene during generation.

Authors’ abstract

Controllable Dialogue Generation (CDG) enables chatbots to generate responses with desired attributes, and weighted decoding methods have achieved significant success in the CDG task. However, using a fixed constant value to manage the bias of attribute probabilities makes it challenging to find an ideal control strength that satisfies both controllability and fluency. To address this issue, we propose ECO decoding (Entropy-based COntrol), which dynamically adjusts the control strength at each generation step according to the model's entropy in both the language model and attribute classifier probability distributions. Experiments on the DailyDialog and MultiWOZ datasets demonstrate that ECO decoding consistently improves controllability while maintaining fluency and grammaticality, outperforming prior decoding methods across various models and settings. Furthermore, ECO decoding alleviates probability interpolation issues in multi-attribute generation and consequently demonstrates strong performance in both single and multi-attribute scenarios.