LLM-MC-Affect: LLM-Based Monte Carlo Modeling of Affective Trajectories and Latent Ambiguity for Interpersonal Dynamic Insight

AI summary

Overview

• Research area: Natural Language Processing / affective computing, with an applied focus on dyadic (two-person) interaction analysis in instructional dialogue.
• Technical level: Intermediate. The framework is conceptually clear (sample many times, average, measure spread), but the downstream analysis uses normalized cross-correlation and least-squares slope estimation.
• Scope: The paper proposes a probabilistic pipeline that replaces deterministic sentiment labels with latent affective distributions estimated via stochastic LLM decoding and Monte Carlo sampling, then uses the resulting trajectories to characterize affective coupling between a teacher and a student.

What This Paper Is About

Most text-based sentiment systems assign a single fixed score to each utterance or speaker, which collapses subjective variability and hides ambiguity. The authors argue that emotion in conversation is better modeled as a continuous latent probability distribution, and that the randomness already present in LLM decoding can be exploited to approximate that distribution without collecting labels from many human raters. The goal is to turn these uncertainty-aware sentiment trajectories into interpretable indicators of who leads and who follows emotionally in a conversation.

Key Contributions

1. A probabilistic affect pipeline. LLM-MC-Affect uses stochastic LLM decoding combined with Monte Carlo estimation to model utterance-level affect as a latent distribution, quantifying ambiguity through variance rather than relying on deterministic sentiment labels. Each dialogue turn is sampled K = 20 times.
2. An interaction-analysis layer. The framework extracts interpretable dyadic indicators: a normalized sequential cross-correlation function, an estimated influence lag, and slope-based trajectory signals, which together yield a structured typology of interpersonal affective dynamics (Table 1).
3. Feasibility demonstration in instructional dialogue. Using teacher-student dialogues, the authors show that uncertainty-aware trajectories remain robust under decoding stochasticity and support interpretable affective synchrony insights in a text-only, zero-shot setting with no task-specific fine-tuning.
4. A cross-model comparison. Six LLMs are compared at a fixed decoding temperature, exposing systematic differences in affective resolution and ambiguity across GPT-family and open-source models.

Main Findings

• Mean is stable, variance is not. While the variance roughly doubles as decoding temperature τ scales from 0.1 to 1.0 (σ̃²ₜ rising from 0.010 to 0.024 for one utterance), the sample mean stays within a narrow band, fluctuating only between −0.11 and −0.26. The authors treat this as evidence that the framework filters sampling noise while preserving the underlying affective signal.
• Temperature changes the shape of the distribution, not necessarily accuracy. For Utterance 6 of the Personification dialogue, τ = 0.1 produces a bimodal distribution with two sharp peaks near −0.20 and 0.00 (mean s̃ₜ = −0.12, σ̃²ₜ = 0.010). At τ = 1.0 these peaks merge into a broader distribution, making latent ambiguity more visible rather than making the estimate more accurate.
• GPT-4.1 and GPT-3.5-Turbo show the strongest structural consistency. Both capture the characteristic V-shaped affective progression (early negative dip, then recovery). GPT-3.5-Turbo nevertheless shows substantially higher affective ambiguity, indicated by a wider ±σ band.
• Llama 3.3 70B exhibits extreme positivity bias. It shows a near-total failure to identify the early negative emotional shift, following a continuous near-linear upward trend from the neutral baseline with no dip at Turn 2. The authors speculate this stems from alignment for harmlessness.
• Open-source models underestimate affective turning points. GPT-OSS 120B shows a markedly shallow emotional dip during the Turn 1–2 transition; Phi-4 14B displays a mild positive bias with recovery capped at approximately 0.40; Gemma 3 4B detects the initial dip but plateaus near the neutral baseline (∼0.15), a "recovery bottleneck."
• The Personification case is classified as "Effective Scaffolding." With GPT-4.1 at τ = 0.7, the estimated influence lag is L* = +1 and the cross-correlation is R_TS(L*) = 0.999, indicating the teacher leads and the student's affect is highly predictable from the teacher's previous turn. Slopes were β_T = 0.1621 and β_S = 0.2532.
• Quantitative metrics matched the scripted personas. The teacher's leading role aligned with the "insistence on experiential methods" setting, and the high correlation plus positive student slope indicated the student's preference for hands-on activities was met, avoiding the scripted rude reaction.

Methodology in Plain English

Each utterance is evaluated by a large language model under a fixed psychometric prompt that scores affect on a 0-to-5 scale in 0.5 increments, where 2.5 is neutral, scores closer to 0 are more positive, and scores closer to 5 are more negative. Using an unsigned scale is a deliberate design choice to avoid sign confusion during inference. The same utterance is scored many times (K = 20) at a non-zero decoding temperature, and these repeated scores are treated as samples drawn from the model's implicit belief about the speaker's emotional state. The mean of the samples becomes the affect estimate, and the variance becomes a measure of ambiguity. Raw scores are then linearly remapped to a centered [−1, 1] interval so that positive values mean positive valence, with the variance scaled by (2/5)² to stay consistent.

Once each speaker has a sequence of mean-and-variance pairs across the dialogue, the authors compute a normalized cross-correlation function between the teacher and student trajectories over conversational lags L ∈ ℤ ∩ [−3, 3]. The lag that maximizes the absolute correlation, L*, indicates direction of influence: L* ≥ 0 suggests teacher-leading, L* < 0 suggests student-driven adjustment. Separately, a least-squares slope is fit to each trajectory to summarize long-horizon drift. Combining the lag sign, the correlation sign, and the relationship between the two slopes produces a nine-cell interpretive typology (e.g., Effective Scaffolding, Negative Contagion, Counterproductive Support, Feedback Burnout, Dynamic Compensation).

Evaluation used Google's Education Dialogue Dataset, a synthetic multi-turn instructional dataset, because real classroom data requires extensive IRB approval and raises student privacy concerns. Models tested were GPT-4.1 (snapshot 2025-04-14), GPT-3.5-Turbo (snapshot 2024-01-25), Gemma 3 4B, Llama 3.3 70B, Phi 4 14B, and GPT-OSS 120B, all in their original non-fine-tuned configurations.

Why This Matters

Impact on research. The paper reframes inter-annotator disagreement and model stochasticity as signal rather than noise, offering a way to study affective ambiguity without expensive multi-rater labeling. It also links low-level probabilistic affect measurement to higher-level dyadic interpretation through a reusable indicator typology, and does so using only text, without biometric instrumentation or neural alignment measures.

Real-world applications.

• Instructional analytics: detecting whether a teacher's emotional stance is actually reaching a student, and flagging patterns like counterproductive support or shared fatigue.
• Adaptive tutoring systems: using uncertainty-aware affect signals to decide when to change instructional strategy.
• Conversational agent evaluation: measuring whether a deployed chatbot or virtual instructor leads or follows a user's emotional trajectory.
• Broader behavioral research: studying affective coordination in counseling, negotiation, or customer-service transcripts where controlled instrumentation is impractical.

Industry relevance. The approach is zero-shot and deployable without task-specific fine-tuning, and it requires no specialized sensing hardware, which the authors frame as a scalable pathway for affect-aware adaptive systems. The cross-model comparison is also practically useful: it tells practitioners which off-the-shelf models preserve fine-grained affective dynamics and which ones systematically flatten them. Any organization choosing an LLM to power emotion-aware analytics would find the observed positivity bias in Llama 3.3 70B and the recovery bottleneck in Gemma 3 4B directly relevant to model selection.

Future Directions

• Validation on real classroom data. The current study is explicitly positioned as an empirical pilot on a simulated dataset; the authors note that naturalistic deployment awaits the IRB approvals and privacy safeguards that motivated using synthetic dialogues in the first place.
• Disentangling sources of variance. The stated limitation is that Monte Carlo variance may conflate epistemic effects from model biases, prompt sensitivity, and decoding stochasticity with genuine ambiguity in the input utterance. Because LLM internals are opaque, the authors say these factors cannot currently be isolated.
• Improving affective resolution in non-GPT models. The systematic underestimation of affective turning points in GPT-OSS 120B, Gemma 3 4B, and Phi-4 14B raises the question of whether architecture, parameter scale, or alignment procedures are responsible, and whether any intervention closes the gap.
• Extending beyond education. The authors describe the framework as generalizable to broader social and behavioral research, but the paper only demonstrates the typology on instructional dialogue, so its transferability to other dyadic settings remains untested.

Target Audience

Researchers in affective computing and computational social science who need uncertainty-aware affect measurement without biometric sensing; NLP practitioners building emotion-aware conversational systems; learning-analytics and educational-technology researchers interested in quantifying teacher-student affective coupling; and model-selection engineers who need evidence about which LLMs preserve fine-grained emotional dynamics versus which ones systematically flatten them. Readers should be comfortable with basic statistics (means, variance, correlation) and familiar with how LLM sampling and decoding temperature work.

Authors’ abstract

Emotional coordination is a core property of human interaction that shapes how relational meaning is constructed in real time. While text-based affect inference has become increasingly feasible, prior approaches often treat sentiment as a deterministic point estimate for individual speakers, failing to capture the inherent subjectivity, latent ambiguity, and sequential coupling found in mutual exchanges. We introduce LLM-MC-Affect, a probabilistic framework that characterizes emotion not as a static label, but as a continuous latent probability distribution defined over an affective space. By leveraging stochastic LLM decoding and Monte Carlo estimation, the methodology approximates these distributions to derive high-fidelity sentiment trajectories that explicitly quantify both central affective tendencies and perceptual ambiguity. These trajectories enable a structured analysis of interpersonal coupling through sequential cross-correlation and slope-based indicators, identifying leading or lagging influences between interlocutors. To validate the interpretive capacity of this approach, we utilize teacher-student instructional dialogues as a representative case study, where our quantitative indicators successfully distill high-level interaction insights such as effective scaffolding. This work establishes a scalable and deployable pathway for understanding interpersonal dynamics, offering a generalizable solution that extends beyond education to broader social and behavioral research.