Before You Say It: Anticipating Verbal Behavior from Longitudinal Everyday Conversations with LLMs

AI summary

Overview

Research area: Human-Computer Interaction, specifically anticipatory/proactive AI, wearable conversation sensing, and LLM-based user modeling.

Technical level: Intermediate. The paper combines wearable sensing engineering, LLM prompting pipelines, and a human study, but the core ideas (patterns of the form IF-THEN-EXCEPT, and comparing prediction methods) are explained in accessible terms.

Scope in one sentence: The paper tests whether large language models, given longitudinal conversations captured by an always-on smartwatch, can predict a specific person's likely next communicative intention in an unfolding conversation, and how participants feel about such predictions.

What This Paper Is About

Existing AI assistants are largely reactive: they answer questions or respond to explicit requests, but they do not anticipate what a particular user is about to do. The authors argue that this kind of anticipation requires longitudinal, person-specific observation of how someone behaves across everyday situations. Their goal is to build and evaluate an interpretable, LLM-based method that mines situation-dependent behavioral patterns from long-term everyday conversation and uses those patterns to predict the user's likely next verbal behavior.

Key Contributions

1. A longitudinal wearable conversation dataset comprising over 1,000 hours of naturalistic speech collected from 14 participants wearing an always-on smartwatch for 7–10 days during normal daily activities.
2. Situational Reasoning, a training-free approach that learns interpretable, situation-dependent behavioral patterns as they unfold over time, and which the authors report significantly outperforms existing LLM baselines when predicting verbal behavior from unfolding conversational context.
3. An examination of behaviors participants wanted to change, where all participants flagged patterns (114 patterns total, mean 12 per participant) they wished to change, and prediction performance on this subset was measured separately.
4. Semi-structured interviews exploring how participants perceived behavioral predictions and what forms of future behavioral support they would find useful, including follow-up interviews with 7 participants several months later.

Main Findings

• Person-specific prediction is possible at turn level: Pattern-conditioned prediction scored M: 0.597, SD: 0.350, outperforming the zero-shot baseline (M-0.463, SD-0.330) by +28.9% and the all-in-context condition (M-0.502, SD-0.339) by +18.9%.
• Performance improves as more conversational data accumulates: Pattern-conditioned prediction improved by +23.6% with accumulated data, reaching 0.597 at full accumulation, while Zero-Shot and All-in-Context remained relatively flat.
• Predictions are person-specific, not generically accurate: In a cross-participant transfer condition, where each participant's behavioral patterns were replaced with those of a randomly assigned other participant, the score dropped to M-0.460, SD-0.389, a +29.8% gap relative to pattern-conditioned prediction.
• Human raters agreed with the automated judge: With 40 independent crowdsourced raters on a randomly sampled set of 200 prediction scenarios, Pattern-Conditioned Prediction ranked first in 43% of comparisons (All-in-Context 24%, Natural-Language Summary 15%, Zero-Shot 18%), with Kendall's τ = 0.83 among raters.
• Predictions are strongest on behaviors people want to change: On the subset of participant-flagged intention-to-change patterns (n = 912), pattern-conditioned prediction scored 0.858, outperforming the zero-shot baseline (0.607) by +41.3% and the all-in-context condition (0.619) by +38.5%.
• Quality varies across participants: Pattern-conditioned prediction outperformed the baselines for most participants, with the exception of p7, where it degraded; the authors attribute this to variation in the quality of mined patterns and to lower pattern activation rates (participants with low activation rely more often on the fallback mechanism when no pattern is activated).
• Transparency was valued: P10 appreciated being able to read, correct, and set personal goals for inferred patterns.
• Participants named three recurring challenges in changing unwanted tendencies: falling back on default habits under stress or when deeply ingrained (P11, P15, P6); uncertainty in socially ambiguous situations or around authority figures (P13, P5); and difficulty noticing the pattern early enough in the moment (P6, P11, P15).
• Preferred support style: Some participants favored private, nonjudgmental reminders over interpersonal correction — P11 described a watch-based nudge as "less judgmental" and "more like information," while P15 valued reminders because they are "something I can't really avoid" (P11, P15, P6). Others described benefits from reflective self-checking, rehearsing difficult conversations, or pausing to reevaluate priorities (P13, P15). Participants also wanted situationally tailored suggestions — reframing the moment, redirecting attention, offering alternatives, reducing commitment size, or reinforcing previously stated goals (P10, P11, P5).
• Caution about mistimed interventions: Some participants were wary of overly directive interventions, especially in nuanced "gray area" situations where the system might miss context or undermine personal agency (P13).

Methodology in Plain English

Data collection. Fourteen participants (age 18+, fluent English speakers, no speech impairments) each wore an always-on smartwatch for 7–10 days and were compensated $100 USD each. The watch used on-device voice activity detection and forwarded 2–3 minute audio clips to a backend pipeline when speech was detected. Participants were instructed to tell people around them that conversations were being recorded and to obtain consent before each interaction. The study was approved under MIT COUHES #2402001229.

Audio processing. Speech was transcribed and diarized with Deepgram's nova-3-meeting model, the target participant's voice was identified using a speaker verification model trained on a 20-second enrollment sample provided at onboarding (tagging utterances as "User" or "Other"), and personally identifiable information was anonymized using SpaCy's Named Entity Recognition model (e.g., [PERSON], [ORGANIZATION]). Raw audio was deleted immediately after processing, and transcripts were stored locally using AES-256-GCM encryption.

Review and statistics. Participants reviewed their transcripts through a web interface, removing data they did not wish to share and correcting misattributed speaker labels and conversational context (such as role, situation, emotion). On average they removed 0.16% of transcribed data and flagged 2.48% of segments as misclassified. The final dataset contains 15,066 utterances across 14 participants (mean 1,256 per participant), 57% "User" and 43% "Other", with a mean utterance length of 49 words.

Cleaning. Three steps were applied: word count filtering (turns with fewer than 4 words in the user utterance or interlocutor turn were excluded); disfluency removal using a BERT-based disfluency classifier; and utterance re-segmentation using wtpsplit for sentence boundary detection. After cleaning, 9,901 utterances were retained.

Task definition. Following Verbal Response Mode (VRM) theory, verbal behavior is defined as the communicative intention an utterance serves, independent of its surface wording. Given the conversational context (all preceding utterances in the episode), the system generates a natural-language characterization of the communicative intention the user is likely to express next, at either single-utterance or multi-utterance granularity. Instantiation used Gemini 2.5 Pro prompted with the VRM taxonomy.

Conditions compared. Four methods were evaluated: (1) Zero-Shot, using only local context; (2) All-In-Context, conditioning on the full longitudinal history truncated to the most recent 500,000 characters when it exceeds the context window; (3) Natural-Language Summary, where the history is distilled into a free-form narrative of recurring tendencies; and (4) the proposed Situational Reasoning approach.

Situational Reasoning. Behavioral patterns are represented as contrastive IF-THEN-EXCEPT rules with three parts: a situational condition (when, where, with whom, and internal state), a counter-situation that suppresses the tendency, and a behavioral tendency. An example given is: "IF challenged by an authority figure, THEN the user tends to deflect, BUT NOT when challenged by a peer." Pattern probability is estimated as the ratio of supporting to total evidence, bounded in [0,1], and mapped to qualifiers — never, sometimes, often, always. Patterns with overlapping conditions are merged as evidence accumulates and are updated as new interactions arrive. At inference, patterns whose antecedent matches the current context and whose exception condition is absent are activated, weighted by their estimated probability, and supplied to the LLM alongside the current context to generate a prediction.

Evaluation. Predictions were compared against ground-truth behavioral descriptions generated from the actual utterance using the same prompt. Three dimensions were scored on [0,1] by GPT-5 as an LLM judge: Pragmatic Function Alignment (d1), Behavioral Specificity (d2), and Compositional Alignment (d3). LLM judgments were validated against 40 independent human evaluators.

Interview study. All participants reviewed a subset of inferred patterns and flagged those reflecting behaviors they wanted to change; a subset returned several months later for follow-up semi-structured interviews.

Why This Matters

Impact on research. The paper provides evidence that person-specific verbal behavior can be predicted from longitudinal conversational data, and offers an interpretable alternative to conditioning LLMs on raw long histories — which the authors note is brittle, because models struggle to surface person-specific regularities buried in long contexts and outputs are hard for users to inspect or correct. It connects wearable sensing, VRM-based dialogue-act work, and proactive-assistant research.

Real-world applications suggested by the paper:

• Just-in-time personalized reminders that surface before a user's action unfolds, such as catching an impulse purchasing pattern the user wants to change (illustrated in Figure 1).
• Private, nonjudgmental wearable nudges that prompt reflection without shame, which participants preferred over interpersonal correction.
• Rehearsal and self-checking support for difficult conversations and socially ambiguous situations.
• Situationally tailored support around behavior change goals, such as reframing the moment, redirecting attention, offering alternatives, reducing commitment size, or reinforcing previously stated goals.

Industry relevance. The approach is relevant to developers of always-on wearables, personal AI assistants with persistent memory, and behavior-change or wellbeing products. Because it is training-free and produces human-readable patterns, it is implementable with existing commercial LLMs and wearable hardware, and the mined patterns are inspectable and correctable by end users — a property the authors highlight as central to trust.

Future Directions

• Extension through reinforcement learning from user feedback, enabling adaptive refinement of patterns over time.
• Longer-horizon and cross-context user modeling across messaging, social media, and conversational platforms, since predictions were evaluated only at the turn level and cross-context generalization remains open.
• More robust fallback mechanisms for when personalized situational evidence is sparse or unavailable, because prediction performance was limited by the availability of activated patterns and low-activation participants relied more heavily on the fallback (e.g., P7).
• Improving data quality and deployment scale, since the authors note limitations from microphone noise and a limited deployment window (7–10 days per participant), and pattern mining quality varied across participants.

Target Audience

This paper is most useful to HCI and ubiquitous-computing researchers working on proactive or anticipatory interfaces, wearable sensing, and LLM-based personalization; to practitioners building personal assistants, memory-augmented agents, or behavior-change applications; and to researchers in user modeling and dialogue who are interested in interpretable, situation-conditioned representations of individual behavior. Readers seeking purely clinical or diagnostic applications will not find them here, as the work is framed around everyday conversational behavior and participant-recognized patterns rather than clinical outcomes.

Authors’ abstract

Knowing someone deeply means not just understanding what they say or do but also how they will likely think, react, and engage across situations. Such predictions could eventually inform systems to anticipate when the individual is about to deviate from their goal, catch regrettable behaviors before they are made, and surface blind spots before they take hold. While many interactive systems model users to enable more personalized interactions, most cannot make such behavioral predictions, as this often requires longitudinal observation and inference of how the individual's behaviors unfold across various everyday situations. In this work, we introduce a novel LLM-based predictive behavioral modeling approach that anticipates a user's likely behavior across everyday conversational situations. We (1) collect a longitudinal dataset of over 1000 hours of naturalistic conversations from 14 participants using a wearable smartwatch; (2) evaluate LLM-based predictions against ground truth behaviors; and (3) use semi-structured interviews to explore participants perceptions of behavioral predictions and their views on possible forms of future behavioral support. Altogether, our findings provide evidence that person-specific verbal behavior can be predicted from longitudinal conversational data. This opens up new possibilities for potential future context-aware, anticipatory, proactive and personalized AI systems.