TIDES: A Longitudinal Bilingual Dataset for Modeling Multi-Party Social Dynamics

AI summary

Overview

Research area: Natural Language Processing / computational social science — multi-party dialogue modeling, corpus construction, and socially-aware conversational AI.

Technical level: Intermediate. The dataset construction is accessible to social scientists and HCI researchers, while the modeling experiments (fine-tuning, logit scoring, transfer learning) assume familiarity with standard NLP evaluation practice.

Scope: This paper introduces TIDES, a semester-long bilingual corpus of 12 real university project teams with layered social annotations, and uses it to test whether models can learn group turn-taking structure — and whether that structural skill translates into human-preferred utterance generation.

What This Paper Is About

Large language models handle one-on-one conversation reasonably well but still struggle in group settings, where they must track who is speaking, what social role each person occupies, and how a team's dynamics shift over weeks. Existing multi-party datasets are mostly short lab sessions, scripted media, or synthetic dialogue, so they miss the real stakes, shared history, and evolving relationships of genuine teamwork. TIDES addresses this by recording 12 student project teams across a full semester and annotating both what was said and the social structure beneath it.

Key Contributions

1. A longitudinal, in-the-wild bilingual corpus. 75,971 utterances from 88 dated meetings (~110 hours) across 12 teams (50 participants), spanning 6–12 weeks in Korean and English, captured during real graded coursework rather than contrived lab tasks.

2. Layered socio-structural annotations at three timescales. Utterance-level interaction types using a 15-category extension of act4teams-SHORT (5,705 human-labeled gold; 70,266 model-labeled silver), meeting-level emergent roles derived from post-meeting peer surveys mapped onto the TRIAD framework (352 role labels), and meeting-level Tuckman development stages, plus silence gaps.

3. Empirical demonstration that structure alone predicts turn-taking. Fine-tuned models reach 64.53% next-speaker accuracy, beating a bigram baseline by 13.8 points and exceeding proprietary zero-shot models, while a text-free structural condition (speaker IDs, roles, stages only) matches or surpasses text-based variants at 65.74%.

4. A cautionary finding on structure-versus-naturalness. Despite strong predictive performance, human evaluators significantly preferred vanilla model outputs over fine-tuned ones for naturalness, coherence, and speaker consistency — exposing a mismatch between modeling conversational structure and generating conversation humans find convincing.

Main Findings

• Next-speaker prediction improves sharply with fine-tuning. The primary condition (Speaker + Utterance) reaches 64.53% versus a 50.75% bigram baseline and 43.56% for the untuned Gemma-3-12B, surpassing Opus 4.6 (63.25%) and GPT-5.4 (61.97%) in zero-shot.

• Text is not required for turn-taking. Removing all utterance content and keeping only structure (speaker IDs, annotated roles, Tuckman stages) yields 65.74%, the best result in the study. Team-specific lexical patterns apparently hinder transfer, whereas structural regularities generalize.

• Intention prediction remains hard. The best fine-tuned model scores 32.96% across 14 substantive classes, only marginally above the 30.05% majority baseline, and no zero-shot model beats that baseline. Unlike speaker prediction, this task depends on utterance text.

• Team-specific adaptation happens fast. In single-team chronological analyses, accuracy rises steeply within the first three to four meetings (Team 10: 50.87% with one meeting to 62.62% with four; Team 5: 50.00% to 63.84% by meeting three, then plateauing near 65%). The authors treat this as descriptive evidence, since the held-out set shrinks as training grows.

• Longitudinal signal appears in prediction accuracy. Accuracy climbs from 57.8% in Forming-stage meetings to 70.3% in Performing-stage meetings (+12.5 points), consistent with the theory that established teams develop more predictable interaction patterns.

• Cross-corpus transfer is competitive but balance-sensitive. On the AMI Meeting Corpus, a balanced TIDES+AMI mix reaches 45.79%, within 2.06 points of published state of the art while using roughly 42% less training data. An unbalanced 170K-example mix drops to 30.02%, indicating that data balance matters more than volume.

• Fine-tuning helps structural coherence in generation but not human preference. FT-Reason achieves 57.0% one-turn speaker accuracy versus 49.0% for FT-Plain and 14.0% for Vanilla, with far fewer invalid speaker assignments. Yet human judges preferred Vanilla: FT-Plain won only 9.3% of judgments versus Vanilla's 21.1% (p < .001), and FT-Reason 12.7% versus 21.9% (p = .004). The gap persisted after post-processing FT outputs for punctuation and truncation.

• Task-oriented behavior dominates the corpus. Giving Information accounts for 35.29% of utterances, followed by Active Listening (13.09%) and Linking Solutions (12.19%); negative categories appear rarely. Problem Solver (17.61%), Coordinator (16.76%), and Critic (15.91%) are the most common emergent roles.

Methodology in Plain English

The researchers recruited 12 student teams from full-time Korean universities during the Fall 2025 semester, spanning design, computer science, and industrial engineering courses. Teams of 3–5 members recorded all their project meetings and submitted audio after each session. After every meeting, each member filled out a survey rating teammates on dominance, sociability, and task orientation, which let the researchers assign an emergent role per person per meeting. At the end, teams collectively reviewed their meetings and agreed on which Tuckman development stage each one represented.

Converting 110+ hours of bilingual audio into research-ready transcripts took a six-stage pipeline: Whisper Large-V3 transcription, custom speaker diarization combining pyannote 3.1 with ECAPA-TDNN embeddings, removal of personally identifiable information (Microsoft Presidio for English teams, a locally run Qwen3-30B-A3B for Korean teams, since no comparable off-the-shelf Korean tool exists), translation of Korean transcripts to English, unification of speaker identities across sessions using WeSpeaker embeddings and the Hungarian algorithm, and team-member validation against the original audio. Utterance boundaries were then refined with GPT-5-mini using established transcription conventions.

For interaction-type labels, 260 crowdworkers on Prolific each annotated 100 utterances, with three annotators per item and majority voting producing 5,705 gold labels. Because human agreement was only moderate (Fleiss' κ = 0.400) and the 15-category scheme is subjective, the researchers fine-tuned Gemma-3-12B on the gold data and used it to label the remaining 70,266 utterances, marking gold versus silver provenance in the release.

Modeling used LoRA fine-tuning on Gemma-3-12B-IT with 5-turn context windows, holding out Teams 6 and 7 for testing and Team 4 for validation. Three context conditions isolated the effect of content versus structure: SU (speaker plus utterance text), SRU (adding roles and stage), and S+R (structure only, no text). A parallel generation experiment compared fine-tuned plain generation, fine-tuned with explicit social-cue reasoning, and vanilla generation, assessed through automatic metrics, 1,142 human pairwise judgments from 66 Prolific evaluators, and an LLM-as-judge setup.

Why This Matters

Impact on research. The paper supplies what the field has lacked: a longitudinal, naturalistic, bilingual multi-party corpus with social structure annotations rather than just transcripts. It also delivers a methodologically important negative result — that optimizing structural prediction does not yield human-preferred generation — which challenges an implicit assumption in much socially-aware dialogue agent work. The fast-adaptation finding (three to four meetings suffice) and the transfer results on AMI give concrete baselines for future work.

Real-world applications:

• Meeting assistants that track participation balance and surface under-contributing or dominating members in long-running teams.
• Team facilitation tools that detect development-stage transitions (for example, Forming to Storming) and adapt their interventions accordingly.
• Socially-aware conversational agents for group settings — classrooms, collaborative workspaces, online multiplayer — that decide when and whether to speak.
• Cross-cultural and multilingual collaboration research, since the corpus includes both Korean- and English-speaking teams with matched annotation schemes.

Industry relevance. Organizations running distributed teams could use the corpus and methods to build meeting-analytics products that model role emergence and interaction quality over months rather than single sessions. The finding that structural features generalize across corpora while lexical ones do not is directly actionable for teams building turn-taking or speaker-prediction systems that must deploy to new groups with minimal data. The generation results also warn product teams against assuming that a model which predicts conversation well will produce conversation users accept.

Future Directions

1. Bridge the structure–generation gap. The authors explicitly call for investigating how structural understanding can be made to support natural utterance generation, noting that the mismatch between predictive accuracy and human preference remains unresolved — and that their evaluators were outsiders to the teams, so domain familiarity cannot be fully ruled out as a confound.

2. Add general dialogue-act annotations. Extending TIDES with AMI-DA or MRDA labels would improve comparability with prior work and broaden downstream task applicability, addressing the moderate inter-annotator agreement of the current 15-category scheme.

3. Model dynamics across the full team lifespan. The current experiments evaluate local prediction in short context windows. Predicting role transitions, detecting phase shifts in real time, and adapting to evolving team norms across months are named as open problems the dataset is designed to support.

4. Explore cross-linguistic and cross-cultural variation. With Korean and English teams in the corpus, the authors want to examine whether the evolution of within-team social dynamics differs across primary languages and cultural contexts.

Target Audience

This paper serves NLP researchers working on multi-party dialogue, turn-taking, and socially-aware conversational agents; HCI and CSCW researchers studying team collaboration and meeting technologies; organizational and social scientists interested in computational approaches to team development, emergent roles, and Tuckman's model; and industry practitioners building meeting-analytics or group-facilitation systems who need realistic data and honest baselines. Researchers focused on low-resource and bilingual corpora will also find the six-stage processing pipeline and the locally-run Korean anonymization approach directly reusable.

Authors’ abstract

Group conversations are fundamental to human collaboration, yet standard large language models (LLMs) still struggle with the complexities of multi-party interaction. This challenge persists in part because existing group conversation datasets are often limited to short-term lab settings with contrived tasks, failing to capture the long-term social dynamics of real-world teams. To bridge this gap, we introduce TIDES, a high-resolution longitudinal dataset tracking 12 university project teams over a full semester. Comprising 75,971 utterances in both English and Korean from in-person meetings, TIDES provides a naturalistic record of teams working on self-managed projects. Our socio-structural annotations-covering interaction types, emergent roles, and development stages-allow for modeling of team evolution over months. Experiments show that fine-tuning on TIDES improves next-speaker prediction by 13.8 percentage points over a bigram baseline (64.53%) and yields performance comparable to strong proprietary zero-shot models. The model also comes within 2.1 percentage points of the published state of the art on the AMI Meeting Corpus while using approximately 42% less training data. However, human evaluations suggest that better next-speaker prediction does not necessarily yield more natural or coherent utterances, as fine-tuned models were generally less preferred than vanilla models. This potential mismatch motivates further study of how structural modeling can support natural multi-party generation.