A Survey of Large Models in Sports

AI summary

Overview

Research area: Natural language processing and multimodal large models applied to sports, spanning sports understanding, analysis, and interaction.

Technical level: Intermediate. The survey is organized around task definitions, modeling paradigms, and evaluation metrics rather than derivations, so readers need general familiarity with LLMs, MLLMs, fine-tuning, and retrieval-augmented generation.

Scope: A survey of 241 academic papers (published January 2020 through July 2025) that applies large models to sports, organized into 6 stakeholder groups and 19 tasks, with a companion analysis of sports datasets and benchmarks.

What This Paper Is About

Sports generates enormous amounts of data, but turning that data into useful language-based insight for athletes, coaches, referees, fans, and industry staff has historically required separate, task-specific systems. The paper surveys how (multimodal) large language models are now being applied across the whole sports ecosystem, cataloguing which tasks exist, which datasets and benchmarks support them, and how well different modeling approaches actually perform. Its goal is to consolidate fragmented research into a single taxonomy and to identify the field's structural gaps.

Key Contributions

1. A two-level taxonomy of sports applications. The authors group work into 6 stakeholder groups — athletes and trainers, coaches and educators, referees, researchers, fans and social media, and the sports industry — and break these into 19 specific tasks, each analyzed in terms of task definition, technical paradigm, and common evaluation metrics.

2. A multi-dimensional analysis of sports datasets. The survey separates task-specific datasets from sports understanding datasets, then analyzes the landscape by sport type, application, modality, annotation source, and modeling paradigm.

3. A quantitative comparison of modeling paradigms. Table 1 puts fine-tuned, trained-from-scratch, few-shot, and zero-shot approaches side by side on four representative tasks, showing where pre-training helps and where it does not.

4. A structured statement of challenges and future directions. The paper distills 4 fundamental deployment barriers and 4 corresponding research directions, plus an openly maintained GitHub repository of the reviewed literature.

Main Findings

• The field grew sharply. Papers on large models in sports went from just 1 in 2020 to 78 in 2024, with continued growth in 2025. The final corpus consists of 241 core papers selected from roughly 2,200 candidate records.

• Pre-trained models beat from-scratch models in open-ended generation. On the YourSkatingCoach dataset, a fine-tuned T5 reaches a BLEU-4 of 0.27, while a vanilla Transformer trained from scratch reaches only 0.04.

• Visual pre-training alignment matters more than textualized visual cues for action spotting. On the SoccerNet-v2 benchmark, fine-tuned Soccer-CLIP achieves 75.7% t-AmAP, slightly ahead of the specialized COMEDIAN spatiotemporal Transformer at 73.1%, while a Llama 3.1-8B approach using textual commentary in a few-shot setting reaches only 60.8%.

• Specialized small models can still beat general MLLMs on precision scoring. On the FineFS benchmark for action quality assessment, Beats-to-Scores (a video-audio fusion Transformer) reaches 0.88 Spearman's rho, ahead of fine-tuned InternVL2 at 0.86 and Qwen2-VL at 0.75 — the authors attribute this to explicit audio-visual alignment.

• Task-adapted MLLM architectures dominate semantically rich commentary generation. On SoccerNet-Caption, MatchVoice reaches a CIDEr of 38.42 and SoccerComment 36.58, well above the from-scratch SN-Caption encoder-decoder at 23.74, while zero-shot Video-LLaMA nearly fails at 3.44 — evidence that domain-specific fine-tuning or RAG is required to bridge raw visual signals and professional terminology.

• Dataset coverage is heavily skewed. Soccer leads with 74 datasets, while individual disciplines such as cycling and boxing have only a single dataset each. Fitness-related datasets number 21 and esports datasets 3.

• Some stakeholder groups are severely underserved. Referee-relevant datasets account for only 2% of the collection, and data for sports researchers and the sports industry is described as scarce, despite robust coverage for athletes, coaches, and fans.

• Expert-level annotation is rare. In general datasets, expert annotation accounts for only 6% of labels, compared to 41% in specialized sports understanding benchmarks.

• Generative tasks now dominate. Generative-oriented datasets make up 78.5% of the landscape, compared to 21.5% for discriminative-oriented datasets, which the authors read as a shift from categorical labeling toward open-ended synthesis and reasoning.

• Four deployment barriers remain. Bias, fairness and privacy (skew toward popular sports, elite competitions, and men's sports); real-time and high-FPS understanding (latency, hours-long broadcasts, millisecond-scale events lost to temporal downsampling); hallucination and interpretability; and practicability in real-world conditions such as occlusion, shifting camera viewpoints, and low-quality footage.

• Several tasks lack mature methodology. Sports injury and rehabilitation is described as in its infancy with no unified evaluation metrics; tactical analysis relies mainly on prompt engineering rather than post-training; and sports narratives, highlight generation, and sports psychology lack standardized benchmarks.

Methodology in Plain English

Rather than running new experiments, the authors performed a structured literature review. They started from 3 existing survey papers chosen for comprehensiveness, recency (2024–2025), academic standing, and their role as citation hubs. From those seeds they applied systematic snowballing — reading reference lists backward and using Google Scholar's "Cited by" forward search — with a Boolean filter of "Sports" AND ("Large Language Model" OR "LLM" OR "GPT" OR "BERT" OR "T5") to keep the forward search on topic.

The process was repeated iteratively, adding newly qualifying papers as new seeds, until a filtered search returned no new relevant papers. Inclusion required that a study use large models as a core methodological component, be a full-length technical contribution (peer-reviewed papers or preprints, excluding editorials, posters, extended abstracts, and short papers), fall within January 1, 2020 to July 31, 2025, and be written in English. This screening reduced roughly 2,200 candidates to 241 core papers. The authors then built their taxonomy, tabulated datasets and benchmarks, and compared reported performance across modeling paradigms.

Why This Matters

Impact on research. The survey gives the field a shared vocabulary — a fixed set of 6 stakeholder groups and 19 tasks — plus a consolidated map of which datasets exist and which are missing. The quantitative tables make it possible to compare paradigms across tasks rather than within a single paper, and the dataset analyses expose concrete structural gaps (sport coverage, expert labels, referee tools) that individual studies would not reveal.

Real-world applications supported by the reviewed work:

• Athletes and trainers using LLM-generated personalized exercise prescriptions, training plans, and motion instruction derived from 3D skeletal data for sports such as figure skating and boxing.
• Coaches and analysts using action spotting, action quality assessment, and game or player performance prediction to prepare strategy.
• Referees using explainable MLLM assistance — X-VARS, fine-tuned with QLoRA, is described as a first step toward explainable LLMs for refereeing.
• Broadcast and social media workflows covering commentary generation, highlight generation, sports news generation, and public opinion analysis.

Industry relevance. The paper notes that dataset availability aligns with the commercial popularity of segments — athletes, coaches, and fans are well covered, while the sports industry and referees lag. It also identifies sports management (financial, database, and facility management over structured and unstructured documents), talent scouting, and sports tourism as emerging industrial uses. The authors acknowledge as a limitation that their analysis centers on academic research and that discussion of commercial and industrial systems remains limited.

Future Directions

1. Trustworthy sport AI. Implement rigorous data balancing and cleaning alongside sport-specific alignment via Reinforcement Learning from Human Feedback to reduce bias — particularly the underrepresentation of women's sports, Paralympic sports, youth programs, and school sports.

2. Streaming and long-video mechanisms. Develop streaming inference with more efficient KV-cache and attention mechanisms for low latency, adapt long-video modeling via memory, parallelism, and token compression or architectures like Mamba for multi-hour matches, and build vision backbones that handle dense high-frame-rate frames without losing fine-grained dynamics.

3. Knowledge grounding and tool use. Improve factual reliability through explicit retrieval from structured knowledge with RAG, grounding claims in visual evidence, producing explicit rationales, and enabling tool-enabled models that query live databases, rule engines, or match-tracking APIs.

4. In-the-wild evaluation and edge deployment. Build ecologically valid, workflow- and latency-aware benchmarks that stress-test models under real sports conditions, and enable on-device inference through quantization, knowledge distillation, and mobile (M)LLMs for wearables and mobile platforms.

Open questions the survey raises but does not answer include whether unified evaluation metrics can be established for injury management, psychology, narratives, and highlight generation, and whether the field's skew toward a few dominant sports can be corrected given the underlying data imbalance.

Target Audience

Researchers and graduate students entering the intersection of large models and sports who need a map of tasks, datasets, and benchmarks before choosing a problem; practitioners in sports analytics, broadcasting, and team operations evaluating which tasks are technically mature; and dataset or benchmark builders looking for documented coverage gaps. Readers seeking detailed experimental methodology or commercial system analyses should look elsewhere, since both are explicitly outside this survey's scope.

Authors’ abstract

Sports have witnessed growing global enthusiasm in recent years, serving as a vital force for physical health, cultural exchange, social connection, and economic growth. The rapid advancement of large models, particularly (multimodal) large language models (M)LLMs, has demonstrated transformative potential to reshape sports understanding, analysis, and interaction across diverse domains. This paper presents a comprehensive survey of large models in sports, including (i) an overview of tasks and applications across different participant groups; (ii) a detailed analysis of sports-related datasets and benchmarks; and (iii) a critical discussion of current challenges and future directions. Our goal is to establish a foundation for advancing research and practical development of large-model-driven sports intelligence. An open-source GitHub repository is maintained at: https://github.com/Road2Redemption/Awesome_Large_Models_In_Sports1.