ARCADE: A City-Scale Corpus for Fine-Grained Arabic Dialect Tagging

AI summary

Overview

Research area: Natural Language Processing, specifically Arabic dialect identification (ADI) and speech corpus construction; sits at the intersection of computational linguistics, speech processing, and geolinguistics.

Technical level: Intermediate. The paper is a dataset/resource paper. It requires some familiarity with speech corpora, annotation schemes, and evaluation metrics such as Cohen's Kappa, but it contains no model training, no neural architecture, and no benchmarking experiments.

Scope: This one sentence: The paper describes ARCADE (Arabic Radio Corpus for Audio Dialect Evaluation), a corpus of 3,790 unique 30-second Arabic radio clips spanning 58 cities in 19 countries, annotated with city-level dialect labels plus emotion, speech type, keep/skip, and confidence metadata.

What This Paper Is About

Most Arabic dialect identification resources label speech at the country or broad-region level (for example "Egyptian" or "Levantine"), which makes it impossible to study variation between cities inside the same country. The authors set out to build the first Arabic speech dataset designed explicitly with city-level dialect granularity by automatically recording publicly available Arabic radio streams and having native speakers annotate each clip with rich metadata. The goal is to release a corpus that makes fine-grained dialect attribution from audio measurable in a way that earlier datasets could not support.

Key Contributions

1. A radio-based recording pipeline. The authors designed and implemented a pipeline that discovers and records Arabic radio streams from public directories, validates each stream, and captures 30-second clips at random intervals, filtered toward monologue-like speech. Clips are stored with structured filenames encoding city and timestamp (<city>_<timestamp>.mp3) along with station name, city, country, start time, and stream URL.

2. A city-level annotated corpus with rich metadata. ARCADE comprises 6,907 annotations over 3,790 unique audio segments from 58 cities across 19 countries, totaling 31.6 hours. Each clip carries labels for emotion, speech type, keep/skip decision, dialect category (MSA, dialect, mixed, or not applicable), and annotator confidence, plus annotator identity and timestamp for traceability.

3. A full descriptive analysis of the corpus. The paper reports distributions of emotion, speech type, keep/skip, dialect category, and confidence, along with geographic coverage and inter-annotator agreement metrics for every annotation category.

4. A transparent collection and annotation protocol, including a custom tool. The authors built a Gradio-based web annotation interface with standardized dropdowns, sequential navigation, and a real-time statistics dashboard, and they release the protocol to support future expansions and community contributions.

Main Findings

• City-level granularity is the distinguishing feature. The corpus spans 58 cities across 19 countries, and the paper states this is the largest number of Arab countries covered among compared resources with verified annotations. Comparison targets include ADI17 (~1M clips, 3,000 hours, country-level, weak channel-based labels, 17 countries), MGB-2 (~550k clips, 1,200 hours, regional, lightly supervised), MGB-3 (13,825 clips, 16 hours, 5 countries, manually verified), MGB-5 (~1.06M clips, 3,000 hours, country-level, weak channel-based), QASR (1.6M clips, 2,000 hours, regional, lightly supervised), and CALLHOME (120 calls / ~60 hours, Egyptian, manually verified).

• Roughly two-thirds of clips were retained. Of 6,907 annotated clips, 4,539 (65.7%) were marked "keep" for dialect identification, while 34.3% were marked "skip," predominantly because of Quranic recitation, music, or crosstalk.

• Audio type distribution. Regular single-speaker speech makes up 37.3% of segments, multiple speakers 35.4%, music or non-speech 22.7%, and Quranic recitation 4.6%. The authors excluded recitation because it follows the standardized phonology of Classical Arabic and often features transnational reciters, which decouples speaker origin from station location.

• Dialect category distribution. Dialectal speech accounts for 41% of clips, MSA 21%, mixed 18%, and not applicable 20% (where music or background noise overwhelms the voice). MSA clips were retained because regional origin can still be inferred from accentual cues, producing distinct "regional standards" of MSA.

• Emotion is heavily skewed. Neutral dominates at 87.8% of annotations, with excitement, happiness, sadness, frustration, anger, surprise, and fear appearing much less frequently. Emotion annotations were mainly applied to retained segments; 42 retained recordings are missing emotion annotations and are excluded from that graph.

• Annotation redundancy was uneven but confidence was high. 14.4% of the corpus received three annotator passes, 53.4% received two, and 32.2% received a single pass (so 67.8% were validated by multiple annotators). Confidence labels were 91.9% "sure," 7.1% "unsure," and 1.0% "no idea."

• Inter-annotator agreement varies by category. Keep/skip achieved 91.76% raw agreement with Cohen's Kappa 0.507; speech type 87.71% with Kappa 0.586; MSA/dialect 83.16% with Kappa 0.310; emotion 90.53% with Kappa 0.179. The authors attribute the low Kappa values for MSA/dialect and emotion to Cohen's Kappa's sensitivity to class imbalance, and note that distinguishing MSA from dialect is genuinely hard when speakers code-switch or produce regionally accented MSA.

• Audio quality was measured on four signal metrics. Mean SNR centers at 13.24 dB with values ranging from about −10 dB to over 50 dB; mean silence ratio is 15.98%; mean dynamic range is 15.83 dB, concentrated between 10–20 dB; mean spectral centroid is around 1602 Hz.

• Retained clips are acoustically better as a byproduct. Kept segments show mean SNR of 15.25 dB versus 9.36 dB for skipped clips, higher silence ratio (19.30% vs. 9.56%), higher dynamic range (16.94 dB vs. 13.72 dB), and lower spectral centroid (1535 Hz vs. 1731 Hz). The authors note that quality was not an explicit filtering criterion; the improvement results from content-based filtering.

• Geographic coverage and quality are imbalanced. Algeria contributed 1,509 clips (21.8%) and Bahrain 1,014 (14.7%), while Oman (0.8%) and Tunisia (0.6%) contributed the least. In the spatial visualization, North African cities in Morocco and parts of Algeria show high-quality recordings (mean SNR > 25 dB), as do Riyadh and Kuwait, while certain cities in Sudan and parts of the Levant show lower signal quality.

• No modeling results are reported. The paper explicitly leaves benchmarking, dialect classification experiments, and multi-task modeling to future work.

Methodology in Plain English

The authors did not collect speech in a lab. Instead, they built a system that automatically finds publicly available Arabic radio streams — from Radio Garden, World Radio Map, and other public radio directories — verifies that each stream is reachable and of usable quality, and then records 30-second clips at random times of day. The 30-second length was chosen deliberately: prior work cited by the authors shows 30-second segments give strong performance for Arabic dialect identification (exceeding 81% accuracy), and shorter clips also make it more likely that a clip contains one speaker and one consistent speech style. Cities were chosen based on population, media prominence, and geographic spread. The target was a minimum of 10 kept recordings per city and at least 5 cities per country, though the second target could not be met consistently because several countries lack 5 identifiable radio stations from different cities broadcasting online.

Annotation was done by 11 trained native Arab annotators through a custom Gradio-based web interface. For each clip, annotators played the audio and assigned labels from standardized dropdowns: emotion, speech type (regular single speaker, multiple speakers, no speech/music, or Quran recitation), keep/skip, dialect category (MSA, dialect, mixed, or not applicable), and confidence (sure, unsure, or no idea). The confidence field exists mainly to flag cases where a non-local speaker appears on a local station. The interface included navigation controls and a live statistics dashboard so the team could monitor progress and consistency.

The paper then analyzes the resulting corpus descriptively: label distributions, country and city counts, inter-annotator agreement (raw percentage agreement plus Cohen's Kappa for each category), and four acoustic measurements per clip (SNR, silence ratio, dynamic range, and spectral centroid) computed to characterize audio quality and to compare retained versus skipped clips.

Why This Matters

Impact on research. Existing Arabic speech resources such as ADI17, MGB-2, MGB-3, MGB-5, QASR, and CALLHOME provide country- or region-level labels at best, and several rely on weak supervision where dialect labels are inferred from channel metadata rather than verified by humans. ARCADE offers verified, multi-annotator, city-level labels, which makes it possible to study intra-country dialect variation, urban versus rural speech, and the confounding effects of recording quality that earlier corpora could not isolate. The paper also provides a reusable protocol so others can extend the corpus.

Real-world applications (as framed by the paper).

• Fine-grained dialect attribution and audio geolocation/geolinguistics, including identifying a speaker's sub-regional origin from speech.
• Sociolinguistic studies of dialectal variation across the Arab world, supported by city-level metadata.
• Robustness evaluation of speech systems under channel and domain shift, since the corpus contains genuinely heterogeneous broadcast audio.
• Transfer learning to related media such as telephony and online video, which the authors list as a target domain.

Industry relevance. The paper notes that finer-grained dialect signals benefit applications that coarse country-level labels cannot serve, and it raises culturally aligned speech models as a motivation. Voice assistants, contact-center routing, content recommendation for regional audiences, and broadcast/media analytics are the kinds of systems that would draw on this kind of label resolution. The paper itself does not report any deployed system or industry partnership; those connections are framed as motivation and potential use cases rather than demonstrated outcomes.

Future Directions

1. Benchmarking and modeling. The authors explicitly defer all experimental benchmarking and modeling, including multi-task learning frameworks that jointly predict dialect category, emotion, and sub-regional origin, and contrastive approaches that align audio representations with spatial identifiers.

2. Correcting geographic and emotional imbalance. Some cities are over-represented (Algeria and Bahrain) and positive emotions are under-represented. The paper suggests targeted data collection in under-sampled regions such as rural areas, collecting programming types that elicit varied emotions, and using sampling strategies or weighted loss functions during training.

3. Better filtering and denoising. The radio environment introduces noise, music, and crosstalk; the authors propose improved filtering or adaptive denoising to raise signal quality.

4. Separating acoustic degradation from dialectal signal. Because SNR varies systematically by city, the paper warns that models may conflate audio quality with dialectal features, and flags this as an issue worth addressing in future modeling work.

Target Audience

This paper is most useful to researchers building or evaluating Arabic speech and dialect identification systems, and to dataset builders who want a concrete template for geographically fine-grained speech collection and multi-annotator labeling protocols. It also suits sociolinguists studying regional Arabic variation, speech-processing engineers who need a domain-shift or robustness test set, and machine learning practitioners working on multilingual or culturally specific speech applications. Readers looking for model architectures or benchmark leaderboard numbers will not find them here; the paper is intentionally a data-description contribution with all modeling left to subsequent studies.

Authors’ abstract

The Arabic language is characterized by a rich tapestry of regional dialects that differ substantially in phonetics and lexicon, reflecting the geographic and cultural diversity of its speakers. Despite the availability of many multi-dialect datasets, mapping speech to fine-grained dialect sources, such as cities, remains underexplored. We present ARCADE (Arabic Radio Corpus for Audio Dialect Evaluation), the first Arabic speech dataset designed explicitly with city-level dialect granularity. The corpus comprises Arabic radio speech collected from streaming services across the Arab world. Our data pipeline captures 30-second segments from verified radio streams, encompassing both Modern Standard Arabic (MSA) and diverse dialectal speech. To ensure reliability, each clip was annotated by one to three native Arabic reviewers who assigned rich metadata, including emotion, speech type, dialect category, and a validity flag for dialect identification tasks. The resulting corpus comprises 6,907 annotations and 3,790 unique audio segments spanning 58 cities across 19 countries. These fine-grained annotations enable robust multi-task learning, serving as a benchmark for city-level dialect tagging. We detail the data collection methodology, assess audio quality, and provide a comprehensive analysis of label distributions. The dataset is available on: https://huggingface.co/datasets/riotu-lab/ARCADE-full