AssurAI: Experience with Constructing Korean Socio-cultural Datasets to Discover Potential Risks of Generative AI

AI summary

Overview

Research area: AI safety evaluation and benchmark dataset construction for generative AI, with a focus on Korean language and socio-cultural context.

Technical level: Intermediate. The paper combines dataset engineering methodology, annotation quality control, and standard statistical analysis (ANOVA, Levene's test, effect size, confidence intervals), but assumes familiarity with LLM safety evaluation concepts.

Scope: The paper documents the construction, quality control, and pilot evaluation of AssurAI, a 11,480-instance Korean multimodal dataset covering 35 AI risk factors across text, image, video, and audio modalities.

What This Paper Is About

Existing AI safety benchmarks are predominantly English-centric and almost entirely text-only, so they miss risks that arise specifically from Korean linguistic and socio-cultural norms and cannot assess the harmful outputs of multimodal generative models. The authors build AssurAI, a large-scale Korean multimodal safety dataset organized around 35 risk factors, produced through a two-stage process of expert seeding and crowdsourced scaling followed by triple annotation and expert red-teaming. They then run pilot evaluations with recent open-weight and commercial models to check whether the dataset effectively exposes differences in model refusals and safety behavior.

Key Contributions

1. Korean socio-cultural risk taxonomy. A multidisciplinary expert group (artificial intelligence, education, psychology) curated 35 risk factors adapted from existing frameworks, organized into six higher-level categories: Harmful & Violent Content, Interpersonal Harm, Sensitive & Adult Content, Misinformation & Manipulation, Illegal & Unethical Activities, and Socioeconomic & Cognitive Risks. Risk factors 1–30 were adapted from the AIR 2024 study and risk factors 31–35 from the MIT FutureTech study.

2. Large-scale Korean multimodal dataset. AssurAI contains 11,480 instances distributed across text (9,560), image (1,160), video (430), and audio (330), covering all 35 risk factors and eight prompt types. The dataset is released publicly.

3. Systematic, quality-controlled construction process. The dataset was built through a two-phase methodology (expert-led seed data generation covering 10% of the total target amount, followed by crowdsourcing-based mass production), triple independent annotation by three workers per instance to support Inter-Annotator Agreement computation, and an iterative expert red-teaming feedback loop.

4. Pilot validation on current models. The dataset was applied to four open-weight models (EXAONE 3.5, Llama 3.1, Mistral, Qwen 2.5) in a Text EvalTrack and to Gemini Live 2.5 Flash Preview (audio), Gemini 1.5 Flash (image), and Veo 2.0 Generate 001 (video) in a Multimodal EvalTrack.

Main Findings

• Modality and prompt-type distribution: Text accounts for 83.3% of instances, image 10.1%, video 3.7%, and audio 2.9%. By prompt type, the largest shares are Q Only (3,751 instances, 32.7%) and Role-Playing (2,362, 20.6%), while Reflection is smallest (170, 1.5%). Multi-Session accounts for 980 (8.5%), Chain-of-Thought 1,490 (13.0%), Multiple-Choice 1,430 (12.5%), Expert Prompting 727 (6.3%), and Rail 570 (5.0%).

• Uneven risk-factor coverage is intentional: The distribution across the 35 risk factors is concentrated, most notably in Discriminatory Activities (1,000 instances) and Unauthorized Privacy Violations (900 instances), because these factors were structured to encompass broad sub-scenarios.

• Text track scores are stable overall: Mean safety scores ranged between 3.3 and 3.9 with standard deviations of 0.28–0.32. EXAONE scored 3.90 (σ = 0.30, CV = 7.7%), Qwen 3.87 (σ = 0.32, CV = 8.3%), Mistral 3.79 (σ = 0.30, CV = 7.9%), and Llama 3.30 (σ = 0.28, CV = 8.5%). All CV values were below 9%.

• Models differ significantly despite stable variance: A one-way ANOVA found a significant difference across models, F(3,136) = 23.2893, p = 3.142e-12, with η² = 0.0555. Levene's test indicated homogeneity of variances (W = 0.0393, p = 9.894e-01). Post-hoc Tukey HSD testing found no significant differences among EXAONE 3.5(Q8), Qwen 2.5(Q8), and Mistral(Q8), but all three scored significantly higher than Llama 3.1(Q8) (p < .001), which the authors attribute to Llama favoring contextual reasoning over explicit refusal.

• Category-level patterns: EXAONE 3.5(Q8) and Qwen 2.5(Q8) scored particularly high in Hate Speech, Child Harm, and Privacy Violation categories, while Llama 3.1(Q8) scored lower on violent and sexual content. Mistral(Q8) showed moderate means with narrow dispersion (σ ≈ 0.30).

• Prompt type exposes behavioral differences: EXAONE 3.5(Q8) and Qwen 2.5(Q8) were more consistent in refusals on Role-Playing, Reflection, and Multi-Session prompts, while Llama 3.1(Q8) was more stable on Chain-of-Thought and Multiple-Choice. All models shared a weakness on the Rail prompt type.

• Audio results are conservative: For Gemini Live 2.5 Flash Preview across 330 evaluations, Score 5 accounted for 44.8% and Score 4 for 24.2% (roughly 69% combined); Score 3 was 24.8%, and Scores 1 and 2 were each 3.0%. By prompt type, Chain-of-Thought scored 4.14 ± 0.23 (n = 63), Q Only 4.11 ± 0.18 (n = 139), Role-Playing 3.98 ± 0.21 (n = 106), and Expert Prompting 3.73 ± 0.41 (n = 22).

• Image modality shows heavy policy blocking: With Gemini 1.5 Flash, approximately 40% of the 1,160 prompts were blocked by safety policies even after up to three repeated attempts. Mean scores ranged from 3.3 to 4.5 across prompt types, with Multi-Session highest at 4.38 ± 0.18 (n = 101) and Reflection at 4.46 ± 0.23 (n = 48). Multiple-Choice was almost entirely blocked, leaving n = 3 and a 95% CI of ± 4.97.

• Video modality shows prompt-dependent risk: For Veo-2.0-generate-001, Scores 5 (35.3%) and 4 (18.0%) combined for 53.3% of outputs, Score 3 was 14.7%, Score 2 was 3.8%, and Score 1 was 8.0%. Safely Blocked cases were 15.8%. Q Only scored 4.06 ± 0.14 (n = 274) and Rail 4.11 ± 0.56 (n = 18), but Chain-of-Thought dropped to 1.88 ± 0.51 (n = 17) and Role-Playing to 2.44 ± 0.85 (n = 16), indicating safety enforcement did not fully operate in logic-driven or role-based conditions.

• Acknowledged limitations: The 35 risk factors cannot anticipate all future risks; the dataset is deeply customized to Korea's context and may not transfer directly to other cultures; harmfulness judgments retain some subjectivity despite triple annotation and expert review; and the dataset is static and requires continuous expansion as models are updated.

Methodology in Plain English

The authors started by reviewing established AI risk taxonomies and curating a set of 35 risk factors that balance international universality, relevance to Korean society, and practical feasibility for data construction. These factors were grouped into six categories and paired with eight prompt types (Multiple-Choice, Q Only, Multi-Session, Role-Playing, Chain-of-Thought, Expert Prompting, Rail, Reflection) designed to probe different model capabilities such as inference, role-playing, and information constraints.

Construction ran in two stages. First, five specialized research laboratories each took responsibility for a subset of the risk factors and hand-crafted high-quality seed examples, which totaled 10% of the eventual target amount. These seeds acted as both guidelines for workers and a gold standard for later quality checks. Second, a data company (SelectStar) scaled production through its crowdsourcing platform using workers with prior annotation experience who received intensive training on the risk factors and prompt types.

Quality control had three pillars. Every instance was annotated independently by three workers, with judgments stored separately so the authors could compute Inter-Annotator Agreement. A multidisciplinary expert group regularly reviewed the data using a red-team approach, flagging errors, biases, or contextually inappropriate items and proposing corrective actions by prompt type, with feedback histories documented. Finally, the dataset was piloted on commercial and open-weight models to check empirically whether it discriminated among their safety behaviors, and those results fed back into earlier construction stages.

For evaluation, the authors set up two tracks. The Text EvalTrack used four open-weight models (EXAONE 3.5, Llama 3.1, Mistral, Qwen 2.5) run under Q8 quantization within the AI Inspect framework, with model outputs scored by GPT-4o-mini as the judge on a five-point rubric (1 = Risk, 5 = Safe). Audio outputs were transcribed to text and scored through the same path. The Multimodal EvalTrack used a separate custom script because AI Inspect had limited multimodal support: images were resized, converted to JPEG, and Base64-encoded, and video underwent frame sampling, quality filtering, and normalization before scoring by GPT-5-mini.

Why This Matters

Impact on research. The work extends AI safety benchmarking beyond English and beyond text, providing a replicable construction pipeline that combines expert seeding, crowdsourcing scale, triple annotation, and red-team review. It also gives researchers a way to test whether safety alignment holds under culturally specific contexts and across modalities that existing datasets such as ToxiGen, RealToxicityPrompts, SafetyBench, BBQ, KoBBQ, and Kosbi do not cover.

Real-world applications:

• Model auditing before deployment in Korea. Organizations can use AssurAI to probe whether a model produces unsafe responses to Korean-language prompts about local social, political, or legal scenarios before releasing it to Korean users.
• Multimodal product safety testing. Teams building or integrating image, video, or audio generation systems can use the multimodal portions of the dataset to check whether safety policies hold for non-text outputs and whether blocking behavior is consistent across repeated requests.
• Red-team training and annotation workflow design. The two-stage construction model, triple annotation scheme, and documented expert feedback loop offer a template for other teams building safety datasets in low-resource languages.
• Regulatory and standards work. The 35-factor taxonomy provides a structured checklist that could inform safety documentation, compliance reviews, or disclosure requirements for generative AI products targeting Korean markets.

Industry relevance. The project involved a domestic data company (SelectStar) for production and a leading AI company (Kakao) for pilot deployment, indicating direct industry engagement. The finding that commercial models block a substantial share of image prompts (approximately 40% for Gemini 1.5 Flash) while video generation scored lower on logic-driven and role-based prompts points to concrete gaps that model providers can address in their refusal and safety-enforcement strategies.

Future Directions

• Multi-judge and human-calibrated evaluation. The authors note the judge configuration was limited to GPT-4o-mini and GPT-5-mini and that no human calibration was applied in the vision-based evaluation, and they plan a multi-judge framework.
• Temporal consistency (TC) rubric enhancement. The paper explicitly calls for TC-based rubric enhancement to enable more robust verification of visual safety performance, since video was evaluated on a single representative frame and does not account for temporal coherence or scene transitions.
• Multi-frame comparative experiments. The single-frame video results are framed as a minimal proxy and baseline indicator for subsequent multi-frame experiments.
• Dataset expansion and taxonomy revision. Because the dataset is static and the 35 factors cannot predict all future risks from new AI technologies, the authors state the dataset requires continuous expansion and supplementation.

Target Audience

AI safety researchers and benchmark designers, especially those working on non-English and multilingual safety evaluation. Multimodal model developers and red teams evaluating image, video, and audio generation will find the modality-specific results directly useful. Annotation project managers and data quality specialists can learn from the two-stage construction and triple-annotation design. Policy and standards groups focused on Korean AI deployment, and Korean-language NLP researchers, are also natural readers. Readers need some familiarity with LLM safety evaluation and basic statistics to interpret the quantitative sections.

Authors’ abstract

The rapid evolution of generative AI necessitates robust safety evaluations. However, current safety datasets are predominantly English-centric, failing to capture specific risks in non-English, socio-cultural contexts such as Korean, and are often limited to the text modality. To address this gap, we introduce AssurAI, a new quality-controlled Korean multimodal dataset for evaluating the safety of generative AI. First, we define a taxonomy of 35 distinct AI risk factors, adapted from established frameworks by a multidisciplinary expert group to cover both universal harms and relevance to the Korean socio-cultural context. Second, leveraging this taxonomy, we construct and release AssurAI, a large-scale Korean multimodal dataset comprising 11,480 instances across text, image, video, and audio. Third, we apply the rigorous quality control process used to ensure data integrity, featuring a two-phase construction (i.e., expert-led seeding and crowdsourced scaling), triple independent annotation, and an iterative expert red-teaming loop. Our pilot study validates AssurAI's effectiveness in assessing the safety of recent LLMs. We release AssurAI to the public to facilitate the development of safer and more reliable generative AI systems for the Korean community.