Aligning Cultural Behaviors in Arabic Language Models via Preferences

AI summary

Overview

Research area: Natural language processing — cultural alignment and preference-based post-training of large language models, focused on Arabic and the 22 Arab League countries. Technical level: Intermediate (assumes familiarity with supervised fine-tuning, preference optimization such as DPO/KTO, and LLM-as-a-judge evaluation). Scope: The paper introduces ArabPref, a bilingual (English and Modern Standard Arabic) pan-Arab preference dataset plus a multiple-choice cultural benchmark, and uses it to compare SFT, DPO, and KTO alignment across three open-weight models and reference proprietary models.

What This Paper Is About

Arabic language models have improved quickly, but existing work mostly tunes them for factual knowledge and instruction following rather than for country-specific cultural behavior — what is polite, expected, or discouraged in a given social situation. Because the Arab world spans 22 countries with shared but distinct norms, treating Arabic speakers as one homogeneous group hides exactly the differences that matter. The paper builds a large, native-speaker-validated dataset of preferred versus discouraged behaviors conditioned on country, then tests whether preference-based training actually makes models behave more appropriately.

Key Contributions

1. ArabPref dataset: described as the first large-scale pan-Arab behavioral preference dataset, covering all 22 Arab League countries, 16 cultural categories, and 200 fine-grained aspects, released in English and Modern Standard Arabic. It contains 31,060 training and 6,264 test preference pairs.
2. Native-speaker-validated MCQ benchmark: 1,984 multiple-choice questions (992 English, 992 Arabic) for evaluating culturally aligned behavior, built only from held-out country–aspect pairs so that no training prompt or response is reused as a question stem or answer option.
3. Systematic alignment comparison: supervised fine-tuning (SFT), Direct Preference Optimization (DPO), and Kahneman–Tversky Optimization (KTO) applied to multilingual (Llama-3.1-8B-Instruct, Qwen-2.5-7B-Instruct) and Arabic-centric (Jais-adapted-7B-chat) models, evaluated on both free-form generation and the MCQ benchmark, alone and combined with the existing CARE and Palm datasets.
4. A public evaluation foundation: the benchmark and data are released so that culturally aligned behavior in Arabic models can be studied and compared systematically.

Main Findings

• Alignment helps, but no method wins everywhere: training on ArabPref improves culturally aligned behavior and downstream cultural reasoning, yet the best method varies by model, language, and task. In the LLM-as-a-judge generation evaluation, Jais improved most consistently, with ArabPref + CARE (SFT) raising its English average from 3.74 to 4.62 and its Arabic average from 3.54 to 4.43 on a 1–5 scale.
• Qwen shows a language trade-off: Qwen-2.5-7B-Instruct reached its best English average of 4.58 with ArabPref + CARE (SFT), but its Arabic average fell from 3.98 to 3.57, driven mainly by a Literacy drop from 3.98 to 2.60. ArabPref + CARE (KTO) instead gave it the highest Arabic average of 4.47.
• Llama improves mainly in English: its best Arabic average was 3.69 with ArabPref + CARE (DPO).
• SFT is weak on multiple-choice, DPO/KTO are stronger: SFT gave limited MCQ benefit and sometimes reduced accuracy (Palm SFT lowered Qwen from 63.3 to 63.0 English and 57.6 to 56.9 Arabic, and Jais from 49.6 to 40.4 English). ArabPref + CARE (DPO) produced the largest Qwen gain, to 68.6% English (+5.3) and 58.7% Arabic (+1.1), and lifted Jais to 43.1% Arabic (+3.7).
• Combining general and culture-specific preference data helps most: merging CARE with ArabPref gave the best overall MCQ result (68.6% English for Qwen), suggesting the two data sources are complementary.
• Proprietary models remain ahead: zero-shot Claude-Sonnet-4 had the highest English generation average (4.92), while DeepSeek-V4-Flash led in Arabic (4.77). On MCQ, Qwen3.6-Plus scored 80.5% English / 76.5% Arabic and GPT-5.4 79.2% / 75.9%, above the best fine-tuned open-weight results. The paper notes the gaps are modest on some dimensions — Jais matched or beat DeepSeek-V4-Flash on English General Quality (4.77 vs. 4.56), and Qwen beat Qwen3.6-Plus and Claude-Sonnet-4 on Arabic General Quality (4.68 vs. 4.63 and 4.36).
• Scaling helps but plateaus: MCQ accuracy rises with model size; Qwen3 at 32B reached 73.9% English and 71.4% Arabic. Jais-adapted improved Arabic from 49.2% at 7B to 55.3% at 13B but not English. English accuracy was consistently above Arabic across all three families.
• Head-to-head wins favor the merged data: under DPO, ArabPref + CARE won in five of six model–language comparisons, with the biggest gap for Jais in English (58.5% vs. 30.3% for CARE-only); Jais in Arabic was the exception (49.6% vs. 42.6%). Under SFT, ArabPref + CARE was preferred in all six comparisons, including 82.8% English wins for Jais.
• Monolingual versus combined training is architecture-dependent: Llama and Jais performed best with the whole combined dataset (e.g., Llama English 3.78 whole vs. 3.53 English-only training), while Qwen was more sensitive to the language of the training data and did not improve with the combined set.
• Culture varies by country within the dataset: alcohol guidance ranges from strict prohibition (Saudi Arabia, Libya) to venue-restricted permissibility (UAE, Jordan) to comparatively liberal norms (Lebanon); dress-code and public-affection norms show similar gradients (e.g., hand-holding discouraged in Egypt and Syria but acceptable in Lebanon and urban Tunisia).
• Annotation quality is reported as strong: 90.1% raw agreement on preference data, Cohen's κ of 0.693 on MCQs, 15.25% of collected English preference candidates rejected by annotators, and Spearman correlation above 0.60 between human and machine judgments.

Methodology in Plain English

The team first gathered descriptions of cultural behavior from travel blogs, official embassy and government travel advisories, and CultureBank entries, all originally in English. They organized this around 16 cultural categories (social etiquette, religion and rituals, business and work, food and dining, family, transport, and others), expanded into 200 aspects, and paired each aspect with each of the 22 countries. Using Gemini-2.5-Flash with the source material as context, they generated contrastive pairs — one behavior that is preferred in that country and one that is discouraged. Automated filters then checked language, removed near-duplicates with MinHash, and removed semantically overlapping items using GemmaEmbedding cosine similarity (threshold 0.90 for preferences, 0.80 for MCQs, after testing thresholds from 0.70 to 0.95).

A team of 40 annotators, with 2–3 experts per country and a "native-in-country" policy (95% living in their native country, the remaining 5% native speakers with over 15 years in the region), reviewed the test triples for cultural authenticity and checked the Arabic translations. Quality was monitored with randomly inserted control items requiring at least 80% accuracy, and 10% of annotated data got secondary expert review. Some MCQs were reduced to three options when annotators judged multiple answers defensible.

For evaluation, MCQs were generated only from held-out country–aspect pairs, with three distractors: one opposite-polarity behavior from the same country and two behaviors from the same aspect but different countries, filtered so the choices are not too similar. Models were then trained with SFT, DPO, or KTO on ArabPref alone, on CARE's Arabic portion or Palm's MSA portion alone, or on combinations, and judged on a 1–5 scale for cultural appropriateness, general quality, and literacy by GPT-4o, plus MCQ accuracy.

Why This Matters

Research impact: Prior Arabic cultural resources (Cidar, Palm, ArabCulture) target instruction following or evaluation, and CARE offers limited preference coverage. ArabPref contributes explicit, country-conditioned behavioral preferences together with a benchmark, and its negative results — SFT degrading MCQ accuracy, Qwen's literacy collapse under some recipes, no single method dominating — are directly useful to anyone doing preference optimization for cultural alignment.

Real-world applications:

• Customer service and government chatbots that must avoid advice that is offensive or illegal in a specific Arab country.
• Travel, hospitality, and relocation assistants that explain local etiquette, dress, and behavior norms.
• Content moderation and localization tools that flag culturally insensitive phrasing for a target market.
• Educational or onboarding tools for staff interacting with clients across Arab regions.

Industry relevance: The paper gives product teams a concrete recipe and dataset for post-training regional variants of a model, and shows that merging general preference data with culture-specific data is typically the safer default. It also quantifies how far open-weight models remain from proprietary systems on cultural benchmarks — useful for build-versus-buy decisions.

Future Directions

• Extend coverage to regional dialects, code-switching, underrepresented subcultures, and within-country variation, since the authors acknowledge the dataset cannot capture full national diversity.
• Track how cultural preferences shift over time, and study multi-turn interactions shaped by speaker relationships, social setting, and conversation history.
• Develop evaluation methods beyond multiple choice that can assess cultural alignment in open-ended, dynamic, real-world responses — the authors note MCQs may not capture this complexity.
• Broaden annotator coverage and collect longitudinal data to separate shared pan-Arab norms from context-specific preferences without homogenizing national cultures; the paper also points to sampling-efficiency experiments (DITTO) in its appendix as a further line.

Target Audience

Researchers and engineers working on Arabic NLP, multilingual model alignment, or preference optimization; dataset curators interested in human-in-the-loop cultural annotation and quality control; and product teams deploying conversational systems in Arab markets who need to reason about country-level behavioral norms. Readers should already understand post-training objectives (SFT, DPO, KTO) and LLM-as-a-judge evaluation to get the most from the experimental sections.

Authors’ abstract

Arabic LLMs have improved rapidly, but aligning them with country-specific cultural behavioral preferences remains underexplored. Previous work has focused primarily on instruction-tuning for factual knowledge, overlooking nuanced cultural behaviors. We propose ArabPref, which to our knowledge is the first large-scale pan-Arab preference data set for the alignment of cultural behavior conditioned on the country. Our contribution includes two resources in English and Modern Standard Arabic: a culturally grounded preference dataset for training and evaluation, and a multiple-choice benchmark designed to assess culturally aligned behavior across nations. All test data are validated by native speakers to ensure authenticity. To optimize cultural alignment, we experiment with post-training methods across multilingual and Arabic-centric language models, evaluating performance on both generation tasks and our cultural multiple-choice benchmark. By releasing an evaluation benchmark, we provide a foundation for systematically studying and improving culturally aligned behavior in language models.