Detecting Experiential Intertextuality Across Migration Routes: Beyond Surface Similarity in French Narratives

AI summary

Authors: Sakayo Toadoum Sari, Nelly Robin, Michelle Auzanneau, Lakhdar Sais, Véronique Petit, Marie Veniard, Said Jabbour, Fabien Delorme Affiliations: CRIL (CNRS – Université d'Artois), CEPED (Université Paris Cité), EDA (Université Paris Cité) arXiv: 2607.29188v1 [cs.CL], 31 Jul 2026 — License CC BY-SA 4.0

Overview

• Research area: Natural Language Processing / computational social science, at the intersection of discourse analysis and digital humanities, applied to French-language migration narratives.
• Technical level: Intermediate. The paper combines classical lexical similarity, sentence embeddings, POS/dependency parsing, LLM prompting, and Ridge regression, but presents the task framing and results in largely non-specialist terms.
• Scope (one sentence): The paper introduces and evaluates the task of automatically detecting "experiential intertextuality" — shared lived experiences expressed in different words — across sentences drawn from French migration narratives on two geographically distinct routes, using 16 annotation-free methods validated against 816 expert judgments.

What This Paper Is About

Migrants travelling along completely different geographic corridors (the Trans-Saharan route and the Balkan route) often describe strikingly similar lived experiences: police violence, smuggler exploitation, dangerous crossings, and family separation. The paper formalizes the problem of automatically deciding whether two sentences from different migration narratives describe the same kind of experience, even when they share no vocabulary, syntax, or keywords. The goal is a scoring function that ranks sentence pairs by experiential parallelism without any annotated training data, validated by how well it correlates with continuous expert judgments.

Key Contributions

1. Formalizes a new NLP task. The paper defines experiential intertextuality detection as a scoring task: given a sentence pair from narratives on different routes, produce a value in [0,1] approximating expert-assessed experiential parallelism, distinguishing it from traditional literary intertextuality (Kristeva 1969) and from semantic textual similarity. The phenomenon is decomposed into three levels: thematic, functional, and pragmatic.
2. Builds an annotation-free pipeline over a sensitive corpus. From 108 French narratives (99 Trans-Saharan, 9 Balkan), the authors automatically extract sentences and generate cross-route and intra-route pairs, producing 31 scores per pair across five method families, with no manual annotation used in pair generation.
3. Validates against 816 expert judgments with formal inter-annotator agreement. Two HSS migration experts scored pairs on a continuous 0.0–1.0 scale (283 dual-annotated, 533 single-annotated), yielding Krippendorff's α = 0.273 and a computed noise ceiling of r_max = 0.662.
4. Introduces context-aware narrative features and quantifies the ceiling. Novel journey-position features (position similarity, journey-phase match, theme density) are shown to predict intertextuality, and a supervised Ridge hybrid over all 31 features establishes an upper bound of r = 0.454. Code is publicly released at https://github.com/Toadoum/IntertextMigra.

Main Findings

• Surface, structural, and embedding methods correlate only weakly with experts. The best individual annotation-free result is Qwen2.5-7B zero-shot at Pearson r = 0.375 (the abstract rounds this to 0.38), followed by Qwen2.5 CoT at r = 0.365 and Mistral few-shot at r = 0.327. No annotation-free method exceeds r = 0.38. Lexical methods top out at theme-lexicon r = 0.287, TF-IDF r = 0.254, and char-3gram r = 0.230. Embeddings range from multi-MiniLM r = 0.295 down to e5-large r = 0.219. POS features are weak (r ≤ 0.119), and Mistral zero-shot is the weakest of all at r = 0.134.
• A curated theme lexicon rivals neural embeddings. The 130-term, 8-category theme lexicon score (r = 0.287) outperforms CamemBERT-STS (r = 0.284), LaBSE (r = 0.249), and e5-large (r = 0.219), though Qwen2.5 significantly outperforms it (Williams test: t = 3.03, p = 0.003).
• Prompting effects are model-dependent and asymmetric. Few-shot examples degrade Qwen2.5 from r = 0.375 to r = 0.289 (−23%), whereas they improve Mistral from r = 0.134 to r = 0.327. Qwen's CoT prompting yields the best ranking quality overall (Spearman ρ = 0.331). The zero-shot gap between Qwen and Mistral (0.375 vs. 0.134) is reported as 2.8× larger than any within-model prompting effect, suggesting score anchoring in Qwen's case.
• Journey position significantly predicts intertextuality. Expert scores are highest for departure-phase pairs (departure × departure, μ = 0.280) and lowest for mismatched phases (arrival × departure, μ = 0.129). Position similarity correlates at r = 0.131 (p < 0.001). An off-diagonal case, arrival × late transit (μ = 0.275), also scores high.
• Some experiences are systemic across routes; others are route-specific. The strongest thematic echoes are life-threatening danger (μ = 0.340, n = 30) and labor exploitation (μ = 0.326, n = 69), followed by document fraud (μ = 0.286, n = 57) and the football dream (μ = 0.254, n = 49). The weakest are unaccompanied-minor experiences (μ = 0.117, n = 46) and waiting/stagnation (μ = 0.125, n = 123).
• The supervised hybrid reaches r = 0.454. The Ridge model over all 31 features achieves Pearson r = 0.454 and Spearman ρ = 0.383, significantly beating the best single method (Williams test: t = 4.61, p < 0.0001) and reaching 68.6% of the noise ceiling, versus 56.6% for Qwen2.5 zero-shot.
• LLM features dominate the hybrid; embeddings are redundant. LLM features alone achieve r = 0.416. Removing them costs Δr = −0.059. Removing lexical features costs Δr = −0.024. Removing POS costs only Δr = −0.001. Removing all four embeddings improves the hybrid from r = 0.454 to r = 0.463, indicating the embedding signal is subsumed by LLM scores.
• LLM and lexicon signals are not merely surface or theme overlap. Partialling out Jaccard similarity, Qwen2.5 zero-shot drops only from r = 0.375 to r_partial = 0.351; partialling out the theme lexicon yields r_partial = 0.336. The theme lexicon itself drops from r = 0.287 to r_partial = 0.253 when controlling for Jaccard, meaning roughly 12% of its signal comes from simple word overlap.
• Feature importance is counterintuitive. The top-5 Ridge coefficients are ROUGE-1 (β = −0.111, a suppressor), Jaccard (β = +0.077), theme_lexicon (β = +0.045), Qwen2.5-CoT (β = +0.041), and ctx_position_sim (β = +0.036).
• Where methods fail. The hardest high-expert/low-predicted cases encode intertextuality purely in pragmatic intent (e.g., a father's encouragement versus a self-directed decision to leave). Low-expert/high-predicted cases share words like "police" or "frontière" across different experiences (routine identity check vs. violent refoulement). The largest annotator disagreements (|diff| > 0.6) involve implicit cause–consequence links requiring domain knowledge about smuggling logistics.
• Ceiling context. The best single method explains only 14% of variance (r² = 0.141). The best individual method reaches 56.6% of the theoretical maximum r_max = 0.662, and the hybrid 68.6%; the paper notes that 31.4% of the hybrid's gap to perfection is attributable to irreducible annotator noise.

Methodology in Plain English

The researchers assembled 108 French-language narratives collected through semi-structured interviews between 2015 and 2022 at transit points in Niger, Algeria, Senegal, Morocco, Serbia, North Macedonia, and Bosnia, within the ANR HYCI project. Trans-Saharan narratives average 787 words; Balkan narratives average 1,672 words (corpus average 876 words), reflecting the more complex multi-country journeys of the latter. Total unique sentences: 5,922 (2,330 Trans-Saharan, 3,592 Balkan).

From these raw narratives, sentences were extracted automatically and paired using stratified sampling across three configurations: cross-route (one Balkan, one Trans-Saharan), intra-Balkan, and intra-Trans-Saharan. Each pair was automatically labeled with one or more of 15 expert-defined thematic categories via keyword matching — no manual annotation in pair generation.

Each pair then received 31 automatically computed scores from five method families:

• Lexical baselines (7 scores): Jaccard, ROUGE-1, BM25-approximate, character 3-gram overlap, overlap coefficient, TF-IDF cosine (fitted on the full 108-narrative corpus for better IDF estimates), and a theme-lexicon score over 8 thematic word sets totaling 130 curated terms (categories like autre and rêve_football were excluded for lacking stable keywords).
• POS-structural features (4 scores): POS n-gram Jaccard (n ∈ [2,5]), POS edit similarity using Levenshtein distance, dependency-triple overlap, and verb-frame overlap — all using spaCy's French model fr_core_news_sm. The hypothesis was that parallel experiences might share grammatical structure despite different vocabulary.
• Context-aware narrative features (5 scores, presented as novel): position similarity based on normalized character offset in the source narrative; journey-phase match across four quartile phases (departure, early transit, late transit, arrival); theme density computed over a 500-character window; and window overlap.
• Sentence embeddings (4 scores): cosine similarity over L2-normalized embeddings from sentence-camembert-large, paraphrase-multilingual-MiniLM-L12-v2, LaBSE, and multilingual-e5-large.
• Zero-shot LLMs (6 scores): Qwen2.5-7B-Instruct and Mistral-7B-Instruct-v0.3, both NF4-quantized via bitsandbytes with temperature 0.1 and max_new_tokens 256, under three prompting strategies — zero-shot (French system prompt defining the task and 0–1 scale, JSON output), few-shot (three expert-annotated demonstrations at approximately 0.1, 0.4, and 0.8), and chain-of-thought (identify theme, compare parallelism, then score).

Validation used 816 expert-annotated pairs, 815 of which are cross-route. For the 283 dual-annotated pairs, the consensus score is the arithmetic mean; for the remaining 533, the single annotation is used directly. No per-annotator normalization was applied, as the annotators' means were similar (0.389 vs. 0.405). A supervised Ridge regression over all 31 features, trained with narrative-level grouped 5-fold cross-validation (GroupKFold) to prevent leakage, serves as an upper-bound reference. Experiments ran on a single NVIDIA Quadro RTX 8000 (48 GB VRAM).

Why This Matters

The paper shows that experiential parallels across migration routes are real, measurable, and systematic — but that standard similarity tooling largely misses them. That is a methodological result with consequences for how computational social science handles first-person narrative data, and it supplies a reproducible, annotation-free starting point on a corpus that cannot be publicly released for ethical reasons.

Potential real-world applications:

• Migration research and comparative HSS: Automatically surfacing shared experiential patterns across corridors so researchers can identify which experiences transcend specific routes and which are shaped by local infrastructure or border policy.
• Evidence-based policy input: The paper explicitly frames detecting universal patterns as a way to "support policy-making with evidence-based insights" — for example, the finding that life-threatening danger and labor exploitation show the strongest cross-route echoes while waiting/stagnation is route-specific.
• Humanitarian and NGO monitoring: Locating clusters of parallel testimony (e.g., document fraud, detention, smuggler exploitation) in interview transcripts without needing a labeled training set.
• Model evaluation and deployment guidance: The finding that prompting strategy must be chosen per model — few-shot helps Mistral but hurts Qwen — informs how organizations deploy open-weight LLMs for qualitative coding tasks.

Industry relevance: Relevant to organizations deploying LLMs for qualitative text analysis, to teams building multilingual NLP for low-resource or sensitive domains, and to anyone interested in the practical limits of zero-shot LLM scoring where no gold labels exist. The result that a 130-term domain lexicon can rival million-parameter embeddings is a useful cost/benefit data point for applied NLP pipelines.

Future Directions

1. Fine-tuned models and larger LLMs. The authors deliberately avoided fine-tuning to preserve the annotation-free framing, and evaluated only 7B-class models at 4-bit quantization; they flag that 70B+ or full-precision inference may change conclusions, and that quantization effects on score distributions were not isolated.
2. Event-structure representations. The paper proposes semantic role labeling, frame-semantic parsing, and predicate–argument structures to abstract over vocabulary while preserving "who did what to whom," noting that robust French SRL tools remain limited and French FrameNet coverage is incomplete for migration-specific events like refoulement and border crossing.
3. Discourse-level context features. Current context-aware features use only positional information; coreference and causal chains remain unexplored, as does building "experiential graphs" linking shared experiences across the full corpus.
4. Improving the gold standard. With α = 0.273 and a noise ceiling of 0.662, a substantial portion of the remaining gap is irreducible annotator disagreement, especially on implicit cause–consequence pairs — raising the open question of how to define and collect more reliable experiential intertextuality judgments.

Target Audience

This paper is most valuable to computational social scientists and digital humanities researchers working with first-person narrative corpora; NLP researchers interested in tasks that fall outside standard semantic similarity benchmarks; and applied practitioners evaluating embedding models versus LLMs versus domain lexicons for sensitive, low-resource, multilingual text. It is also directly relevant to migration studies scholars who want a quantitative complement to qualitative observation, and to LLM evaluation researchers interested in how prompting strategy interacts with model identity.

Authors’ abstract

Migrants traversing geographically distinct routes such as the Trans-Saharan and Balkan corridors often recount strikingly parallel lived experiences: police violence, smuggler exploitation, dangerous crossings, and family separation. We introduce the task of experiential intertextuality detection: automatically identifying shared experiential echoes across migration narratives without requiring annotated training data. From 108 French migration narratives spanning both corridors, we automatically generate sentence pairs and score them using annotation-free methods: lexical baselines, sentence embeddings, POS-based structural features, a migration-specific theme lexicon, context-aware narrative features, and zero-shot LLM scoring with Qwen2.5-7B and Mistral-7B under three prompting strategies. We validate all methods against 816 expert-annotated intertextuality judgments (inter-annotator Krippendorff's $α= 0.27$). Our results reveal that all surface, structural, and embedding methods correlate only weakly with expert judgments ($r \leq 0.30$); Qwen2.5-7B zero-shot achieves the best single-method correlation ($r = 0.38$); few-shot examples degrade Qwen but dramatically improve Mistral; narrative position significantly predicts intertextuality, with departure-phase pairs showing the highest experiential echoes; and a supervised hybrid combining all 31 features achieves $r = 0.45$, a 21% improvement over the best individual method.