Shaer: Controlled Arabic Poetry Generation with Meter Subform and Semantic Conditioning

AI summary

Overview

Research area: Natural Language Processing — controllable text generation, with a focus on Classical Arabic poetry and prosody-aware language models.

Technical level: Advanced. The work assumes familiarity with supervised fine-tuning, parameter-efficient adaptation methods (QLoRA), completion-only training objectives, and Arabic poetic meter.

Scope: The paper introduces Shaer, a framework that generates Classical Arabic poetry under three simultaneous controls — a natural-language description of meaning, a specified meter subform, and a target number of hemistichs (half-lines) — along with the corpus and evaluation suite built to support it.

What This Paper Is About

Generating classical Arabic poetry is hard because a valid poem must satisfy meaning, language, and strict rhythmic (prosodic) rules at the same time. Prior systems tend to control only broad poetic attributes, and do not jointly handle semantic intent, meter subform, and poem length. Shaer aims to satisfy all three constraints at once, and to be measured on how well it actually honors each requested constraint rather than only on whether the output "looks like" poetry.

Key Contributions

1. A jointly conditioned generation framework. Shaer generates Classical Arabic poetry conditioned simultaneously on natural-language semantic descriptions, meter subforms, and target hemistich counts — a combination the authors state existing systems do not model jointly.

2. An enriched poetry corpus. The authors construct a dataset of 116,032 classical Arabic poems derived from Ashaar, annotated with normalized meter-subform labels and automatically generated semantic descriptions that were validated.

3. A fine-tuned model. They adapt the Yehia-7B foundation model using QLoRA-based supervised fine-tuning with a completion-only objective.

4. A multi-pronged evaluation. Assessment combines automatic checks of base-meter conformity, requested-subform adherence, and length control with three LLM judges, a blinded human evaluation, and a memorization analysis over all test generations.

Main Findings

• Base-meter accuracy of 95.17%. Shaer produced the correct base meter in the large majority of cases; the abstract reports this as the highest base-meter accuracy among all evaluated systems.

• Poem-level meter-subform accuracy of 91.75%. The model usually honored the specific subform requested, not just the broader meter family.

• Exact count accuracy of 83.40%. Roughly five of six generations hit the requested hemistich count exactly.

• Large gains over the untuned foundation model. The abstract reports improvements of 68.68, 57.77, and 38.93 percentage points on the three metrics respectively, relative to the same model before tuning.

• Competitive semantic and literary quality. Multi-LLM evaluation and a blinded human assessment of top-ranked outputs indicate the generated poems are competitive in meaning and literary quality — the abstract does not report the specific scores or rater counts behind this claim.

• No detected memorization. Analysis of all 3,481 test generations found no exact copies from the training corpus or from the paired source poems.

• Resources released. Code, models, and datasets are stated to be publicly available.

Methodology in Plain English

The authors treat poetry generation as a conditional task: instead of asking a model to "write a poem," they give it three explicit instructions — what the poem should mean (written as a normal-language description), which meter subform to use, and how many hemistichs to produce.

To make this trainable, they first needed data with those annotations. They started from an existing collection of classical Arabic poems (Ashaar) and added two things: normalized labels identifying each poem's meter subform, and semantic descriptions generated automatically and then validated. The result is a corpus of 116,032 poems.

They then took an existing 7-billion-parameter Arabic language model, Yehia-7B, and fine-tuned it on this corpus. The tuning was done with QLoRA, a technique that trains a small set of added parameters instead of the whole model, making the process more computationally practical. The training used a completion-only objective, meaning the model was trained to produce the poem portion of the input rather than to reproduce the conditioning information as well.

Evaluation was deliberately multi-sided. Automatic metrics checked the three controllable properties (base meter, requested subform, length). Beyond that, three LLM judges assessed quality, human raters evaluated top-ranked outputs in a blinded setup, and a memorization check verified that the model was not reproducing training or source poems verbatim.

Why This Matters

Impact on research. The paper frames a generation problem around fine-grained, simultaneously satisfiable constraints rather than a single stylistic attribute, and shows that constraint adherence can be measured directly (base meter, subform, exact length) with a memorization check alongside quality judgments. The released corpus with normalized subform labels and validated semantic descriptions is likely to be as useful to follow-up work as the model itself, since annotated classical Arabic poetry data of this kind is scarce.

Real-world applications:

• Assistive and educational tools for Arabic poetry. Learners studying arud (Arabic prosody) could generate examples in a specific meter subform and length on demand, with the semantic description acting as an exercise prompt.
• Controlled creative writing assistance. Poets and lyricists could request drafts that conform to a chosen meter and structure while steering meaning, using output as a starting point rather than a finished work.
• Content generation for media and cultural products. Calligraphy projects, greeting cards, and cultural programming that require short metrically valid verse in a given form.
• Digitization and analysis pipelines for heritage corpora. The subform labeling approach and the discriminator-style metrics could be repurposed to classify or verify meter in existing archives of classical Arabic poetry.

Industry relevance. The work sits in the broader trend of parameter-efficient fine-tuning of mid-sized open models for domain-specific, constraint-heavy generation — a pattern relevant to any market where output must satisfy hard structural rules (verse, legal phrasing, regulated formats). For Arabic-language NLP specifically, it adds a publicly released model and dataset in a domain where tooling has been comparatively thin, and it demonstrates that a 7B model with QLoRA tuning can reach very high adherence on formal constraints without full-scale pretraining.

Future Directions

• Broadening the constraint set. The abstract covers meter subform and length; rhyme scheme, enjambment, and classical rhetorical devices are not addressed and would be natural next constraints to control jointly.
• Verifying semantic fidelity more rigorously. Semantic descriptions were generated automatically and validated; how faithfully the final poem expresses the requested meaning is evaluated through LLM judges and a blinded human assessment, but the abstract does not detail the measurement approach. Stronger, more interpretable semantic-fidelity metrics would sharpen this.
• Generalizing beyond the Ashaar-derived corpus. The data is drawn from a single source collection; whether the approach transfers to other periods, dialects, or modern poetic forms is open.
• Scaling and ablating the recipe. The abstract reports the effect of tuning versus the untuned foundation model, but not how corpus size, description quality, or the completion-only objective each contribute. Isolating those factors would clarify what actually drives the gains.

Target Audience

This paper is most valuable to NLP researchers working on controllable text generation and low-resource or morphologically rich languages, and to computational linguists studying Arabic prosody and computational poetry. Practitioners interested in parameter-efficient fine-tuning of open models for rule-constrained domains will find the training recipe and evaluation design instructive. Readers with no background in Arabic meter or model fine-tuning will need to consult the full paper, since the abstract presupposes familiarity with both.

Authors’ abstract

Classical Arabic poetry generation requires simultaneously satisfying semantic, linguistic, and fine-grained prosodic constraints. Existing systems typically control broad poetic attributes but do not jointly model semantic intent, meter subform, and poem length. We present Shaer, a controllable Classical Arabic poetry generation framework jointly conditioned on natural-language descriptions, meter subforms, and target hemistich counts. To support this task, we construct an enriched corpus of 116,032 classical Arabic poems derived from Ashaar, containing normalized meter-subform labels and automatically generated, validated semantic descriptions. We then adapt Yehia-7B using QLoRA-based supervised fine-tuning with a completion-only objective. Our evaluation combines automatic assessment of base-meter conformity, requested-subform adherence, and length control with three LLM judges, blinded human evaluation, and memorization analysis. Shaer achieves 95.17% base-meter accuracy, 91.75% poem-level meter-subform accuracy, and 83.40% exact count accuracy. Relative to its untuned foundation model, these results represent gains of 68.68, 57.77, and 38.93 percentage points, respectively; Shaer also attains the highest base-meter accuracy among all evaluated systems. Multi-LLM evaluation and a blinded human assessment of top-ranked outputs further indicate competitive semantic and literary quality. Finally, analysis of all 3,481 test generations finds no exact copies from the training corpus or paired source poems. Code, models, and datasets are publicly available.