MIRA: A Musical Intent Refinement Agent for Aligning Text-to-Music Generation with User Intent

AI summary

Overview

Research area: Text-to-music generation, specifically intent alignment evaluation and inference-time prompt refinement (cs.SD).

Technical level: Intermediate. The paper combines agentic search, rubric-based evaluation, and audio-language model scoring; the core ideas are explainable without deep background, but the scoring formulas and search details reward careful reading.

Scope: The paper proposes a request-specific rubric formulation of text-to-music intent alignment, instantiates it as the MuRA-Bench benchmark, and introduces MIRA, a verifier-guided agent that refines prompts for black-box music generators under a bounded generation budget.

What This Paper Is About

Text-to-music systems now produce convincing audio, but a single global text–audio relevance score cannot show whether a specific request's requirements were actually met. A high-scoring track may still omit a required instrument, ignore a requested mood progression, or miss stylistic traits implied by a reference to an artist, scene, or use case. The authors reframe intent alignment as satisfying a per-request rubric of independently verifiable items, covering both explicitly stated constraints and inferred musical intent, and then build an agent that uses that rubric to revise prompts for any black-box generator.

Key Contributions

1. A rubric-based formulation of intent alignment. The authors define text-to-music intent alignment as prompt-specific musical rubric satisfaction, where each request x has a gold rubric R*(x) = B*(x) ∪ C*(x): B*(x) for requirements directly supported by the prompt, and C*(x) for musical traits completed from references, use cases, and context. Each item is independently auditable from the generated audio, and different requests get different rubrics rather than a fixed checklist.

2. MuRA-Bench. A public benchmark of 100 controlled music-generation requests derived from anonymized intent seeds collected on the Mureka platform. Each request's rubric was drafted by a language model and then reviewed and revised by three music experts, with named artists, songs, and albums converted into generalized, audibly verifiable musical traits rather than direct imitation targets. It contains 68 instrumental and 32 vocal requests, and 1,566 rubric items (996 base, 570 completion), with 6–31 items per request (mean 15.66, median 14.5) across seven dimensions: style (135 items), instrumentation/vocal (303), mood (218), rhythm (257), harmony/melody (114), structure/energy (199), and production/texture (340).

3. MIRA (Musical Intent Refinement Agent). A test-time agent built on three principles: externalizing latent user intent into grounded rubrics, using tree search to balance exploration and exploitation, and maintaining branch-specific memory. MIRA works outside the generator, requires no parameter updates, and supports structurally different black-box backends.

4. Validation and comparative evaluation. Evaluator agreement against expert judgments, main results across three open-source backends and ten commercial systems, equal-budget comparisons against Best-of-9 sampling, component analysis, and a blinded system-level human study.

Main Findings

• MusicFlamingo with AQA scoring agrees best with experts. Across 1,536 clip–item ratings from 100 clips and 25 requests, MF-AQA achieved the highest agreement on all six reported metrics: item-level SRCC .690, Kendall's τ_b .532, LCC .817; clip-level SRCC .823, τ_b .646; pairwise accuracy 79.59%. The next-best item-level SRCC was MF-Prompting at .552; CLAPScore was lowest at .245.

• MIRA improves alignment on every evaluated backend. On the full 100-request benchmark with B = 9, Overall rose from 67.6 to 76.3 on ACE-Step v1.5 Turbo (+8.7), from 60.5 to 72.0 on SongGeneration2 Large (+11.5), and from 76.5 to 82.1 on YuE2-3B (+5.6). The paper reports these as relative Overall increases of 12.9% on ACE-Step, 19.0% on SongGeneration2, and 7.3% on YuE2-3B.

• Larger budgets help. Going from B = 3 to B = 9 further improved both explicit and inferred intent on all three backends. For example, ACE-Step's Completion score moved from 66.1 (direct) to 71.1 (B = 3) to 73.4 (B = 9).

• Inferred intent gains most. SongGeneration2's production/texture dimension rose from 60.7 (direct) to 73.4 (B = 9), a +12.7 gain, and its instrumentation/vocal dimension rose from 51.1 to 63.5.

• An open-source generator with MIRA reaches commercial-level alignment. YuE2-3B with MIRA scored 82.1 Overall, compared with 81.9 for Suno v5.5, the strongest commercial baseline on that metric. ACE-Step with MIRA improved from 67.6 to 76.3, versus 75.9 for MiniMax music-2.6.

• Version updates in commercial systems trend toward better alignment. Suno's Overall score rose from 73.6 in v4 to 81.9 in v5.5 and 81.8 in v6; MiniMax and Mureka also improved across their evaluated versions.

• MIRA uses the generation budget better than independent sampling. On the 20-request subset with nine generations each, MIRA improved Overall by 3.6 points on ACE-Step and approximately 2.35 points on YuE2-3B over Best-of-9, with higher D-Macro on both.

• Every component contributes. On the 20-request subset, tool grounding raised Completion from 0.719 to 0.748 on ACE-Step and from 0.775 to 0.800 on YuE2-3B. Adaptive search then raised ACE-Step's Completion to 0.781 and its Overall to 0.826. Full MIRA reached the highest Overall and D-Macro on both backends (0.836/0.826 on ACE-Step; 0.872/0.868 on YuE2-3B).

• Blinded human ratings support the automatic results. In a study of 400 clips from all 100 requests, ACE-Step with MIRA received higher mean ratings than direct ACE-Step for overall, explicit, and inferred intent, exceeded MiniMax music-3.0, and remained below Suno v5.5 on all three dimensions.

Methodology in Plain English

The authors start by changing how alignment is measured. Instead of asking a model for one overall "does this audio match this text" score, they break each request into a short list of yes/no questions, each about one audible property. A request for RPG battle music, for instance, yields items such as "flute parts," "African drum rhythms," and "drum patterns progressively intensifying to build battle readiness." Some items come straight from the prompt (base items); others are added by experts to capture what the request implies (completion items). A music-specialized audio-language model, MusicFlamingo, answers each item, and instead of forcing a hard threshold, the authors convert the model's yes/no logits into a soft probability. Scores are averaged within each audio, then equally across audios, so a request with many rubric items does not dominate.

For refinement, MIRA treats prompt revision as a search problem. Given a user request and a black-box generator, it first builds an online rubric, calling external tools for music knowledge, reference-audio understanding, or mapping a use case to musical attributes when needed. Tool outputs must pass an evidence gate—traceable source, supporting evidence, sufficient confidence—before they can shape the rubric.

MIRA then builds a search tree. Each node stores a complete generation prompt, the audio generated from it, the item-level verification results, and a branch-specific memory. At each layer, the agent proposes several candidate prompts from retained parents, generates and verifies each independently, and keeps a bounded frontier: the highest aggregate score, a complementary candidate with stronger performance on its weakest quarter of rubric items, and then remaining positions by aggregate score. Items scoring below η = 0.60 get targeted feedback. Memory records, for each item, whether a revision improved it (change greater than ε = 0.03), left it stable, or regressed it, comparing both against the parent and against sibling candidates at the same layer. Memory propagates only along its own branch, preserving diversity across the search, and resets between requests. At the end, MIRA returns the highest-scoring successfully visited candidate.

The evaluation spans three open-source backends (ACE-Step v1.5 Turbo, SongGeneration2 Large, YuE2-3B) with Kimi 2.6 as the text planner and MusicFlamingo as verifier, all running on a single NVIDIA A800, compared against ten commercial models prompted directly through their official APIs. Gold rubrics and human labels are never shown to the generation or refinement system.

Why This Matters

Impact on research. The paper argues that a single global relevance score hides which requirements failed. Item-level, dimension-tagged rubrics make evaluation diagnostic by intent source (explicit vs. inferred) and by musical dimension, and the expert-agreement study gives evidence that a music-specialized audio-language model can serve as a requirement-level judge. It also shows that intent alignment is a measurable axis on which commercial systems are progressing across versions, and that test-time agentic refinement can close much of the gap between open-source and commercial systems without retraining.

Real-world applications

• Music production tools where a user describes a track in plain language and needs specific instrumentation, tempo, and structure honored.
• Game, film, and advertising work, where a brief references a scene or emotional function rather than naming exact musical traits.
• Platform-side quality assurance: automatically flagging which parts of a user's brief a generated track failed to meet, and why.
• Prompt-assistant features for consumer music-generation apps that rewrite underspecified requests before generation.

Industry relevance. Because MIRA sits outside the generator and needs no parameter updates, it can be layered onto existing commercial or open-source music APIs. The commercial-reference comparison positions intent alignment as a competitive dimension alongside raw audio quality, and the version trends reported for Suno, MiniMax, and Mureka suggest vendors already treat it as one.

Future Directions

• Scaling the benchmark and its rubric coverage. MuRA-Bench has 100 requests and 1,566 items; extending it to more languages, genres, and request types would test whether the rubric formulation generalizes beyond the curated Mureka-derived seeds.
• Improving the verifier. Even the best evaluator, MF-AQA, reached an item-level SRCC of .690 and 79.59% pairwise accuracy, so verifier errors remain a ceiling on both measurement and search. Stronger music-specialized audio judges are a clear next step.
• Budget efficiency and search design. The paper evaluates B = 3 and B = 9. Whether other frontier-selection rules, memory representations, or larger budgets yield better returns—and how cost scales with alignment gains—is left open.
• Broader generator and modality settings. The method is demonstrated on three text-to-music backends; extending the same rubric-and-search loop to lyrics, reference-audio conditioning, long-form structure, or other audio domains is an untested direction.

Target Audience

Researchers and practitioners in music generation and audio-language modeling who care about evaluation beyond a single similarity score; engineers building or integrating text-to-music systems who want request-level diagnosis and inference-time improvement without retraining; and music-informatics or HCI researchers interested in turning real user requests and expert musical judgment into structured, verifiable benchmark criteria.

Authors’ abstract

Text-to-music systems produce increasingly convincing audio, yet evaluation reveals little about whether the result matches user intent. A global text-audio relevance score can overlook the implicit intent in underspecified prompts and mask failures in specific requirements, such as instrumentation, structure, rhythm, or mood progression. To bridge this gap, we formulate text-to-music intent alignment as satisfying a per-request rubric of independently verifiable items covering both a request's explicit requirements and its implied musical intent. Scoring items individually makes evaluation diagnostic by intent source and musical dimension, rather than a single opaque score. We instantiate this as MuRA-Bench, a benchmark of real-world platform requests curated by music experts. We further propose MIRA (Musical Intent Refinement Agent), a test-time agent that first grounds a request's intent into rubrics, then searches over prompt revisions for a black-box generator under a bounded budget, iteratively generating music, verifying it against the rubrics, and using this feedback to guide a trajectory-aware tree search. Experiments across open-source and commercial backends show that MIRA improves intent alignment, enabling an open-source generator to achieve performance comparable to representative commercial systems (e.g. Suno and Mureka). Project page: https://mirareview.github.io/.