Agentic Artifact Creation: Systems, Evaluation, Principles, and Opportunities

AI summary

Overview

Research area: Agentic AI, generative models, and artifact-producing systems across multimedia and software domains (arXiv:2608.28122v1 [cs.MM], 28 Aug 2026).

Technical level: Advanced. The paper is a survey that introduces a formal, function-level vocabulary (state updates, policies, verification functions) and assumes familiarity with agent architectures such as ReAct, Reflexion, MetaGPT, and ChatDev.

One-sentence scope: A survey of 259 works (230 systems and 29 benchmarks available through August 20, 2026) that defines "agentic artifact creation" as stateful construction in which an AI system materially builds or revises a deliverable and intermediate observations redirect later work.

What This Paper Is About

Generative models can turn prompts into images, text, code, and other content, but practical value increasingly depends on whether those pieces become complete, dependable deliverables whose interdependent requirements all hold at once. The paper argues that failures in such deliverables are hard to trace and repair when requirements interact, so it defines and organizes a distinct research area, agentic artifact creation, around the delivered artifact as the unit of analysis. The goal is to compare how six artifact families, different application settings, and different evaluation practices handle construction state, feedback, and repair.

Key Contributions

1. A functional definition and model: The survey characterizes agentic artifact creation as stateful construction and provides a functional model organized around three roles — an Operational Representation, a Construction Policy, and Runtime Verification — for analyzing how intermediate evidence guides revision.

2. A coded corpus and family comparison: It assembles and releases a coded corpus of more than 200 works and compares six artifact families (textual, 2D visual, audio, video, spatial, and behavioral), highlighting recurring differences in decision interdependence, failure observability, and repairability.

3. Independent treatment of applications and evaluation: It analyzes application context and evaluation separately from artifact family, separating evidence about the delivered artifact from evidence about the trajectory and system.

4. Principles and a research agenda: It synthesizes four principles for inspectable construction control and uses the problems they expose to structure a research agenda with six challenge–opportunity directions.

Main Findings

• Six artifact families share a state–edit–observe grammar but differ in when failures become visible. Textual and 2D visual artifacts are dominated by static inspection (reading and rendering), audio and video by temporal playback, and spatial and behavioral artifacts by runtime consequences (traversal, execution, interaction). The paper notes these are tendencies rather than exclusive modes.

• Construction difficulty reflects coupling and observability, not just modality. Across families, challenges reflect how tightly decisions are coupled and whether failures become visible while they remain repairable. Changing content can alter layout, behavior, or downstream constraints, so a check that fires late or becomes stale after revision is of limited use.

• Decomposition is a trade-off. Breaking work into stages can reduce local complexity and expose specialist interfaces, but it increases coordination costs and dependency-bearing handoffs. The paper states that decomposition can reduce local complexity while increasing coordination and reassembly costs.

• Feedback only helps when it matches repairable scope. The control structure can expose dependencies and support targeted revision, but only when observations identify failures at a scope that the available actions can repair. Detailed diagnosis cannot support local repair when the available edits remain broad — for example, an object-structured form paired only with whole-artifact regeneration provides little local control.

• Learned judges may not add independent evidence. The survey observes that learned judges may add little independent evidence when they share the generator's preferences or blind spots.

• Three construction affordances. Composability (explicit steps as replacement points), Traceability (recorded links among requirements, sources, artifact regions, actions, observations, and decisions), and Revisability (explicit edit units and dependencies) emerge only from alignment among represented structure, decisions, interfaces, records, and runtime evidence; no component suffices alone.

• Corpus composition. The review covers 259 strict records: 230 systems and 29 benchmarks. Family profiles cover 251 family-assigned records, 13 video records remain Cross-profile, Spoken Audio has no strict record, and eight application-workflow systems are omitted from that view. Of the strict records, 120 are published or accepted, spanning ML and AI or general science, multimedia and graphics, data and web, natural language processing, and human-computer interaction communities.

• Direct generation remains appropriate in some cases. Because planning, coordination, tool execution, and verification all add overhead, direct generation can remain suitable for simple artifacts whose failures are cheap to detect and regenerate.

• Retrieval accounting was not fully retained. Database-specific query exports, per-source yields, and pre-reconciliation labels were not retained; the released files document the resulting coding and limitations.

Methodology in Plain English

The authors treat the delivered artifact as the organizing unit and ask four review questions about how construction is organized (RQ1), how requirements and application contexts shape it (RQ2), how it is evaluated (RQ3), and what principles apply across artifacts and applications (RQ4).

They searched arXiv, Google Scholar, Semantic Scholar, ACM DL, and IEEE Xplore for work from January 2023 through August 20, 2026, using agentic-process terms such as "agentic creation," "iterative artifact generation," and "tool-augmented generation," crossed with terms for the six artifact families. Citation tracing and targeted venue audits supplemented retrieval. After title- and identifier-based deduplication, candidates were screened against the stateful-construction definition: an AI system must materially construct or revise the deliverable, carry artifact or process state across decisions, and use at least one intermediate observation to redirect later artifact-related work. Direct generators, observation-independent workflows, and artifact-consuming or evaluation-only systems were excluded. Retained records were coded by corpus role, primary family, subtype, cross-cutting facets, and provenance, with ambiguous cases reconciled iteratively.

Analytically, the authors decompose the construction loop into three roles. The Operational Representation holds state and exposes an edit interface; the Construction Policy selects the next action given the task, state, and feedback; Runtime Verification evaluates observations against acceptance criteria and returns feedback. Each role is then categorized: Intermediate Form (Artifact Instance, Structured Model, Executable Program), Edit Interface (Unit Edit, Relation Edit, Whole-Artifact Edit), Decision Control (Workflow, Autonomous), Agent Topology (Single-Agent, Centralized Multi-Agent, Decentralized Multi-Agent), Observation Source (Artifact State, Runtime Behavior, Evaluation Signal, External Response), and Feedback Function (Criterion Status, Failure Diagnosis, Revision Guidance). Representative systems are tabulated against these categories, and family-level dependency regimes and observation modes are summarized compactly.

Why This Matters

Impact on research. The survey gives the field a shared unit of analysis — the delivered artifact and its acceptance criteria — rather than model families, agent counts, or domains. This makes concepts such as editable state, observation timing, diagnosis granularity, and repair scope comparable across very different media, and it exposes specific failure modes: failure propagation, weak localization, and broad regeneration in direct pipelines. It also flags that evaluation evidence can be confounded when judges share the generator's preferences, and it separates evidence about the artifact from evidence about the construction trajectory.

Real-world applications (drawn from the systems the survey reviews):

• Software engineering: agents such as SWE-agent edit repository state and interpret test and execution feedback, as motivated by SWE-bench, rather than emitting isolated code.
• Scholarly and professional communication: Paper2Poster builds scientific posters that must preserve source content, fit a constrained page, and communicate the paper clearly; related systems produce manuscripts, reports, and slides.
• Design and visualization: chart, diagram, and layout systems such as GenArtist, EvoDiagram, and Data Formulator 2 coordinate data, specifications, and rendered output under layout constraints.
• Media production and interactive content: audio, music, video, 3D asset, and scene systems such as CoComposer, WavCraft, GenMAC, SceneCraft, ShapeCraft, and SPADA, plus game and interface builders such as DreamGarden and DuetUI.

Industry relevance. Teams deploying generative models in professional workflows need deliverables, not drafts, and the paper's control framing maps directly onto pipeline design decisions: which state to persist, what granularity of edit to expose, which observations to collect and when, and what feedback can actually trigger a repairable change. Mixed-initiative systems where human input changes later system-mediated artifact work fall inside the definition, so the framework also applies to review-and-approve workflows in production settings.

Future Directions

• Sustaining coherent, accountable control as things change. The paper identifies as a challenge keeping control coherent and accountable as artifacts, creator intent, and construction systems all evolve over time.

• Diagnosing hard-to-trace failures. When failures are difficult to diagnose, the framework's promise of targeted repair depends on observations revealing the responsible unit and its dependencies at a scope the available edits can fix; how to guarantee that alignment is left open.

• Handling multiple valid outcomes. The survey raises the case where several outcomes may all be valid, which complicates criterion status, diagnosis, and acceptance.

• Six challenge–opportunity directions. Section 8 structures the agenda into six challenge–opportunity directions, and Section 7's four principles for inspectable construction control are proposed as starting points rather than settled answers.

Target Audience

Researchers and graduate students working on LLM agents, generative content pipelines, or multimedia systems who need a shared vocabulary and a map of the literature; practitioners designing artifact-producing pipelines in software, publishing, design, media, or interactive content; and evaluators or benchmark designers interested in why construction evidence should be separated from trajectory and system evidence. The survey is technical, so readers without background in agent architectures will need to follow the cited systems (ReAct, Reflexion, MetaGPT, ChatDev, SWE-agent) alongside it.

Authors’ abstract

Generative models can turn natural-language prompts into images, text, code, and other content, lowering the cost of producing drafts and components. Their practical impact increasingly depends on whether those pieces can become complete, dependable deliverables. This survey examines agentic artifact creation, which we define as stateful construction in which an AI system materially constructs or revises a deliverable and intermediate observations redirect later work. Functionally, the process links an operational representation of the artifact, a construction policy, and runtime verification whose feedback can redirect later actions. We reviewed 259 works available through August 20, 2026: 230 systems meeting this definition and 29 benchmarks of agentic artifact construction. We compare six artifact families, then analyze application settings and evaluation practice as separate dimensions. Across families, construction challenges reflect not only modality but also how tightly decisions are coupled and whether failures become visible while they remain repairable. Decomposition can reduce local complexity while increasing coordination and reassembly costs. Learned judges may add little independent evidence when they share the generator's preferences or blind spots. We formulate principles for keeping commitments and responsibility explicit, turning feedback into targeted repair, and revalidating affected state after change. We also identify opportunities for sustaining coherent, accountable control as artifacts, creator intent, and construction systems evolve. A curated paper list is available at https://github.com/GeminiLight/awesome-agentic-artifact-creation.