Infinite-Story: A Training-Free Consistent Text-to-Image Generation

AI summary

Overview

Research area: Computer vision, specifically consistent text-to-image (T2I) generation for multi-prompt visual storytelling.

Technical level: Advanced. The paper assumes familiarity with scale-wise autoregressive image models, self-attention key/value features, classifier-free guidance, and consistency metrics such as DINO, CLIP-I, CLIP-T and DreamSim.

Scope: The paper introduces Infinite-Story, a training-free method built on the Infinity 2B scale-wise autoregressive model that generates a batch of images from multiple prompts while keeping subject identity and overall visual style consistent, and that runs at 1.72 seconds per image.

What This Paper Is About

Generating a sequence of images that tell one coherent story requires the same character and the same visual style to survive across different prompts and scenes. Existing consistent T2I methods are mostly diffusion-based and focus on identity alone, ignoring style drift between images, and typically take more than 10 seconds per image — past the point where users lose focus during interactive sessions, according to Nielsen's usability guidelines as cited in the paper. Infinite-Story instead works entirely at test time on a faster scale-wise autoregressive backbone, aligning both identity attributes and global style across all images in a batch without any fine-tuning or training.

Key Contributions

1. Infinite-Story framework. The authors present what they describe as the first training-free, scale-wise autoregressive framework for consistent text-to-image generation, requiring no architecture modification and no additional training.

2. Identity Prompt Replacement (IPR). A prompt-embedding technique that aligns identity attributes (such as gender, age, and species) across prompts by replacing all identity embeddings with those of a reference instance, mitigating the context bias of the text encoder.

3. Unified Attention Guidance (UAG). A combination of two mechanisms — Adaptive Style Injection (ASI) and Synchronized Guidance Adaptation (SGA) — applied to early-stage self-attention layers to enforce both appearance-level identity consistency and global style consistency while preserving prompt fidelity.

4. Demonstrated efficiency and generality. The method achieves state-of-the-art harmonic score performance while running 1.72 seconds per image (over 6x faster than the existing fastest consistent T2I models), and is shown to improve other scale-wise autoregressive models (Switti and HART) as well.

Main Findings

• Best overall consistency score. In Table 1, Infinite-Story achieves the best harmonic score S_H of 0.8538, ahead of 1Prompt1Story (0.8395), IP-Adapter (0.8323), OneActor (0.8088), ConsiStory (0.7902), Vanilla Infinity (0.7891), StoryDiffusion (0.7634), Vanilla SDXL (0.7408), PhotoMaker (0.7223), and The Chosen One (0.6494).

• Leading identity and style metrics. The method records the highest DINO similarity (0.8089), the highest CLIP-I (0.9267), and the lowest DreamSim distance (0.1834) among all compared methods. Its CLIP-T is 0.8732, which is not the highest in the table (Vanilla SDXL has 0.9074, ConsiStory 0.9019, 1Prompt1Story 0.8942).

• Large speed advantage. Inference takes 1.72 seconds per image, compared with 1.71 for Vanilla Infinity, 10.27 for Vanilla SDXL, 10.40 for IP-Adapter, 13.47 for The Chosen One, 19.52 for PhotoMaker, 22.57 for 1Prompt1Story, 23.68 for StoryDiffusion, 24.94 for OneActor, and 37.76 for ConsiStory. The paper states the method is over 6x faster than the existing fastest consistent T2I models and over 13x faster than 1Prompt1Story. Generating four 1024x1024 images in parallel on a single A6000 GPU takes approximately 6.88 seconds in total.

• Ablation confirms each component. Baseline S_H is 0.7891. Adding IPR raises it to 0.8013, adding ASI to 0.8481, and adding SGA to 0.8538. IPR notably improves CLIP-I (0.8955 to 0.9046) and DreamSim (0.2780 to 0.2569); ASI raises DINO from 0.7119 to 0.8082; SGA restores CLIP-T from 0.8625 to 0.8732. The paper notes a slight trade-off in prompt fidelity compared with the baseline, whose CLIP-T is 0.8836.

• User study preference. In Table 3, participants preferred Infinite-Story for identity consistency (58.4%), style consistency (50.0%), and prompt fidelity (56.5%). The next highest values were 1Prompt1Story at 18.0% (identity), IP-Adapter at 29.6% (style), and 1Prompt1Story at 28.2% (prompt). OneActor received 7.2%, 7.2%, and 10.6% respectively, and IP-Adapter 16.4%, 29.6%, and 4.7%. The main text reports 55 participants; Appendix A.4 states the study involved 50 participants aged between 20 and 50.

• Generalizes to other autoregressive models. In Table 4, applying the technique to Switti improves S_H from 0.7719 to 0.8146 and DINO from 0.6595 to 0.7441; applying it to HART improves S_H from 0.7434 to 0.7894 and DINO from 0.6381 to 0.7048.

• Scaling coefficient trade-off. Table 5 shows that increasing the ASI scaling coefficient generally improves DINO, CLIP-I, and DreamSim, with the best DINO (0.8102) and DreamSim (0.1826) at λ = 0.9, while CLIP-T peaks at λ = 0.6 (0.8745). The default λ = 0.85 (CLIP-T 0.8732, S_H 0.8538) was chosen as the balance point.

• Nature of the IPR effect. Appendix B clarifies that IPR does not copy the exact visual appearance of an object across scenes; it preserves semantic attributes such as gender and age consistently across frames.

Methodology in Plain English

The system takes a batch of text prompts, each consisting of a shared identity prompt plus a different expression prompt, and processes them all in parallel through a frozen pretrained model. Three test-time interventions do the work.

First, Identity Prompt Replacement addresses the fact that wording changes identity: the paper's example is that "a dog springing toward a frisbee" tends to produce a Welsh corgi while "a dog on a porch swing with pillows" tends to produce a calmer, domesticated breed such as a Golden retriever. To counter this, the identity portion of the text embedding is overwritten with the reference sample's identity embedding (by default the first sample in the batch), and the expression embeddings are rescaled by the ratio of identity-embedding magnitudes so the balance between identity and expression is preserved.

Second, Adaptive Style Injection operates inside the early self-attention layers. For every sample, the key features are replaced with the reference sample's key features, so attention is anchored to semantically consistent regions. The value features are blended with the reference's value features using a weight computed from the cosine similarity between them, scaled by a coefficient λ, so appearance-level identity and global style flow from the reference.

Third, Synchronized Guidance Adaptation repeats the same key/value modification on the unconditional branch of classifier-free guidance using the identical interpolation weights computed from the conditional branch. Without this, the balance between the conditional and unconditional signals would break and prompt fidelity would suffer.

Implementation specifics: the baseline is Infinity 2B, which predicts over 12 scale-wise steps with a codebook of dimensionality 2^32, producing quantized feature maps of 64x64 resolution with 32 channels. The early steps used for guidance are {2, 3}, λ is 0.85, and all model parameters stay frozen. For more than four prompts, the identity prompt paired with the first expression prompt is always placed first in each batch and the remaining positions are filled with other prompts, so identity propagates across batches.

Why This Matters

Impact on research. The work reframes consistent T2I generation as a test-time, architecture-agnostic intervention rather than a fine-tuning problem, and it argues that style consistency has been neglected relative to identity consistency in prior work. It also moves the discussion toward autoregressive, next-scale-prediction backbones as an alternative to diffusion for consistency tasks, and its demonstrated transfer to Switti and HART suggests the techniques are not tied to a single model.

Real-world applications.

• Sequential visual narratives, comic strip generation, and character-driven content creation, where one character must appear coherently across many panels.
• Storytelling and children's book illustration, where a uniform rendering style across pages matters as much as a consistent character.
• Advertising and branding, where repeated campaign imagery needs a stable subject and a stable look.
• Design prototyping and content generation pipelines, where 1.72 seconds per image enables interactive iteration rather than batch-and-wait workflows.

Industry relevance. The method needs no fine-tuning or extra training data, runs on a single A6000 GPU, and stays under the roughly 10-second interactive-latency threshold that most diffusion-based consistent T2I models exceed — with the compared methods ranging from 10.27 to 37.76 seconds per image. That combination of frozen pretrained weights plus low latency is what makes deployment in interactive creative tools plausible.

Future Directions

• Temporal consistency for video. The authors explicitly name extending the method to support temporal consistency in video generation as future work.
• Adaptive reference selection. The paper calls for more adaptive reference (anchor) selection strategies and identifies, in Appendix E, that reliance on a single anchor per batch is a limitation: a low-quality or stylistically off-target anchor can propagate degradation to the entire batch. Because the method does not alter the generation capability of the underlying Infinity model, output quality is tied to the quality of the initial output.
• Balancing the consistency/fidelity trade-off. Table 5 shows CLIP-T degrading as λ increases; finding ways to raise consistency without the observed prompt-fidelity trade-off is an open problem.
• Closing the prompt-fidelity gap. The full configuration does not lead the comparison on CLIP-T (0.8732 versus 0.9074 for Vanilla SDXL and 0.9019 for ConsiStory), leaving room to improve how faithfully expression prompts are rendered while consistency is preserved.

Target Audience

Researchers and engineers working on text-to-image generation, image personalization, and multi-image consistency; practitioners building storytelling, comic, or character-driven content tools who need fast interactive generation; and readers interested in scale-wise autoregressive alternatives to diffusion models. Readers without background in attention mechanisms, classifier-free guidance, and image-consistency metrics will find the method sections challenging, though the problem statement and results tables are accessible.

Authors’ abstract

We present Infinite-Story, a training-free framework for consistent text-to-image (T2I) generation tailored for multi-prompt storytelling scenarios. Built upon a scale-wise autoregressive model, our method addresses two key challenges in consistent T2I generation: identity inconsistency and style inconsistency. To overcome these issues, we introduce three complementary techniques: Identity Prompt Replacement, which mitigates context bias in text encoders to align identity attributes across prompts; and a unified attention guidance mechanism comprising Adaptive Style Injection and Synchronized Guidance Adaptation, which jointly enforce global style and identity appearance consistency while preserving prompt fidelity. Unlike prior diffusion-based approaches that require fine-tuning or suffer from slow inference, Infinite-Story operates entirely at test time, delivering high identity and style consistency across diverse prompts. Extensive experiments demonstrate that our method achieves state-of-the-art generation performance, while offering over 6X faster inference (1.72 seconds per image) than the existing fastest consistent T2I models, highlighting its effectiveness and practicality for real-world visual storytelling.