The Pulse
Radical Numerics’ Halcyon Tests Text-to-Image Without Language Pretraining
Radical Numerics introduced Halcyon, a family of text-to-image models trained from scratch on 100 million image-text pairs without internet-scale language pretraining. In the team’s controlled tests, Halcyon beat two pretrained-language-mod
AI.info Team · Written with AI · fact-checked ·
On October 15, Radical Numerics introduced Halcyon, a family of text-to-image models trained without the web-scale, text-only pretraining commonly used to build language encoders. The company’s researchers report that Halcyon beat two comparison systems on image-generation scores while using substantially less total pretraining compute. Their claim is specific: in the experiments they ran, language pretraining did not improve text-to-image generation.
One model learns from captions and images
Halcyon uses an early-fusion design: a decoder-only Transformer treats text and image data as one sequence of discrete tokens and learns through next-token prediction. Radical Numerics trained its image tokenizer, text tokenizer and generative model from scratch on GPIC, a dataset of 100 million image-text pairs. The five model sizes range from 0.1 billion to 4 billion parameters; each saw the dataset once at 256-by-256-pixel resolution.
The researchers’ comparison is designed to control for training data. They retrained the LlamaGen and JiT-T2I systems on the same GPIC data, then compared them with Halcyon. The strongest LlamaGen variant uses a separately pretrained FLAN-T5 text encoder, while JiT-T2I uses a pretrained Qwen3 encoder alongside a diffusion image generator.
Lower generation scores, less reported compute
On the researchers’ gFD measure, where lower scores indicate a closer match to the real-image distribution, Halcyon-0.52B scored 48.0 against 63.3 for LlamaGen-XL/8. Radical Numerics says Halcyon used 18.2 times less total pretraining compute in that comparison. Halcyon-1.6B scored 35.3 against 41.0 for JiT-T2I-XXL/8++, with 275 times less total pretraining compute, according to the company’s accounting of the language model, image tokenizer and generator.
gFD does not measure how closely an image follows its prompt, so the team also reports GenEval and DPG-Bench results. Halcyon-0.52B scored 44.91 on GenEval and 76.84 on DPG-Bench, compared with 30.48 and 67.08 for LlamaGen-XL/8. Against JiT-T2I-XXL/8++, Halcyon-1.6B scored 53.48 on GenEval and 77.90 on DPG-Bench; the baseline scored 43.17 and 76.85.
Three tests challenge the value of text-only training
The team varied how much of Halcyon-0.52B’s fixed training budget went to text-only pretraining. Allocating 10%, 20% or 50% of that budget to text before image generation worsened gFD from 48.0 to 51.6, 52.8 and 65.1. A separate run added text pretraining before a full text-to-image epoch, increasing total compute by 50%, but scored 49.2 rather than 48.0. Starting from Qwen3-0.6B-Base, which had been pretrained on 36 trillion text tokens, also failed to beat the comparable model trained from scratch: 47.8 versus 46.6 gFD.
“Across all three, language pretraining does not improve T2I generation in our setting,” write Keshigeyan Chandrasegaran, Yue Zhao, Armin W. Thomas and Michael Poli. The qualification matters: the experiments cover the specified dataset, model configurations and 256-pixel images, not every image-generation task or training setup.
Captioning fits within the same training budget
Halcyon also learns to produce captions from images. Instead of dropping captions from 10% of training examples to teach unconditional image generation, the researchers put those captions after the image tokens. That adds image-to-text supervision without increasing the number of image tokens or the stated pretraining budget, while the team reports comparable text-to-image quality across its five model sizes.
Radical Numerics says it has released ten Halcyon models. The results make a case for testing whether a separate language-pretraining stage is needed for this class of text-to-image model; they do not establish that language pretraining is unnecessary for other uses of language models or for image generators trained under different conditions.