Skip to content
AI.info

The Pulse

Ricursive Releases Ripple-65B Preview, an Open-Weight Model It Says Leads U.S. Open Models on Agentic Coding

Ricursive Intelligence’s Ripple-65B Preview model scored 80.52 on TerminalBench 2.1, the company reports, and was trained on 12 trillion tokens. The company also describes the model’s architecture, training costs and development process.

Ricursive Releases Ripple-65B Preview, an Open-Weight Model It Says Leads U.S. Open Models on Agentic Coding

Ricursive Intelligence has released Ripple-65B Preview, a general-purpose variant of the proprietary model that powers its AI chip design platform. The company trained the model from scratch and, it says, the model sets a new U.S. open-weight state of the art in agentic coding. Ripple-65B Preview was pretrained on 12 trillion tokens and supports up to 1M context length.

TerminalBench 2.1 Score of 80.52 for Ripple

Ripple-65B Preview scored 80.52 on TerminalBench 2.1, according to Ricursive. That result surpasses, the company says, significantly larger U.S. open-weight models such as Reflection Beam 501B at 80.1, Meta Muse Spark Open 816B at 76.4, Thinking Machines Inkling 975B at 63.8, and NVIDIA Nemotron 3 Ultra 550B at 56.4. Ripple is a dense model that, in Ricursive's description, provides strong reasoning performance with low-latency token generation. The model can run efficiently on local devices such as NVIDIA DGX Spark, which feature high compute FLOPs and low memory bandwidth relative to datacenter accelerators, the company says.

Ripple-65B Preview's Development and Training Cost Put at Only $32M

Ripple-65B Preview was developed and trained for only $32M, which Ricursive estimates to be one-tenth the cost of the previous open-weight state of the art on agentic coding, to the best of its knowledge. Ricursive defines that $32M as its full R&D cost starting from scratch. It includes pretraining ladder development, data mixing, evals, debugging, synthetic data generation, posttraining recipe development, and the final hero pretraining and posttraining run.

As context for the expense of model training, Ricursive cites OpenAI CEO Sam Altman saying in 2023 that GPT-4 training costs exceeded $100 million. It also cites Dario Amodei saying the following year that frontier AI models can cost up to $1 billion to train, with costs projected to rise to $10-100B by 2026 to 2027 for pretraining alone. Ricursive opted not to buy any posttraining data and instead devised its own synthetic data pipelines, quality filtering, and coverage metrics.

Only 1.4M Rollouts Behind an 80.5 TerminalBench 2.1 Score

Ricursive carefully iterated on supervised fine-tuning and reinforcement-learning recipes with a focus on compute efficiency, rather than investing in a brute-force reinforcement-learning climb. Its mixed reinforcement-learning recipe uses truncated importance sampling, score centering, a linear length penalty, and high-quality long-horizon synthetic environments. The company tuned tasks to the model's capabilities and continuously added harder tasks as the model improved. A TerminalBench 2.1 score of 80.5 was reached with only 1.4M rollouts, roughly 70x fewer rollouts than Reflection Beam, the prior SOTA, Ricursive reports.

Ricursive Designed Ripple-65B Preview as a Dense Transformer for NVIDIA Hardware

Ricursive designed Ripple-65B Preview's architecture to be trained and served efficiently on NVIDIA hardware. The company chose a modernized dense Transformer rather than a Mixture-of-Experts (MoE) architecture, with the objective of reducing pretraining and posttraining infrastructure complexity while enabling strong reasoning and agentic performance. Training maintained a solid MFU (40%) throughout pretraining and posttraining with simple sharding, no custom kernels, and no quantization, the company reports.

The model has a thin-and-deep aspect ratio, with a narrow d_model of 5120 and 80 total layers. The company overprovisioned the number of attention heads while increasing the SWA-to-global layer ratio and reducing SWA window size to compensate for the increased cost. Ripple uses FFN activation clipping, gated attention, global NoPE, and value residuals.

Model Laddering Credited With Up to 16x Compute Efficiency

Ricursive began its pretraining experiments with Olmo 3's model ladder and tested interventions at small scale, focusing on coding capabilities. Its v1.0 ladder implemented StepFun batch size and learning rate scaling laws, while v1.1 implemented a new ladder that smoothly scales architectural dimensions and model depth. Ladder v1.2 changed the baseline from post-norm to pre+post norm; the company says that change was neutral at small scale but showed improvements starting at 4B.

Ladder v1.3 implemented sigmoid gated attention, while v1.4 implemented 2-stream mHC, which showed good results at smaller scale but did not improve at 4B scale and was therefore rejected. The v1.5 ladder switched to global NoPE and added a final LR cooldown to zero; v1.6 added value residuals. In the final version, Ricursive switched to a custom pretraining dataset with substantially upsampled code.

The compounded changes improved compute efficiency on code, as measured by dolma_stack CE, by up to 16x at 4B model scale, according to Ricursive. The final Ripple model ladder can be used to achieve similar code performance to the Olmo 3 ladder with 16x less compute, the company says. Ripple-65B Preview is available on Hugging Face, Together AI, GMI Cloud, OpenRouter, and Fireworks.

Sources

  1. Original source Ricursive Intelligence ricursive.com

Related stories

Explore

More articles