The Pulse
Cognizant’s 75M-Parameter Model Hits 76% on ARC-AGI 1
Cognizant AI Lab reports that a 75-million-parameter transformer reached approximately 76% pass@2 on ARC-AGI 1’s public evaluation set after a little over four hours of training and evaluation on one NVIDIA H100 GPU.
AI.info Team ·
A small model challenges the scale-first recipe
Cognizant AI Lab says a 75-million-parameter transformer reaches approximately 76% pass@2 on the public evaluation set for ARC-AGI 1, a benchmark built around unfamiliar grid-transformation puzzles. The result places a narrowly trained system in the performance range associated with much larger reasoning models.
The distinction matters. Cognizant’s figure comes from ARC-AGI’s freely available public evaluation set, and the company says the system has not been evaluated on the benchmark’s private leaderboard. Its comparison with larger models also uses different cost measures: Cognizant includes training the model from scratch as well as answer generation, while reported figures for other systems generally cover inference only.
Even with those limits, the experiment challenges the assumption that better abstract reasoning must come from adding parameters and compute. Cognizant’s model is built specifically for ARC-AGI rather than trained as a general-purpose language system, and the company reports that training and evaluation take a little over four hours on one NVIDIA H100 GPU.
ARC-AGI rewards adaptation, not memorization
ARC-AGI tasks present a few input-output grid pairs and require a system to infer the transformation rule before applying it to a new grid. Each task typically provides only two to six demonstrations, and the rule changes from one task to the next. The grids can be no larger than 30 by 30 cells and use ten possible colors, but their small size does not make the reasoning problem simple.
A system may need to complete a pattern, move objects as if affected by gravity, remove all but the largest structure, or preserve an object while changing its position. The benchmark offers no large collection of nearly identical examples from which a model can gradually learn the answer. It tests whether the system can identify the relevant relationships from sparse evidence and apply them to an unfamiliar case.
That setting exposes a weakness in the usual training model for AI systems. Broadly pretrained models gain strength from enormous datasets and general-purpose representations, while ARC-AGI asks for rapid learning from a handful of examples. Cognizant’s approach responds by changing the model’s representation and training objective rather than making the model larger.
Four design choices carry most of the result
The model is an autoregressive transformer, but its input vocabulary contains only 14 tokens: the ten grid colors and four structural tokens used to mark sequence boundaries, separate grids and indicate new rows. Each cell also receives coordinates for its row, column and position among the demonstration and test grids. That preserves the two-dimensional relationships that could disappear when a grid is flattened into a one-dimensional sequence.
Cognizant trains the system on ARC-related data, including the demonstration pairs supplied with public evaluation tasks, but not the held-out test outputs used to calculate the final score. The training set is expanded through recoloring, rotation, reflection and scaling, with each transformation preserving the relationship between an input and its output.
The most direct performance gain comes from multi-token prediction. A conventional version that predicts only the next token reaches approximately 70% on ARC-AGI 1, according to Cognizant. A version trained to predict four tokens ahead reaches approximately 76%, while also supporting faster generation through self-speculative decoding.
At inference time, the system transforms demonstration and test grids in several ways, generates candidate outputs and reverses those transformations. Lightweight symbolic checks remove candidates that violate relationships shared across the demonstrations, such as a consistent input-to-output size ratio. The remaining candidates are ranked through augmented likelihood scoring across the transformed views.
Seventeen minutes buys a weaker but useful testbed
The final system reaches approximately 76% pass@2, meaning one of its top two proposed outputs is correct for roughly three out of four tasks in the public evaluation set. Cognizant also trains eight independent models as an ensemble, raising the ARC-AGI 1 result to approximately 80%.
A faster configuration shows the trade-off between performance and turnaround time. Training takes about 12 minutes and evaluation another five, for a total of roughly 17 minutes. That version reaches approximately 44% on ARC-AGI 1, giving researchers a way to test new representations and objectives without waiting hours for every experiment.
The system generates 48 candidates for each test input and produces the correct answer somewhere in that pool for approximately 83% of ARC-AGI 1 tasks. Its selection process places the correct answer among the top two only about 76% of the time. For the eight-model ensemble, those figures rise to approximately 91% and 80%, respectively.
The gap points to candidate selection as a practical source of improvement. Cognizant’s model sometimes produces the right solution but fails to rank it highly enough, so a better scoring method could raise the final result without changing the underlying transformer.
ARC-AGI 2 shows where specialization stops helping
The result is less decisive on the harder ARC-AGI 2 benchmark. One model reaches approximately 17%, while the eight-model ensemble reaches roughly 22%. Those scores show that the system performs strongly on a narrow, structured evaluation but has not solved abstract reasoning as a general problem.
Cognizant identifies global planning, counting and preserving the exact shape of a moving object as areas where the model still struggles. Its own comparison also rules out a broader claim: specialized systems do not replace frontier models across unrelated tasks. The model’s advantage comes from having a small vocabulary, a spatial representation, targeted augmentation and an objective aligned with ARC-AGI’s structure.
The experiment instead presents a narrower engineering lesson. When data has a clear internal form, flattening it into generic text can waste both information and compute. A model designed around grids may need far fewer parameters than a general system, just as systems for transactions, images, logs or sensor readings may benefit from representations built around their own relationships.
Cognizant AI Lab’s result is therefore best read as a public-set estimate and a fast research platform, not a new official ARC-AGI ranking. Its concrete claim is smaller and more defensible: a 75-million-parameter transformer, trained from scratch and run on one H100, can place a correct answer in its top two for approximately 76% of ARC-AGI 1’s public tasks.