The Pulse
Epoch Puts Open-Weight Models 6–7 Months Behind on Cyber
Epoch AI estimates that Z.ai’s GLM-5.3 trails the closed-weight frontier by six to seven months on its Cyber ECI, despite scoring above GPT-5.4. The estimate rests on a limited set of results, including just three for GLM-5.3.
AI.info Team · Written with AI ·
Z.ai’s GLM-5.3 has moved ahead of some earlier closed models on cyber tests, but Epoch AI estimates it remains six to seven months behind the closed-weight frontier. The comparison, published October 16, puts the open model above OpenAI’s GPT-5.4 while still below higher-scoring closed systems released in April. Epoch’s estimate offers a snapshot of how quickly cyber capabilities become available in models with downloadable weights—and how much uncertainty sits behind that comparison.
GLM-5.3 clears one closed-model benchmark
Epoch gives GLM-5.3 a Cyber ECI score of 160, making it the highest-scoring open-weight model in the analysis. Its score exceeds GPT-5.4, released March 5, but falls below Claude Mythos Preview and GPT-5.5, both released in April. Epoch uses those release dates to place GLM-5.3’s cyber capability roughly where the closed frontier stood six to seven months earlier.
“This places the open cyber frontier roughly where the closed frontier stood 6–7 months ago.”
Jaeho Lee and Alexander Barry, authors of Epoch’s analysis
The comparison is between scores on a composite index, not a claim that the models perform identically across every cybersecurity task.
What Epoch’s Cyber ECI measures
Epoch describes its Epoch Capabilities Index as a scale combining results from multiple benchmarks so models can be compared even as individual tests become less discriminating. The Cyber ECI applies that approach to cybersecurity benchmarks, estimating a model’s position from its cyber results. Epoch says its October update covered 76 models and 35 cybersecurity benchmarks, with 324 model-benchmark results in total.
The data came from evaluations by the UK AI Security Institute, the US Center for AI Standards and Innovation (CAISI), the cyber-evaluation company Irregular, model developers and other public sources. Epoch included models with at least two cyber benchmark results. Its publication list identifies Lee and Barry as the authors of the October 16 data insight, while the index documentation explains the broader ECI approach.
Three results carry much of the estimate
GLM-5.3’s score rests on three benchmark results, all from a CAISI assessment in September; Epoch says those are the only cyber results it has for the model. That small evidence base makes the headline gap less secure than a comparison supported by many independent tests. Epoch’s analysis also says CAISI did not identify the US models it tested, so Epoch assigned some results to the models it judged most likely to have been leading at the time.
The researchers flag additional limits: some model scores rely on only a few benchmarks, and Epoch’s October refit changed several high-end scores. The updated Cyber ECI values therefore should not be treated as directly comparable with figures in Epoch’s earlier publications. The estimate is best read as a rough time comparison based on the available public results, not a precise forecast of when open models will match closed ones.
A narrow lead, not a general cyber ranking
GLM-5.3’s result shows that an open-weight model can surpass a closed model released months earlier on Epoch’s composite cyber measure. It does not establish that open models now match the newest closed systems, or that the six-to-seven-month gap applies to every kind of security work. Epoch’s current comparison still places April’s closed models above GLM-5.3, while the sparse test record leaves room for the measured distance to shift as more evaluations become available.