The Pulse
Z.ai Built GLM-5.3-Flash’s Inference Stack on 100,000 AI Chips
Z.ai says it built the production inference system for GLM-5.3-Flash across more than 100,000 Chinese-made AI accelerators. The company reports a threefold throughput improvement and says an AI coding agent helped complete the work in less
AI.info Team ·
100,000 Accelerators, Six Days, 62 Trillion Tokens
Z.ai says it deployed GLM-5.3-Flash on a cluster of more than 100,000 Chinese-made AI accelerators, then used the system to process more than 62 trillion tokens in six days under real-world traffic. The company describes the work as an early example of an AI system helping build the infrastructure needed to run its successors.
The figures appear in a September 17, 2026 post from Z.ai describing how the company took GLM-5.3-Flash from its first successful run on new hardware to a production service. Z.ai says the model reached production readiness in less than two weeks and that its final serving system delivered roughly three times the end-to-end throughput of the initial baseline.
The deployment is notable because Z.ai says no one had previously operated a cluster of Chinese-made accelerators at that scale. The hardware presented several constraints at once: limited memory capacity and bandwidth, incomplete kernel support, an immature software ecosystem, and sparse documentation for the engineering team to rely on.
GLM-5.3-Flash Forced a Different Serving Design
The model added further pressure to the system. Z.ai says GLM-5.3-Flash supports a 1-million-token context window and multimodal requests, both of which increase the demands on memory management and data movement during inference.
Rather than treat the accelerator cluster as a drop-in replacement for a more established GPU platform, the company redesigned the serving stack around the model’s architecture and the hardware’s limitations. The resulting system combines several forms of parallelism, quantization, memory management, and request scheduling.
Z.ai identifies intra-node tensor parallelism for linear attention and the language-model head, ReplaySSM, W8A8 quantization, mixed-precision cache quantization using INT8, FP8, and BF16, and Layer Split as the main components. The company also added an Encode-Prefill-Decode architecture that separates stages of request processing instead of handling them through one undivided serving path.
Threefold Throughput Came From Trading Memory for Compute
The optimization strategy often exchanged one scarce resource for another. Z.ai says its engineers traded additional computation for bandwidth and communication for device memory, choices intended to keep the accelerator fleet productive despite its capacity limits.
According to the company, the combined changes improved end-to-end serving performance by roughly three times. Z.ai also says hardware-utilization efficiency and per-token cost reached levels comparable to mainstream Nvidia GPU deployments, although the post does not provide a detailed cost table or an independently reproduced benchmark.
The comparison is therefore a company-reported operating result, not a standardized public evaluation. Z.ai’s account does establish the direction of the engineering effort: reduce memory pressure, keep data moving across the cluster, and make the model’s unusual architecture fit hardware with a less mature software stack.
An Infra Agent Worked Inside the Optimization Loop
Z.ai says the work was not performed only by infrastructure engineers. Much of the implementation was carried out with an “Infra Agent” powered by GLM-5.3, which generated and modified code while receiving performance and correctness feedback from the serving system.
The company’s description places unusual emphasis on that feedback loop. A codebase can give an agent static information about functions and dependencies, but it cannot by itself explain why a numerical test failed, why time to first token rose by 30 percent, or why output throughput fell after a change.
Z.ai says it therefore built a process that connected end-to-end results to specific parts of the system, including kernels, parallelism choices, communication patterns, memory allocation, and serving orchestration. The agent could then use those signals to propose a new hypothesis and decide what to test next.
Ox-Alpha Revealed the Production Workload
Before GLM-5.3-Flash launched publicly, Z.ai tested it under the anonymous name Ox-Alpha on OpenCode and OpenRouter. The company says the model became the most-used model on both platforms within a week of launch and processed more than 62 trillion tokens over six days.
Those deployments gave Z.ai a workload beyond internal benchmarks. Real requests exposed the interaction between long contexts, multimodal inputs, model architecture, accelerator memory, and serving decisions at a scale that isolated tests would not reproduce.
The production traffic also gave the Infra Agent a stream of concrete failures and regressions to analyze. Z.ai presents that operating environment as part of the engineering method, not merely as a final validation stage after the software had already been completed.
Z.ai Calls the Result an Early Form of Recursive Self-Improvement
Z.ai frames the project as an early step toward recursive self-improvement, a concept in which an AI system contributes to the design or training of a successor. The company does not claim that GLM-5.3-Flash can independently design and train a new model from start to finish.
Instead, the post describes a narrower capability: GLM-5.3 helped build the inference infrastructure used to serve GLM-5.3-Flash, including code changes and optimization work that Z.ai says would previously have taken a team of experienced infrastructure engineers weeks.
That distinction matters. The demonstrated system still depends on human-designed objectives, hardware, evaluation procedures, and deployment controls. What Z.ai reports is an AI-assisted engineering loop that shortened the path from hardware adaptation to production service while producing measurable gains on a large cluster.
The most concrete result is the serving system itself: more than 100,000 Chinese-made accelerators, a reported threefold improvement over the initial throughput baseline, and a production deployment completed in less than two weeks. Z.ai’s post leaves open how much of that pace came from GLM-5.3’s reasoning and code generation, and how much came from the engineers who designed the feedback system around it.