Skip to content
AI.info

The Pulse

China Telecom details Xing 4 training on 8,192 Ascend chips

China Telecom researchers describe pretraining a 285-billion-parameter Xing 4 model on 8,192 Ascend 910C cards. Their report also details an open-weight 29-billion-parameter model and an agent training platform called TeleGym.

China Telecom details Xing 4 training on 8,192 Ascend chips

China Telecom researchers have described pretraining a 285-billion-parameter Xing 4 language model on 8,192 Ascend 910C cards. The report, a preprint submitted to arXiv on November 17 by the company's Xingchen AGI Lab, also details an open-weight smaller model and software for training models on reasoning and agent tasks.

Xing 4 continues the TeleChat model family and uses a mixture-of-experts design, in which only part of a model's parameters are activated for each token. The larger model activates approximately 17 billion parameters per token and supports contexts of up to 1 million tokens, according to the report.

A smaller open model

The lab has also open-sourced Xing4.0-29B-A4B, a mixture-of-experts model with 29 billion total parameters. It activates 4 billion parameters per token and supports a native context of 256,000 tokens.

The smaller model was trained entirely on Ascend neural processing units using the MindSpore framework. Its model card says the context can be extended to 512,000 tokens and lists a training corpus of 1.5 trillion bilingual Chinese and English tokens. The weights are available under the Apache 2.0 license.

Architecture choices

The report examines how model size, attention design and the balance between depth and width affect computation and memory use. It also considers speculative decoding, a method that uses a draft model in generating output.

The 29-billion-parameter model combines manifold-constrained hyper-connections with multi-head latent attention and multi-token prediction. For the larger configuration, the report describes combining multi-head latent attention, DeepSeek Sparse Attention and an approach called IndexShare that reuses attention indexes across layers.

The researchers study a wider, shallower mixture-of-experts layout alongside these attention choices. They also describe DSpark, a speculative decoding approach that pairs a block-wise draft model with lightweight correction heads.

Training infrastructure

For pretraining, the team built on MindSpore HyperParallel, distributing model state in a way that takes the cluster's bandwidth hierarchy into account. The report also describes assigning optimizer work to designated processes rather than repeating the same computation across them.

Long sequences create additional demands on memory and can leave workloads unevenly distributed. The system addresses those issues with cost-based workload assignment, reusable token layouts and selective recomputation. The researchers also describe using CPU affinity and core isolation to reduce host scheduling interference.

Agent post-training

For post-training, the report introduces TeleGym, an execution platform that connects agent interactions and evaluated outcomes to policy optimization. Its shared trajectory representation records the sequence of interactions used in training.

Generation and optimization run asynchronously on separate device pools. The platform uses bounded oversampling to accommodate tasks that finish at different speeds, while partial rollout allows unfinished trajectories to continue after a policy update.

The workflow covers mathematical reasoning, code generation and multi-turn agent tasks, the researchers say. The lab says work on training at the trillion-parameter scale is ongoing.

Sources

  1. Original source · Preprint TeleChat to Xing4: Advancing Frontier Agentic Models on Domestic Chips and Frameworks arXiv:2611.22115

Other pages read

  1. XingChen-AGI on Hugging Face huggingface.co

Related stories

Explore

More articles