The Pulse
MemTensor Releases Two Models With Persistent Memory
MemTensor released MemT-1.0-Nano and MemT-1.0-Flash, models designed to keep a fixed-size memory state across conversations. The company reports lower storage needs for long histories, while its paper says text-based memory systems still pe
AI.info Team · Written with AI · fact-checked ·
MemTensor released two models on October 20 designed to carry a fixed-size memory state from one interaction to the next, rather than repeatedly feeding a growing conversation transcript back into the model. The release, called MemT-1.0, includes Nano and Flash; the team’s technical report appeared on arXiv the day before.
“By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution,” the authors write. The report describes an approach that puts memory updates inside the model’s computation, then measures the trade-offs against full-context and other memory systems.
Memory that updates between turns
MemT-1.0 adds a Native Memory Block alongside a model’s regular attention. Its local memory stores information that changes during use, while a trainable “hyper memory” supplies the learned machinery for writing, forgetting and retrieving it. A Gated DeltaNet-2 mechanism updates the memory state at token level, and a Mixture-of-Storage design is intended to expand its capacity.
The released models have different sizes and backbones. Nano has 36 billion total parameters, with 3 billion activated, and is built on Qwen3.6-35B-A3B. Flash has 288 billion total parameters, with 17 billion activated, and uses DeepSeek-V4-Flash. MemTensor says it trained the models through continued pre-training and post-training on 34 billion tokens.
The memory state is bounded for a given configuration: it does not grow as the conversation history grows. In the company’s example, a user can save earlier turns to memory, clear the plain-text conversation and resume later from a saved state. The models are available through Hugging Face; the project’s GitHub repository includes code and serving instructions.
The 128K-token efficiency claim
MemTensor reports that Nano used about 66 times less persistent storage than a full-context KV cache at a 128,000-token history length, and ran end to end about 1.86 times faster in that test. The paper’s measurements show the speed advantage changing with history length: roughly 1.20 times at 64K tokens and 3.99 times at 256K. At shorter histories, Nano was slower; the authors say it roughly matched full context at 32K.
The comparison is based on the team’s two-GPU, single-request tests, with batch size one. The reported end-to-end timing includes preparing the history and generating a 128-token response. The researchers say the longer-history gain mainly comes from processing the history incrementally, then querying a fixed-size state instead of attending over all previous tokens.
Benchmark gains come with a memory trade-off
Across 11 memory benchmarks, the report gives Flash an average score of 50.6 and Nano 47.0, ranking them first and second among the evaluated parametric-memory methods. The authors say those averages exceed the strongest parametric baseline by 12.5 and 8.9 points, respectively. But the report also finds that stronger textual-memory systems remain ahead overall, especially on long histories where a task depends on retrieving detailed evidence.
For coding agents, the team tested Nano on SWE-bench Verified, a set of 500 software tasks, and SWE-bench Multilingual, which covers 300 tasks across nine programming languages. In a harness-triggered setup, Nano’s average success rate on Verified was 69.73%, compared with 66.07% for the no-experience baseline; on Multilingual, it reached 63.56%, against 60.00%. The paper says the system uses procedural experience drawn from earlier tasks, but that experience is externally constructed and its quality and retrieval affect results.
Weights and code carry different terms
The release’s licensing is not uniform. Hugging Face lists the Nano weights under Apache 2.0 and Flash under MIT, while the repository says its original software, including the SGLang integration, uses the PolyForm Noncommercial License. The repository states that commercial use of that software is not permitted under those terms; backbone and third-party components retain their own licenses.
MemTensor also acknowledges a limit inherent in its design: a fixed-size state has finite capacity, so adding new information can erode older details. That trade-off is most consequential for precise recall, the same area where the report says text-based systems retain an advantage.