The Pulse
OpenAI Reveals Habitat Storage System Serving 1 Billion Weekly Users
OpenAI says its Habitat storage platform now serves products used by more than 1 billion people each week. The system handles more than 70 million requests per second and stores over 500 petabytes of data across almost 40 regions.
AI.info Team ·
More than 70 million storage requests per second now pass through a system OpenAI built to support products used by over 1 billion people each week. The company disclosed the figures on September 11, 2026, in an engineering post detailing Habitat, an internal storage platform that has grown from a small Python library into a distributed system serving more than 500 petabytes of data.
Habitat sits beneath ChatGPT, Codex, OpenAI’s API and internal services. It handles the less visible work required before a product can respond: locating data, checking permissions, selecting a storage region, managing encryption, routing requests and deciding when to use a cache instead of a database.
The disclosure offers a rare look at the application infrastructure behind OpenAI’s user growth. The company says Habitat operates across almost 40 geographic regions, while its products reach more than 1 billion weekly users. OpenAI does not provide an independent audit of that user figure in the post; the number is the company’s own measurement.
Habitat Started as a Python Library for GPTs
OpenAI launched Habitat at its DevDay event in 2023 to support custom GPTs. The first version was a small Python client library connected to ChatGPT’s main server and backed by Azure Cosmos DB. Product engineers could use it to store and retrieve data without handling database routing, schema details or access controls themselves.
The design reflected a practical problem inside OpenAI: every product team needed storage, but few teams should need to understand the underlying database system. Habitat presented a narrow interface while taking responsibility for routing requests to Azure Cosmos DB, caches and other storage resources.
OpenAI’s engineers write that each ordinary product action can require many separate data lookups. Logging in, opening a conversation, checking Codex settings or loading a feature can all trigger multiple reads. A slow or failed lookup can therefore affect the entire user experience rather than a single isolated function.
“If those requests are slow, the product feels slow.”
Jon Lee, Member of Technical Staff, OpenAI
As more teams adopted the library, Habitat accumulated features for caching, compression, encryption and other platform functions. By the middle of 2025, OpenAI says the client-side model had reached its limits. The number of services had grown, and making backward-compatible changes across a shared library had become increasingly difficult.
OpenAI Turned Habitat Into a Central Service
OpenAI responded by moving Habitat into a standalone service. That change gave the company one place to deploy updates, observe traffic and apply platform-wide controls. Instead of distributing storage logic across individual products, OpenAI could make a change centrally and expose it to every service using Habitat.
The service also became a central enforcement point for data controls. OpenAI says Habitat applies access-control policies, records audit logs and restricts access to underlying resources such as Azure Cosmos DB. The system also supports data-residency decisions, encryption, tenant isolation, request shaping and rate limits.
That centralization matters because OpenAI’s products do not share one uniform workload. ChatGPT conversations, Codex configuration, API services and internal tools can generate different patterns of reads and writes. Habitat’s job is to provide a common control layer without forcing each product team to solve the same storage problems independently.
The company kept the service in Python during its first major expansion, despite recognizing that the language imposed costs at high throughput. OpenAI says the decision was deliberate: the immediate priority was to stabilize the platform and give product developers a consistent interface, not to minimize every unit of CPU and memory use from the beginning.
Python Carried Habitat Past 20 Million Requests Per Second
Running a high-throughput service in Python created a specific performance problem. Habitat used asynchronous I/O to handle many network requests at once, but asynchronous execution did not provide parallel CPU processing. Routing, compression, encryption, checksums, health checks and other tasks could occupy the event loop while responses waited to be processed.
OpenAI began measuring the delay between when background tasks were scheduled and when they actually ran. At high utilization, the delay could reach hundreds of milliseconds and, in some cases, several seconds. The team responded by limiting the number of concurrent requests handled by each process and scaling out the number of worker processes.
One performance issue came from feature-flag configuration. OpenAI found that every worker on a pod could pause at roughly the same time to parse a large configuration file. Engineers reduced the size of the configuration, lengthened the refresh interval and added timing variation so that workers would not all perform the same expensive task simultaneously.
Connection pooling created another failure mode. OpenAI found that a default last-in, first-out reuse pattern in Python’s networking stack could send new traffic back toward already overloaded processes. The effect could concentrate requests on slow workers and allow the degradation to continue after the original burst had ended.
The company now relies primarily on Istio and Envoy for connection pooling and server-aware load balancing. Envoy also allows OpenAI to multiplex traffic over HTTP/2 connections, reduce connection overhead and place rate limits and circuit breakers in a central layer.
Codex and GPT-5.5 Helped Rewrite the Service in Rust
OpenAI eventually moved Habitat’s service implementation to Rust. The company says two engineers completed the rewrite in the second quarter of 2026 with assistance from Codex and GPT-5.5. The Rust version now handles 95% of production requests, while OpenAI plans to retire the Python implementation in the coming weeks.
OpenAI reports that the Rust service is six times more efficient in CPU use and 15 times more efficient in memory use than the Python version. The company also says the rewrite reduced average and tail latency, although the post does not provide a detailed benchmark table or disclose the workload conditions behind those comparisons.
The migration came after Python had already carried Habitat beyond 20 million requests per second at peak. That figure illustrates the engineering tradeoff OpenAI made: the first version did not need to be the final version if it could give product teams a stable platform while the company addressed more immediate scaling problems.
OpenAI says Habitat’s restricted API also helped the system scale. Clients cannot issue arbitrary SQL queries that might trigger large table scans or complex joins. Instead, Habitat exposes a simpler NoSQL-style interface for routine online storage, while Rockset handles workloads that require more complex analytical or search queries.
Azure Cosmos DB Remains a Major Storage Layer
Habitat is not itself a single database. It coordinates several systems, including Azure Cosmos DB, Nanobase, Valkey caches, blob storage and change-data-capture services. The platform decides how a request should be routed and which storage resource should handle it.
Azure Cosmos DB plays a major role in the architecture. OpenAI’s engineers describe it as the database application beneath the first Habitat library and say the company has continued to migrate shardable, write-heavy workloads to systems such as Cosmos DB. The goal is to keep high-volume writes away from the company’s primary PostgreSQL deployment.
OpenAI separately disclosed in January 2026 that PostgreSQL still supports core ChatGPT and API workloads. That system uses one primary Azure PostgreSQL flexible server instance and nearly 50 read replicas across multiple regions. OpenAI says PostgreSQL handles millions of queries per second for read-heavy workloads, while sharded systems take on workloads that are more difficult for a single primary database.
The two engineering disclosures describe complementary layers rather than a replacement. PostgreSQL continues to serve established application data, while Habitat provides a common storage interface across several systems and newer products. The result is a deliberately mixed architecture rather than a single database intended to handle every type of request.
The Next Constraint Is Data, Not Just Traffic
OpenAI says the current Habitat system processes more than 70 million requests per second and stores more than 500 petabytes. Those figures create operational problems beyond raw throughput. Data placement, residency rules, authorization, encryption, cache behavior and tenant isolation all have to work across regions and products.
The company’s next engineering post will focus on the storage layer itself. OpenAI says it plans to explain how Habitat serves more than 500 petabytes and handles more than 70 million requests per second, including its work with Azure Cosmos DB. The first post leaves several questions unanswered, including the geographic distribution of the data, the proportion held in caches and the storage cost of supporting the user base.
OpenAI’s disclosure also shows how quickly the responsibilities of an internal platform can expand. Habitat began as a convenience library for custom GPTs in 2023. Three years later, it has become a global service that connects consumer products, developer tools and internal systems to multiple storage technologies.
The concrete measure of that growth is not the name of the platform but the workload it now carries: more than 70 million requests every second, over 500 petabytes of data and products used by more than 1 billion people each week. OpenAI says the next part of the series will explain how the database layer is being pushed to handle those numbers.