Codex · ChatGPT · OpenAI · OpenAI
Rapidly scaling online storage to serve over 1 billion ChatGPT users
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
★ Tier-1 Source
How they adapted their application storage platform, Habitat, in Python to manage unprecedented growth.
Key facts
- Habitat now handles more than 70 million requests every second, supporting products used by over 1 billion people each week, across almost 40 geographic regions
- Often, system engineers build for 10x scale, and hope for it to hold for a few years while preparing for the next 10x
- Elsewhere, an architectural decision was made to run up to 8 Python processes per pod to push higher CPU usage and provide lower latencies
- In their case, they've grown more than 10x year-over-year for the last three years
Summary
Every OpenAI product depends on fast, reliable access to data, whether someone is logging in, checking their Codex settings, or starting a new conversation in ChatGPT. Habitat is the online storage platform they built so OpenAI products can quickly and reliably access needed information. Building and operating infrastructure at this scale is no easy feat, but also not particularly challenging. Often, system engineers build for 10x scale, and hope for it to hold for a few years while preparing for the next 10x. As OpenAI grew, Habitat had to grow with it: first by becoming reliable enough for mission-critical product traffic, then fast enough for global users, and finally, to deftly operate at massive scale.