S3 hits 20, 'hundreds of exabytes'
Amazon S3 marked its 20th anniversary and the service has reached “hundreds of exabytes” of stored data — a reminder of the durability, replication, and API expectations modern backends must match. The milestone underlines design lessons for planetary-scale storage: stateless front-ends, elastic sharding, and background validation at enormous scale.
Amazon says S3 manages more than 500 trillion objects and answers over 200 million requests per second across 123 Availability Zones in 39 AWS Regions. (aws.amazon.com) The service began on March 14, 2006 with roughly one petabyte of capacity spread over about 400 storage nodes in 15 racks across three data centers and a total of 15 Gbps of bandwidth. (aws.amazon.com) Amazon documents S3’s durability target as 99.999999999% (eleven 9s) and describes continuous integrity monitoring and periodic checks of data-at-rest. (aws.amazon.com) AWS moved default client-side checksum verification into S3 on December 1, 2024 so SDKs now calculate and send CRC-based checksums that S3 validates before durable storage. (aws.amazon.com) Public engineering writeups and papers show S3’s control plane is layered (REST front-end, namespace service, storage fleet, and a background storage-management fleet) and uses shuffle-shard assignment to limit blast radius when rolling new hardware or software. (highscalability.com) Recent operational work includes a Rust-based ShardStore storage-node project with lightweight formal-methods verification for consistency and a claim that all request-handling code has been rewritten while preserving the original S3 API compatibility. (jamesbornholt.com) S3’s product roadmap also exposes data-layer features for AI workloads — S3 Vectors now supports up to 20 trillion vectors per bucket and offers 2–3× faster query performance for large vector indexes. (itpro.com)
