S3 hits 20 years, exabyte scale
Amazon S3 turned 20 and now stores 'hundreds of exabytes', reinforcing its role as the default backing store for ML pipelines and RAG document lakes reported. The milestone underscores data gravity risks and the need for multi‑region architectures and clear migration plans.
Amazon’s anniversary post said)) S3 now holds “more than 500 trillion objects” and serves “more than 200 million requests per second” across 123 Availability Zones in 39 AWS Regions. (theregister.com) Amazon highlighted a generational shift toward AI-native storage with Amazon S3 Vectors going GA, claiming up to 1 billion vectors per index and sub‑100ms warm query latency while reducing vector storage/query costs by up to 90% versus specialized databases announced)). During the S3 preview and ramp, AWS reported customers created over 250,000 vector indexes and ingested more than 40 billion vectors, and the GA limits expanded to support multi‑billion vector scales per index and multi‑trillion vectors per vector bucket reported)). AWS published concrete RAG deployment patterns that pair S3 Vectors with Amazon Bedrock Knowledge Bases and SageMaker to scale knowledge stores to millions of documents while keeping retrieval cost‑effective showcased)). For multi‑region, latency and compliance, AWS recommends S3 Multi‑Region Access Points to route to the lowest‑latency replica and S3 Cross‑Region Replication plus S3 Batch Replication for migrating existing objects, with S3 Replication Time Control (RTC) promising most objects replicated in seconds and 99.99% within 15 minutes documented)). Production blueprints emerging from AWS and community samples show patterns to avoid separate vector DB ops by colocating vector indexes in S3 Vectors, integrating OpenSearch or SageMaker for hybrid query/ML, and adding observability (example GitHub projects use OpenTelemetry) to measure retrieval latency, QPS, and cold‑start behavior at scale published)).
