AWS teams with Cerebras
AWS partnered with Cerebras to offer a premium fast‑inference tier inside AWS data centers, creating a new cloud option for latency‑sensitive serving and competing directly with GPU inference stacks. That tightens the tradeoff for startups evaluating cloud custom silicon versus NVIDIA GPU-based deployments. (futurumgroup.com)
The implementation uses an explicit "inference disaggregation" pipeline that runs prefill on AWS Trainium servers and routes the KV cache to Cerebras CS-3 wafer‑scale engines for decode over Amazon's Elastic Fabric Adapter (EFA). (aboutamazon.com) Cerebras describes the CS-3 as optimized for decode by keeping model weights on‑chip in SRAM to deliver dramatically higher memory bandwidth than GPUs, while Trainium handles the compute‑bound prefill stage. (cerebras.ai) Cerebras says the joint configuration will provide roughly 5x more high‑speed token capacity in the same hardware footprint versus running everything on conventional accelerators, and claims CS‑3 can generate "thousands of tokens per second" compared with "hundreds" on GPUs. (cerebras.ai) The vendor framing highlights "agentic coding" workloads as a target use case, citing that agentic workflows produce about 15x more tokens per query than conversational chat and therefore prioritize decode throughput over bulk parallel compute. (cerebras.ai) AWS says the Trainium+CS‑3 offering will be accessible through Amazon Bedrock and is expected to launch "in the next couple of months," with plans to serve leading open‑source LLMs and Amazon's Nova models on Cerebras hardware later this year. (aboutamazon.com) Both companies described the arrangement as a multiyear collaboration and declined to disclose financial terms of the deal in public comments. (businesswire.com (money.usnews.com)
