China Merchants Bank wins CNCF contest

China Merchants Bank’s win in a CNCF case study contest is a useful example of what large enterprises are now trying to do with AI infrastructure: stop treating training and inference as separate worlds. The bank said it built a single Kubernetes-based control plane that lets model training, fine-tuning and online inference share the same accelerator pool, rather than reserving hardware for each task. (prnewswire.com) CNCF announced the award on September 7 during KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 in Shanghai. The case matters because the numbers are concrete. China Merchants Bank said average accelerator compute utilization rose from about 35% to more than 60%, while inference cost per 1 million tokens fell by more than 60%. Those gains, if they hold in broader use, show why banks and other large companies are pushing to run more AI workloads on shared cloud-native platforms instead of siloed systems. (cncf.io) ### What did China Merchants Bank actually build? China Merchants Bank said its AI infrastructure team combined Kubernetes with Kueue, KEDA, Prometheus, HAMi and Fluid to create a unified control plane for training, fine-tuning and online inference. The platform lets those workloads share nearly 10,000 heterogeneous accelerator cards, according to the bank’s case study and CNCF’s announcement. (prnewswire.com) The bank said the system now covers 99% of its accelerator compute resources. That matters because training jobs tend to be bursty, while inference services often need steadier availability, and running them on separate pools can leave expensive hardware idle. CNCF described the bank’s approach as an example of “composable, vendor neutral infrastructure” built with cloud-native projects. (prnewswire.com) ### Why is unifying training and inference such a big operational issue? AI workloads place different demands on the same chips. China Merchants Bank said training, inference and multi-tenant fine-tuning each compete differently for accelerator capacity, which made fragmented resource management costly and inefficient before the new platform was deployed. (smb.bgdailynews.com) Nearly 10,000 accelerator cards also changes the problem from a simple scheduling issue to a fleet-management issue. A shared Kubernetes layer gives operators one place to allocate resources, monitor usage and move workloads as demand changes, rather than maintaining separate stacks for research, model serving and internal applications. That is an inference from the architecture described in the case study. (prnewswire.com) ### Which numbers in the case study matter most? The clearest metric is utilization. China Merchants Bank said average accelerator utilization rose by more than 25 percentage points, from roughly 35% to above 60%, after the unified platform was put into production. (prnewswire.com) The second metric is cost. The bank said inference cost per 1 million tokens dropped by more than 60%, a figure that speaks directly to the economics of serving large language model applications at scale. CNCF’s announcement did not provide a dollar figure or a time period for that reduction, but it presented the number as a headline result from the production deployment. ### Why is CNCF highlighting this case now? (smb.bgdailynews.com) CNCF published the award announcement during the September 7-9 conference in Shanghai, which for the first time is co-locating KubeCon + CloudNativeCon China, OpenInfra Summit and PyTorch Conference China 2026. The event is framed by organizers as a meeting point for cloud native, open infrastructure and machine learning communities working on production AI systems. (prnewswire.com) The foundation also announced new silver members on September 7 and said enterprises are scaling AI “from training to inference.” China Merchants Bank’s case study fits that theme because it focuses less on model performance than on infrastructure efficiency, scheduling and production operations. ### What happens next? (prnewswire.com) Shanghai’s conference runs through September 9, and CNCF has already published China Merchants Bank’s case study on its website for end users and platform teams to review. The named participants around the event include CNCF, the PyTorch Foundation and companies such as Alibaba Cloud, Ant Group, Cambricon and Huawei, which are also taking part in the conference program. (pytorch.org 1) (pytorch.org 2) (cncf.io)