Nvidia Launches New AI 'Inference' Chip

The new chip for AI "inference" is a direct move to protect Nvidia's commanding 80-92% market share in AI data center chips from growing competition. While Nvidia's GPUs have been the industry standard for training complex AI models, companies like AMD, Intel, and even major customers such as Google and Amazon are developing their own, more specialized and cost-effective chips for inference. AI "training" is the computationally intensive process of teaching a model on vast datasets, a market Nvidia dominates. "Inference," however, is the much higher volume activity of the model actually answering queries and making predictions in real-time. The inference market is projected to grow substantially, with some estimates suggesting it could be 100 times larger than the training market. This new chip will reportedly incorporate technology from Groq, a startup Nvidia recently invested in and licensed technology from in a deal worth a reported $20 billion. This move is designed to offer a faster, more efficient solution for inference workloads, directly addressing a key area where competitors have been focusing their efforts. Key competitors include AMD with its Instinct MI300X chip and Google with its Tensor Processing Units (TPUs), which are already being used to power their own AI services. Amazon's Trainium chips are another significant alternative, with major AI players like OpenAI signing deals to use them. This new offering from Nvidia is a strategic necessity to counter these specialized solutions. The official unveiling of the new processor is expected at Nvidia's GTC developer conference in March. OpenAI, the creator of ChatGPT, has reportedly already agreed to be a major customer for this new chip, signaling strong initial demand and a significant win for Nvidia in this evolving market.