Sarvam AI Open-Sources Efficient MoE Models
Indian AI research firm Sarvam AI just open-sourced two new Mixture-of-Experts (MoE) models, at 30B and 105B parameters. The models reportedly outperform much larger rivals on Indian language tasks while being ~6x cheaper to run, aiming to make frontier AI more accessible for startups.
Sarvam AI was founded in August 2023 by Dr. Vivek Raghavan, who has deep experience building India's Digital Public Infrastructure like Aadhaar, and Dr. Pratyush Kumar, a key figure in open-source Indian language AI from his time at AI4Bharat. The Bengaluru-based startup raised a significant $41 million Series A round led by Lightspeed Venture Partners, with participation from Peak XV Partners and Khosla Ventures. The models' efficiency stems from their Mixture-of-Experts (MoE) architecture, a technique first conceptualized in the 1990s. Instead of using all its parameters for every task, an MoE model contains numerous specialized "expert" sub-networks and a "gating mechanism" that routes each specific task only to the most relevant experts, drastically reducing the computational power needed for inference. The larger model, Sarvam-105B, has 105 billion total parameters but only activates 10.3 billion for any given token, while the 30B model uses just 2.4 billion active non-embedding parameters. This sparse activation is the key to its cost efficiency. The 105B model also features a 128,000-token context window for handling complex, multi-step tasks. On performance benchmarks, Sarvam-105B is competitive with or surpasses larger models like DeepSeek R1 and Gemini 2.5 Flash on certain agentic reasoning and task-completion evaluations. Critically, both models are reported to achieve state-of-the-art results across 22 scheduled Indian languages, a key differentiator from globally-focused models. The models, trained from scratch in India using compute from the IndiaAI mission, are available on platforms like Hugging Face and AIKosh under an Apache 2.0 open-source license. This move is intended to accelerate innovation by allowing developers to freely build upon and fine-tune the models for specific applications without starting from zero. This open-source strategy is coupled with a commercial plan. The 30B model already powers Sarvam's conversational AI platform, "Samvaad," while the 105B model is the foundation for its "Indus" AI assistant for complex workflows. The company is also launching a startup program, offering API credits and technical support to encourage ecosystem development and build a future customer base.
