Cohere Releases Edge-Optimized Multilingual Model
Cohere's new "Tiny Aya" model demonstrates the feasibility of running advanced, multilingual AI directly on mobile devices. This edge-optimized approach reduces latency and eliminates the need for constant cloud connectivity. The technology is relevant for building privacy-preserving, on-device educational tools that can support children from diverse linguistic backgrounds without sending sensitive data to the cloud.
- The Tiny Aya model has 3.35 billion parameters and a context length of 8,192 tokens, and with 4-bit quantization, it has a memory footprint of just 2.14 GB. - This model family includes a globally balanced version and three region-specific variants: "Earth" for African and West Asian languages, "Fire" for South Asian languages, and "Water" for the Asia-Pacific and European regions. - Tiny Aya is part of a broader research initiative that involved 3,000 researchers from 119 countries to create a dataset covering 101 languages, with a focus on those that are often under-represented in other models. - Its architecture uses a combination of sliding window attention for local context and full global attention, along with Grouped Query Attention (GQA) for efficiency. - In a mathematical reasoning benchmark for African languages, Tiny Aya achieved 39.2% accuracy, significantly outperforming comparable models like Gemma-4B (17.6%). - The model is released under a CC-BY-NC license for research and non-commercial use and is available on platforms like Hugging Face and Kaggle. - To prevent "catastrophic forgetting" of safety alignment when creating regional versions, Cohere Labs used a technique called SimMerge to merge specialized checkpoints with the global model. - On an iPhone 17 Pro, the model can generate text at a speed of 32 tokens per second, demonstrating its suitability for real-time, on-device applications.
