Nota AI Slashes LLM Memory Usage by 72%
AI optimization company Nota AI announced it has reduced the memory usage of Upstage's Solar LLM by 72% using a new quantization technique. The breakthrough allows powerful models to run more efficiently, a key hurdle for on-device AI deployment.
Nota AI's proprietary technique, called "Nota AI MoE Quantization," was responsible for the memory reduction. It specifically targets Mixture of Experts (MoE) model architectures, reducing the Solar 100B model's memory footprint from 191.2GB to just 51.9GB. The development was part of the "Sovereign AI Foundation Model Project" led by South Korea's Ministry of Science and ICT. Unlike conventional methods that uniformly compress an entire model, Nota AI's algorithm selectively maintains high precision for critical components while compressing less sensitive parts. This nuanced approach is key to minimizing the performance loss often associated with quantization, a major hurdle in making powerful AI models practical for real-world deployment. The two companies involved, Nota AI and Upstage, are both prominent AI startups based in South Korea. Founded in 2015, Nota AI specializes in AI optimization for on-device applications and has raised approximately $42.6 million. Upstage's Solar family of models are designed to be compact yet powerful, with their 10.7-billion-parameter model previously outperforming models three times its size. This breakthrough directly addresses a core challenge for on-device AI: the immense memory and computational power required by large language models. By drastically shrinking the model size, such technology paves the way for running sophisticated AI directly on consumer hardware, improving privacy, reducing latency, and cutting reliance on cloud infrastructure. Nota AI has established partnerships with major global hardware companies, including NVIDIA, ARM, Intel, and STMicroelectronics, to expand its AI optimization business. The company is using its recent $19.9 million in Series C funding to recruit talent and enhance these collaborations, aiming for an IPO.
