Hardwired AI Chips Emerge to Challenge GPUs
AI hardware startup Taalas is developing hardwired AI chips as an alternative to programmable GPUs, claiming its architecture can achieve inference speeds of 17,000 tokens per second. This approach sacrifices flexibility for efficiency, targeting fixed-model workloads in power-constrained edge environments. In a related development, AI chip startup BOSS Semiconductor, which focuses on RISC-V and edge AI, raised $60 million in Series A funding.
- The founding team of Taalas includes CEO Ljubisa Bajic, who previously founded AI chip company Tenstorrent, and other early Tenstorrent engineers with decades of collective experience at AMD and Nvidia. - Taalas's "direct-to-silicon" process aims to embed an entire large AI model onto a single chip, eliminating the need for external memory which is a primary bottleneck in traditional GPU architectures. - The company's first product is a hardwired chip for the Llama 3.1 8B model, which it claims is ten times faster than the Cerebras wafer-scale engine, a leading inference platform. This performance is achieved in part through aggressive quantization, using a mix of 3-bit and 6-bit parameters. - The push for specialized hardware addresses key challenges in edge AI, where devices face strict constraints on computational power, memory, and energy, making the power efficiency of AI accelerators a critical metric. - BOSS Semiconductor, a fabless startup founded in May 2022, is specifically developing high-performance AI System-on-Chips (SoCs) for the automotive industry, targeting autonomous driving and in-vehicle infotainment (IVI) systems. - The company's flagship product is the "Eagle-N," a high-performance AI accelerator designed to run various in-vehicle AI workloads, including large language models (LLMs). - BOSS Semiconductor's Series A funding was supported by a wide range of investors, including Korea Development Bank, KB Investment, and existing backer Atinum Investment; Hyundai Motor Group has also previously invested in the startup. - The emergence of application-specific integrated circuits (ASICs) and other specialized processors reflects a broader market trend where demand for efficient AI inference is creating opportunities for alternatives to general-purpose GPUs.
