RISC-V Matures for AI with New SoCs and Boards
The RISC-V architecture is showing increased maturity for AI workloads with the introduction of new hardware. The SpacemiT K3 SoC and Milk-V Jupiter 2 board can reportedly run 30-billion-parameter models in edge setups, leveraging RVV 1.0 vector extensions to achieve up to 60 TOPS.
- The SpacemiT K3 SoC features a heterogeneous architecture with eight high-performance 64-bit X100 cores and eight A100 AI cores, delivering up to 130 KDMIPS of general-purpose compute power. The X100 cores are quad-issue and out-of-order, running at up to 2.4 GHz. - The 60 TOPS of AI performance is achieved through the A100 cores, which support up to 1024-bit RVV 1.0 for parallel computing and can handle various data types, including BF16, FP16, FP8, INT8, and INT4. This level of performance enables an inference speed of over 10 tokens per second on 30-billion-parameter models. - The K3 is one of the first SoCs to comply with the RVA23 profile, a standard that ensures a baseline set of features for application processors, including hypervisor and advanced interrupt support, which is crucial for running complex operating systems like Ubuntu. Canonical has announced official support for Ubuntu 26.04 LTS on the SpacemiT K3. - The Milk-V Jupiter 2 is available in a compact Pico-ITX form factor (100mm x 86mm) and as a 260-pin SO-DIMM compute module that is hardware-compatible with Jetson Orin Nano carrier boards, facilitating easier migration from existing ARM-based systems. - Beyond the SoC, the Milk-V Jupiter 2 board provides high-speed connectivity suitable for demanding edge applications, including a 10 Gigabit Ethernet port, a Gigabit LAN port, and Wi-Fi 6 capabilities. - The RISC-V Vector Extension (RVV) is a key enabler for AI workloads, providing a more flexible approach to SIMD-style parallelism than fixed-length vector units in other architectures. This allows the same code to run efficiently on hardware with different vector widths without recompilation, which is beneficial for diverse AI computations like activation functions. - The open and extensible nature of the RISC-V ISA allows for the creation of customized instructions and accelerators tailored to specific AI workloads, fostering innovation and avoiding vendor lock-in, which is a significant advantage over proprietary architectures like x86 and ARM. - Other hardware platforms are also adopting the SpacemiT K3, such as the DC-ROMA RISC-V Mainboard III, which is designed for the modular Framework Laptop 13, indicating the SoC's potential to move into more consumer-facing devices.
