Open-Source Robot Framework Adds Local Model Support
The open-source robot programming framework Dimensional has added support for running AI models locally. A developer demonstrated the framework controlling agents in physical space while running on an NVIDIA Jetson Thor with Nemotron as its brain. This feature provides developers with more control and potentially lower latency for real-time robotics applications.
- The NVIDIA Jetson Thor is a system-on-chip designed for high-performance edge AI, featuring a Blackwell architecture GPU that delivers up to 2070 TFLOPS of performance for AI tasks. It is equipped with 128GB of LPDDR5X memory, providing a 7.5x performance increase over the previous generation Jetson AGX Orin. - Nemotron is a family of open models from NVIDIA designed to create specialized AI agents for tasks like reasoning, visual understanding, and speech. These models are available in different sizes, such as Nano for edge devices and Ultra for multi-GPU data centers, and are released with open weights and training data. - This move toward on-device AI is a broader industry trend; Google DeepMind recently announced Gemini Robotics On-Device, its own vision-language-action (VLA) model optimized to run locally on robots without needing a constant internet connection. - Running models locally is critical for robotics as it ensures the ultra-low latency needed for real-time responses, such as a self-driving car detecting a pedestrian. It also enhances data privacy and allows robots to function in environments with intermittent or no connectivity, like factories or disaster zones. - This technology is a key component of "embodied AI," which is shifting industrial robotics from performing precise, pre-programmed tasks to executing adaptive actions in variable, real-world environments. This shift is opening up automation to less-structured markets like logistics, life sciences, and high-value manufacturing. - The Jetson Thor's Blackwell GPU architecture supports Multi-Instance GPU (MIG), allowing it to be partitioned to run multiple AI models simultaneously. This is crucial for complex robotics applications that must concurrently handle perception, navigation, and manipulation.
