Framework 'Dimensional' Adds Local Model Support
The open-source robotics framework Dimensional has added support for running AI models locally on a robot's hardware. A demonstration showed the framework operating on an NVIDIA Jetson Thor system using the Nemotron model as its 'brain' to control physical agents.
- Running AI models locally on a robot's hardware significantly reduces latency and dependency on cloud connectivity, which is critical for autonomous systems requiring real-time responses. This approach enhances data privacy by processing sensor information on the device and allows for operation in environments without reliable internet access. - The NVIDIA Jetson Thor is a significant leap in edge computing, delivering up to 2,070 FP4 TFLOPS of AI performance, which is a 7.5x increase in AI compute compared to the previous generation's Jetson AGX Orin. It integrates a Blackwell architecture GPU and a 14-core Arm Neoverse CPU, with its 128GB of memory allowing large models to run entirely on the device without disk swapping. - NVIDIA’s Nemotron is a family of open-source models optimized for on-device reasoning and "agentic AI," where an AI can proactively perform tasks. The family includes different sizes, such as the efficient ‘Nano’ model, allowing developers to scale the AI to fit the hardware's capabilities, from the edge to the data center. - The move to powerful, on-device AI hardware and models is a key enabler for "embodied AI," where robots learn and reason through direct interaction with the physical world. This contrasts with traditional robotics that rely more heavily on pre-programmed instructions and less on real-time, learned behaviors. - While specialized frameworks like Dimensional are emerging, the dominant open-source framework in robotics is the Robot Operating System (ROS). ROS 2, the current generation, is designed for commercial and industrial applications with features like real-time support and enhanced security, making it a foundational layer upon which capabilities like local AI models can be integrated. - The demonstration on an NVIDIA Jetson Thor system is significant as this platform is specifically designed for high-performance edge AI in robotics. Early adopters of Thor for developing next-generation robots include Boston Dynamics, Figure AI, and Amazon Robotics. - The Nemotron-3 models use a hybrid Mixture-of-Experts (MoE) architecture, which makes them more compute-efficient than standard transformer models. This efficiency is crucial for deploying powerful AI on resource-constrained edge devices like robots.
