NVIDIA demos Gemma 4 on Jetson

Small edge computers are getting surprisingly good at running real multimodal AI locally. That matters because the usual cloud pattern is slow, expensive, and awkward for anything that needs a camera or microphone always on. The gap has been obvious for a while — big models were powerful, but the hardware that fits inside robots, kiosks, or office devices usually could not run them well enough. What changed is that NVIDIA has now put Google’s Gemma 4 family onto Jetson, and the most interesting demo is the tiny end of that stack: Gemma 4 running fully on a Jetson Orin Nano Super with voice and webcam in the loop. (developer.nvidia.com) ### What actually ran on the tiny box? The demo used a Jetson Orin Nano Super with 8 GB of memory, a webcam, microphone, and speaker. You talk, speech gets turned into text with Parakeet, Gemma 4 answers, and if the question needs vision context the system grabs an image from the(developer.nvidia.com) than bouncing every step to a cloud API. (huggingface.co) ### Which Gemma 4 model fits there? Gemma 4 is a family, not one model. Google released four variants on March 31, 2026: E2B, E4B, 26B-A4B, and 31B. On Jetson, memory is the real constraint, and NVIDIA’s own Jetson AI Lab guide is pretty blunt: Orin Nano is mainly an E2B machine, Orin NX is where E4B starts to make more sense, and AGX Orin or Thor are where the larger models become practical. (ai.google.dev) ### Why is E2B the interesting one? Because E2B is the version built for on-device use. NVIDIA describes E2B and E4B as the on-device branch of the family, with multimodal input support and lower effective size than the big reasoning models. In plain English, this is the difference between “cool benchmark model” and “something you can actually embed in a product with a power budget.” (developer.nvidia.com) ### What software path made this work? Turns out the runtime matters almost as much as the model. NVIDIA’s Jetson guide steers Orin Nano users toward llama.cpp and GGUF builds, while vLLM is the preferred path on larger Jetsons for better serving performance. That split tells you(developer.nvidia.com) matter more. (jetson-ai-lab.com) ### So was the “10x faster” claim real? There is evidence of that number floating around the Jetson ecosystem, but not in the strongest primary-source materials tied to this demo. The solid, documented part is narrower: NVIDIA published the Jetson support path on April 2, and a local voice-plus-vision demo for Orin N(jetson-ai-lab.com)ne — it is that a usable multimodal loop now fits on an 8 GB Jetson at all. (developer.nvidia.com) ### Why does local multimodal matter so much? Latency is the obvious reason. If a device has to ship audio, wait for transcription, call an LLM, maybe call vision, then wait for speech synthesis, the interaction feels mushy. Local inference cuts out a lot of round trips. It also (developer.nvidia.com)ensors. NVIDIA is clearly pitching Gemma 4 for exactly those latency-sensitive and on-prem use cases. (developer.nvidia.com) ### What does this unlock next? Basically, more capable edge assistants. Think meeting-room boxes, industrial terminals, retail kiosks, robots, and camera systems that can answer questions about what they see without needing a fat cloud pipeline for every interaction. The catch i(developer.nvidia.com)t that is still a big shift: the floor for useful local AI just dropped. (jetson-ai-lab.com) ### Bottom line This is less about one flashy demo than about a threshold getting crossed. Gemma 4 is now small and efficient enough that NVIDIA can show a real voice-and-vision assistant running on a tiny Jetson box. That makes local multimodal AI feel less like a lab trick and more like a product category.