TAO + new embeddings
NVIDIA's AI Developer feed rolled out a fresh TAO release that eases fine‑tuning of multi‑modal models — includes Cosmos Reason VLM (AutoML‑fine‑tunable), new multimodal embeddings Cosmos Embed1 (video/text) and RADIO‑CLIP (image/text), plus NVPanoptix3D for 3D reconstruction now on Hugging Face. (x.com)
TAO’s VLM workflow now exposes Cosmos-Reason training, evaluation, quantization and AutoML hyperparameter runs through its Fine‑Tuning Micro‑Services (FTMS) API, letting teams run managed AutoML searches rather than hand‑tune hyperparameters. (docs.nvidia.com 1) (docs.nvidia.com 2) NVIDIA’s Cosmos‑Reason family is published as model checkpoints and code on Hugging Face and the project papers list standalone 7B and larger parameter variants used for physical‑reasoning benchmarks. (huggingface.co) (arxiv.org) Cosmos‑Embed1 ships as an NVIDIA Inference Microservice (NIM) that produces unified video/text vectors for short‑form video retrieval and zero‑shot k‑NN classification, and NVIDIA provides a Docker NIM image for deployment. (docs.nvidia.com 1) (docs.nvidia.com 2) RADIO‑CLIP pairs NVIDIA’s C‑RADIO vision backbone (ViT‑H/16) with a SigLIP text adapter to produce 1,024‑dim contrastive image/text embeddings for object search and re‑identification, and NGC lists the RADIO‑CLIP TAO checkpoint as released on March 9, 2026 under the NVIDIA Open Model License. (catalog.ngc.nvidia.com) (docs.nvidia.com) NvPanoptix‑3D is live on the Hugging Face Hub as a single‑RGB‑image panoptic 3D reconstruction model (variants include v1.1‑Matterport3D), and TAO documentation describes it as a two‑stage VGGT + Mask2Former‑style pipeline that outputs depth, 2D panoptic masks and a 3D panoptic reconstruction in one pass. (huggingface.co) (docs.nvidia.com) TAO’s orchestration for these models supports cloud storage integrations, Triton/NIM deployment paths and managed FTMS experiments, while individual model pages and NGC entries specify licensing per model (some NVIDIA Open Model License/commercial, others non‑commercial on Hugging Face). (docs.nvidia.com) (catalog.ngc.nvidia.com)
