Teleop + VR trains warehouse bots

A warehouse robot can learn a pick the way a trainee does: by watching a human do it in virtual reality, then repeating the motion on real shelves. (arxiv.org) Teleoperation means a person controls the robot from somewhere else, like a remote driver steering a car through cameras and sensors. In recent robot systems, a virtual reality headset tracks the operator’s hands and arms, then turns those movements into robot commands in real time. (open-teach.github.io) That setup is being used to collect demonstration data — recordings of how a person reaches, grips, lifts, and places objects. The Open Teach system from New York University and Meta said in March 2024 that it used a Meta Quest 3 headset, ran at up to 90 hertz, and worked across 38 tasks on robot hands, arms, and mobile manipulators. (arxiv.org) Those motion records can then be fed into robot-learning software, which uses examples instead of hand-written rules for every object and angle. The Open Teach paper said its collected data was compatible with policy learning on 10 dexterous manipulation tasks, including contact-heavy actions where small errors matter. (open-teach.github.io) NVIDIA has pushed the same workflow into its Isaac Lab software stack. In a September 29, 2025 post, the company said Isaac Lab 2.3 added support for teleoperation data collection with devices including Meta Quest virtual reality headsets and Manus gloves to speed up demonstration datasets. (developer.nvidia.com) Warehouse picking is the target because it is still one of the most labor-heavy jobs in fulfillment. A 2025 STIQ report said fulfillment picking, including decant and storage, accounts for about 52% of warehouse operating costs. (lightningpick.com) Picking is also harder than moving a fixed box or pallet because each item can sit at a different angle, under different lighting, in a different bin. That is why developers use human demonstrations to capture the small wrist turns, approach paths, and grip timing that are tedious to program by hand. (arxiv.org) The pitch to warehouse operators is shorter setup time. Instead of waiting for robotics engineers to tune every motion from scratch, a site can have an operator perform representative picks, save those trajectories, and use them to fine-tune the robot for local inventory and shelf layouts. (developer.nvidia.com) The hardware cost is also falling. Open Teach described its Quest 3-based interface as built on a $500 headset rather than specialized teleoperation rigs, and the project was released as open source for Franka, xArm, Jaco, Allegro, and Hello Stretch platforms. (open-teach.github.io) That does not make warehouse robots fully autonomous overnight. It does mean more of the training step can start with a headset, a human picker, and a pile of examples — which is closer to how warehouses already teach people to work. (arxiv.org)