Apple Details Vision Pro Environment Design
The team behind Apple's visionOS 26 has detailed the design process for the Vision Pro's immersive virtual environments. The process relies on advanced LiDAR and environmental scanning to create realistic, location-aware digital experiences, reinforcing Apple's strategy of merging digital content with the physical world.
- The design of Earth-based environments, like Yosemite, involves extensive fieldwork, including 360-degree panoramic photography, high-resolution texture capture of elements like rock and bark, and LiDAR scans to build a cohesive 3D mesh. - For the new Jupiter environment in visionOS 26, Apple collaborated with NASA's Jet Propulsion Laboratory (JPL) to accurately model the surface of its moon, Amalthea, which scientists theorize is composed of rocks held together by ice due to Jupiter's gravity. - Human Interface Designer Matt Dessero and his team built and arranged the virtual environments from within the Vision Pro itself, a necessary step because traditional monitors cannot accurately convey the scale and spatial relationships of the experience. - The LiDAR scanner in the Vision Pro utilizes "Direct Time of Flight" technology, reusing a design first field-tested in the iPad Pro, which measures the time it takes for a laser to reflect off objects to create a detailed 3D map of the user's surroundings. - Apple's long-term strategy for spatial computing began laying its foundation in 2017 with the introduction of ARKit, which accustomed both developers and millions of iOS users to the concept of digital objects interacting with the physical world. - While Apple markets the Vision Pro as a "spatial computer," key competitors in the broader XR market include the Meta Quest 3, which focuses on a more accessible price point, and the Samsung Galaxy XR, which leverages the Android XR ecosystem. - The underlying hardware includes the Apple M-series chip for processing and a dedicated R1 chip that specifically handles the input from 12 cameras, five sensors, and six microphones to ensure a low-latency, 12-millisecond photon-to-photon passthrough of the real world. - To protect user privacy, third-party apps do not get direct access to the camera feeds; instead, they can request access to a 3D model of the "surroundings data" that combines camera and LiDAR information.
