Apple Vision Pro Team Details Design Process

- The design of immersive "Environments" begins with establishing an intended mood, such as calmness or focus, with Human Interface Designer Matt Dessero noting, "Nature is our base." For real-world locations like Mount Hood, this involves extensive fieldwork, combining 360-degree photography with LiDAR scans to build a unified 3D geometry mesh. - To create the new Jupiter environment in visionOS 26, a location impossible to capture physically, Apple's design team collaborated with NASA's Jet Propulsion Laboratory (JPL). This partnership helped theorize the surface of Jupiter's moon Amalthea, which was chosen as the vantage point for its specific compositional scale in relation to the gas giant. - A key part of the design workflow involves designers wearing the Vision Pro headset to guide artists on the layout of virtual elements, from the placement of rocks to larger compositional features. This iterative process is considered critical for ensuring the scale and feel of the environment are correct from the user's perspective. - Sound design is a crucial layer for realism; the team builds custom acoustic meshes that model how sound behaves within each virtual space. This creates effects like faint echoes off granite walls in the Yosemite environment, making the scene feel alive rather than static. - A primary design challenge was overcoming the sense of isolation common in other headsets. This led to the development of core features like EyeSight, which reveals the user's eyes, and a control scheme based on subtle hand and eye movements to ensure users feel comfortable and connected to people in their physical space. - The upcoming visionOS 26 update will use a generative AI algorithm to create "Spatial Scenes" from users' 2D photos, adding realistic depth and multiple perspectives. This allows for more immersive viewing of existing image libraries within the Photos app and Safari. - In addition to visual enhancements, visionOS 26 introduces persistent, spatial Widgets that can be placed in a user's physical space and will reappear each time the headset is worn. Developers can customize these widgets with varying frames, colors, and depths. - The new operating system also adds native support for 180-degree and 360-degree video formats from third-party cameras like GoPro and Insta360, expanding the range of immersive content that can be experienced on the device.