Meta has officially released its VR scanning tool Meta Horizon Hyperscape Capture (Beta), allowing Quest 3 headset users to scan real-world environments and recreate them in virtual reality with photo-realistic quality. The release of this tool fulfills a promise Meta made at the Connect 2024 conference.
From Demo to Reality: User-level Application of Gaussian Splatting
Last year at the Connect 2024 conference, Meta showcased the power of Gaussian Splatting technology through a demo application called "Horizon Hyperscape Demo," presenting real-world scenes with astonishing realism in VR. Now, this technology is no longer just a demonstration but is officially available to regular users.
Creating a Horizon Hyperscape scene involves three steps, with the first two steps performed on the Quest 3 headset and the third step relying on Meta's servers for processing.
Scene Mesh Generation: Users first need to wear a Quest 3 or Quest 3S headset and look around the room to generate a scene mesh. This process is similar to setting up mixed reality features and usually takes between 10 to 30 seconds.
Fine Scanning: Next, users need to walk around the room and bring the headset close to all surfaces to eliminate the 3D mesh and capture details. This fine scanning process is currently the most time-consuming part, which may take several minutes.
Cloud Processing and Transmission: After completing the scanning on the device, the data will be uploaded to Meta's servers. After 2 to 4 hours of processing, users will receive a notification to view and experience the final VR scene.
Cloud Streaming and Experience Details
The core of Horizon Hyperscape is the Gaussian Splatting technology, which is the same as the one used in last year's demo application "Horizon Hyperscape Demo." The rendered scenes are streamed from Meta's cloud server using "Project Avalanche" technology, meaning all complex computations are done in the cloud rather than on the device.
Although this technology brings an unprecedented level of realism, there are still some shortcomings. For example, small text remains blurry, and in certain viewing angles, such as crouching to look under furniture, obvious geometric distortions may occur. These imperfections stem from the headset not capturing those areas during the scanning process. Overall, however, this technology is considered one of the most realistic scene capture technologies available today.
