Apple recently filed a patent titled "Extended Reality Video Adaptive Quantization Matrix," aiming to significantly optimize the transmission efficiency of XR (extended reality) video through intelligent partitioning and differentiated encoding. This innovative technology is expected to bring revolutionary improvements to the real-time transmission of virtual-real fusion images in AR/VR scenarios.
The core of this patent lies in intelligently dividing the XR image into two main regions: the "virtual area" containing virtual objects and the "background area" of reality. Based on the importance of these areas, Apple adopted a refined three-level quantization strategy:
- High-Precision Virtual Object Encoding: For the "virtual area" containing virtual objects, the system will use high-precision encoding methods, with the quantization parameter (QP value) reduced by up to 30%, ensuring the clarity and detail of virtual elements.
- User Gaze Area Optimization: Furthermore, for the region currently being looked at by the user, the system will allocate an additional 15% of bitrate to ensure the visual experience at the user's focus point reaches its best.
- Efficient Background Area Compression: As for the relatively unimportant background area, a more compressed encoding method will be used, thereby effectively reducing the overall data volume.
Test data shows that after adopting this solution, while maintaining the image quality of key areas such as virtual objects and user gaze points, the overall bitrate can be reduced by up to 40%. This is crucial for the real-time transmission of AR/VR content with high bandwidth demands, significantly reducing network latency and bandwidth consumption, and enhancing user experience.
In addition, this patent also proposes a dynamic partitioning algorithm that can automatically adjust block sizes based on the complexity of the image. In complex areas, the smallest block size can be subdivided to 4x4 pixels to ensure that even extreme details are fully preserved.
The launch of this patent technology once again demonstrates Apple's determination to deepen its efforts in the XR field and is expected to provide more efficient and smoother visual transmission solutions for its future AR/VR devices.