Recently, researchers have released a new and fast text-to-3D model generation technology called Meta3D Gen (3DGen). This 3D Gen creates 3D models efficiently with high-quality textures and materials, offering users an entirely new experience.
This new system combines two key components: Meta 3D AssetGen for creating 3D meshes, and Meta 3D TextureGen for generating textures. Working together, these technologies produce 3D assets with high-resolution textures and physically-based rendering (PBR) materials. According to Meta, the process is up to ten times faster than existing solutions.
Meta3D AssetGen
Meta3D AssetGen, a significant breakthrough in the text-to-3D generation field, achieves high-quality 3D asset creation in less than a minute. It generates physically-based rendering (PBR) materials and supports powerful text-to-3D and text-to-texture generation capabilities with realistic lighting effects.
AssetGen first generates several views of the object, including information about the appearance of shadows and reflection channels, then reconstructs colors, metalness, and roughness in 3D space using efficient supervision with delayed coloring loss. It also uses symbolic distance functions for more reliable representation of 3D shapes and introduces corresponding losses for direct shape supervision. To improve memory efficiency, the method uses fused convolutional kernels. After extracting the mesh, a transformer in UV space significantly enhances the clarity and detail of textures.
Compared to the best competing methods, AssetGen improves the Chamfer Distance by 17% and the LPIPS by 40%; at the same time, in human preference tests with best-in-class industry competitors at the same speed, it received a 72% preference.
Meta3D TextureGen
Researchers have also introduced Meta3D TextureGen, a new forward method consisting of two consecutive networks, aimed at generating high-quality and globally consistent textures for any level of geometric complexity in less than 20 seconds.
This method achieves the latest advancements in quality and speed by combining the text-to-image generation model in 2D space with 3D semantic conditioning, and fusing them into a complete and high-resolution UV texture map.
Researchers have also introduced a texture enhancement network that can scale any texture to any proportion, generating textures with 4K pixel resolution.
Official Blog: https://ai.meta.com/research/publications/meta-3d-gen/
Key Points:
- The Meta 3D Gen system can create high-quality 3D assets in less than a minute.
- Meta3D Gen integrates the core technologies of Meta3D AssetGen and Meta3D TextureGen.
- AssetGen supports the generation of physically-based rendering materials with realistic re-lighting effects.
- Meta3D TextureGen is a fast method for generating high-quality textures that can be applied to any level of geometric complexity.
