Audio to Photoreal Embodiment
A framework for generating full-body photorealistic avatars.
CommonProductImageFull-body photorealistic avatarPose and movements
Audio to Photoreal Embodiment is a framework for generating full-body photorealistic avatars. It generates diverse poses and movements of the face, body, and hands based on conversational dynamics. The key to its method lies in combining the sample diversity of vector quantization with the high-frequency details obtained from diffusion, resulting in more dynamic and expressive movements. The photorealistic avatars generated for visualizing the movements can express subtle nuances in poses (e.g., sneering and arrogance). To promote this research direction, we introduce a novel multi-view conversational dataset that enables photorealistic reconstruction. Experiments demonstrate that our model generates appropriate and diverse actions, outperforming diffusion and vector quantization-only methods. Furthermore, our perceptual evaluation highlights the importance of photorealism (compared to meshes) in accurately assessing subtle action details within conversational poses. Code and dataset are available online.
Audio to Photoreal Embodiment Visit Over Time
Monthly Visits
25537072
Bounce Rate
44.24%
Page per Visit
5.9
Visit Duration
00:04:47