In the recently concluded Zhipu Multimodal Open Source Week, the Zhipu team announced the open-sourcing of four core technologies for video generation. These technologies not only demonstrate Zhipu's latest progress in the field of multimodal models but also lay a solid foundation for the future development of video generation.
Over the past week, the Zhipu GLM team has released several multimodal models covering areas such as visual understanding, device operation, and speech processing. These include the GLM-4.6V visual understanding model, the AutoGLM device control model, the GLM-ASR speech recognition model, and the GLM-TTS speech synthesis model. The release of these technologies aims to enable large models to have more human-like world knowledge, memory capabilities, and complex reasoning abilities.
On the last day of the Open Source Week, the Zhipu team introduced four new technologies: SCAIL, RealVideo, Kaleido, and SSVAE, focusing on solving key challenges in the field of video generation. These technologies cover aspects such as fine-grained controllable generation, complex spatiotemporal structure modeling, and large-scale training cost control.
SCAIL technology is dedicated to generating cinematic-level character animations, enabling precise control over complex poses and ensuring structural integrity of generated characters during movement. RealVideo is a real-time streaming video generation system that significantly reduces generation latency, completing video output in just 2-3 seconds, making interactions with AI characters more natural and smooth.
Kaleido technology focuses on multi-agent video generation, ensuring consistency between multiple agents and avoiding common issues of feature confusion. SSVAE, on the other hand, improves the training process, enhancing the training efficiency of video generation models, thereby tripling the convergence speed at the same quality level.
The Zhipu team stated that open-sourcing these technologies aims to inspire innovation within the video generation technology community, providing developers with more engineering solutions and research foundations. At the same time, Zhipu looks forward to collaborating with more developers to explore the future of artificial intelligence and advance the realization of Artificial General Intelligence (AGI).
Key Points:
🌟 SCAIL: Achieves cinematic-level character animation generation, supporting complex pose control.
⚡ RealVideo: A real-time video generation system with a generation latency of only 2-3 seconds.
🎨 Kaleido: A multi-agent video generation framework that ensures agent consistency and avoids feature confusion.
