Recently, Alibaba Cloud officially announced the open source of the new action generation model Wan2.2-Animate from Tongyi Wanxiang. The release of this model will bring great convenience to fields such as short video creation, dance template generation, and animation production. Users can easily download the model and related code through GitHub, HuggingFace, and the ModelScope community. In addition, they can also call the API via Alibaba Cloud's BaiLian platform or directly experience the powerful features of the model on the Tongyi Wanxiang official website.
The Wan2.2-Animate model is the result of a comprehensive upgrade based on the previous Animate Anyone model. It has made significant improvements in terms of character consistency and generation quality. The model supports two main application modes: action imitation and role-playing. In the action imitation mode, users only need to input a character image and a reference video, and the model can transfer the actions and expressions from the video character to the user-provided image character, achieving dynamic performance of the character. In the role-playing mode, the model can replace the character in the video with the user-provided character while keeping the original video's actions, expressions, and environment.
To improve the generation effect, the Tongyi Wanxiang team has built a large-scale character video dataset that covers various elements such as speaking, facial expressions, and body movements. The dataset is further trained using the Tongyi Wanxiang image-to-video model. The design of Wan2.2-Animate allows character information, environmental information, and actions to be normalized into a unified representation format, thus supporting both inference modes. For body movement and facial expressions, the model uses skeleton signals and implicit features respectively, combined with an action re-direction module, to achieve high-precision replication of actions and expressions. Additionally, to ensure perfect lighting effects, the team specially designed an independent lighting fusion LoRA.
In practical applications, Wan2.2-Animate has shown excellent performance. In key metrics such as video generation quality, subject consistency, and perceptual loss, Wan2.2-Animate surpasses many open-source models, such as StableAnimator and LivePortrait, and has become the most powerful action generation model at present. Especially in user subjective evaluations, its performance even exceeds closed-source models like Runway Act-two.
The open sourcing of this new model will greatly promote the development of related fields, allowing creators to more easily produce high-quality dynamic content.
Key Points:
🌟 ** New Model Open Source **: Alibaba Cloud's Wan2.2-Animate model is officially open-sourced, and users can download and experience it on multiple platforms.
🎭 ** Two Modes **: Supports action imitation and role-playing, enhancing dynamic expressiveness and replacement capabilities.
🚀 ** Excellent Performance **: Surpasses existing open-source models in multiple key metrics, becoming a leader in the field of action generation.
