Recently, the Bilibili (shortened as B站) team officially open-sourced a new anime video generation model called AniSora on May 12. This tool enables users to create various styles of anime video clips with just one click, covering types such as series episodes, Chinese original animations, comic adaptations, VTuber content, anime trailers, and meme animations, thus enriching the possibilities for anime creation.
The B站 team pointed out that although there are already several natural video generation models available in the market like Sora, Kling, CogVideoX, etc., existing technologies in the anime video field still fall short. Anime, with its unique artistic style, exaggerated movements, and breaking through the physical laws of reality, poses significant challenges for video generation. Therefore, the launch of AniSora will fill this gap in the field.
The AniSora project includes a comprehensive system mainly composed of the following parts: First is the data processing pipeline, which used over 10 million high-quality data samples for model training; secondly, AniSora introduces controllable generation models, incorporating temporal masking modules, supporting multiple key animation production functions such as image-to-video generation, frame interpolation, and local image guidance; finally, the team collected 948 diverse animation videos to build an evaluation dataset. After double-blind experiments and VBench tests, it was concluded that both character consistency and motion consistency reached the current highest standard (SOTA).
To help the public better understand the powerful features of AniSora, B站 also shared a series of generated video examples. For instance, in one video, a character sitting in a moving car exhibits natural dynamics as their hair moves with the wind; in another, two figures dressed in red wedding robes slowly walk away holding a red rope; and in a third, a blonde character gently touches a kneeling person, showcasing delicate emotional expression.
The open-source address for AniSora has been released on GitHub and other platforms. Developers and anime enthusiasts interested can visit and use it. With the introduction of this technology, we may experience more innovation and surprises when creating and watching anime content in the future.