Search AI Products and News

Step1X-Edit is a groundbreaking open-source image editing model that achieves performance comparable to leading closed-source models such as GPT-4o. It offers a powerful and versatile solution for various image manipulation tasks.

In the field of AI art generation, diffusion models are transitioning from U-Net based architectures to Transformer-based architectures (DiT). However, the DiT ecosystem faces challenges in plugin support, efficiency, and multi-conditional control. Recently, a team led by Xiaojiu-z introduced EasyControl, an innovative framework designed to provide efficient and flexible conditional control capabilities for DiT models, effectively giving DiT models the power of ControlNet.

Recently, Kuaishou, along with research teams from Peking University and Beijing University of Posts and Telecommunications, jointly open-sourced an ultra high definition video generation model named Pyramid-Flow. This model can generate videos of up to 10 seconds long, with a resolution of 1280x768 and 24 frames per second, producing high-quality results with excellent lighting effects, action consistency, and overall video quality. Pyramid-Flow operates differently from existing video diffusion models, which typically run at full resolution; while this approach allows for...

At the 2024 Volcano Engine AI Innovation Roadshow, Volcano Engine President Tan Dai unveiled the latest Doubao · Video Generation Model. This model incorporates several advanced technologies, including the efficient DIT Fusion Computing Unit, which enables deep compression encoding of video and text; it also employs a novel diffusion model training approach to ensure consistency in generating multi-camera videos; in addition, the model integrates a deeply optimized Transformer architecture, significantly enhancing the generalization ability of video generation.

The domestic open source video generation model CogVideoX-5B has been officially released in the Mota ModelScope community, significantly improving the quality and visual effects of video generation. Based on the large-scale DiT model, this model utilizes a 3D causal variational autoencoder and expert Transformer technology, achieving spatio-temporal joint modeling through 3D-RoPE positional encoding and a 3D full attention mechanism. The use of progressive training techniques allows the model to generate long videos with distinct motion features, coherence, and high quality.

Tencent's Hunyuan DiT model has recently partnered with the community to release three new controllable plugins for ControlNet: tile (high-definition image enlargement), inpainting (image restoration and expansion), and lineart (line art generation), further expanding its ControlNet matrix. The addition of these plugins enables the Hunyuan DiT model to cover a wider range of application scenarios.

On August 14th, Beijing time, Kunlun Wanwei officially launched Melodio, the world's first AI streaming music platform, along with the AI music commercial creation platform Mureka. Both products are powered by Kunlun Wanwei's latest self-developed DiT (Diffusion Transformer) architecture music model, Skymusic2.0, marking a new height in AI music creation technology.

Comfy Org has made significant progress in technological innovation, with in-depth revisions and updates addressing the original issues of ComfyUI. Support for Flux models (including Flux.1-Dev and Flux.1-Schnell) has been added, along with the integration of Tencent's Hunyuan DiT model, which significantly enhances AI image generation capabilities and expands multilingual support. The fourth stable version of ComfyUI, v0.0.4, has been released, and it is recommended for use in production environments to ensure system stability.

AlphaFold3（AF3）是DeepMind团队的最新增强工具，用于预测蛋白质、核酸和小分子的结构。凭借能够解析单个蛋白质序列及其复合结构的能力，AF3实现了"食谱到三维结构"的转变。其架构由三项核心模块组成：输入准备梳理序列、检索具备相似结构的分子；表示学习运用多层注意力机制更新表征；结构预测通过条件扩散模型实现预测。AF3创造性地将各类分子转化为独特数值张量，仿佛给每个分子一个独一无二的"身份证"，并运用注意力机制捕捉分子间的互相关系。在预测层面，条件扩散模型从随机噪声开始，逐步构建结构预测。训练中多种损失函数和信心评估共同作用，提高了AF3的预测准确性和可靠性，实现自我反省与提升。AF3的可视化指导和先进架构，展示了人工智能在生物学领域预测模型的革命性进步。

Tencent's MetaGen Image Generation Large Model (Hunyuan DiT) has recently been upgraded to a 6GB VRAM version, making it easy for personal computer users to run. This version is compatible with plugins such as LoRA and ControlNet and has added support for the Kohya graphical user interface, lowering the threshold for developers to train personalized LoRA models. The Hunyuan DiT model has been upgraded to version 1.2, with improvements in image texture and composition.At the same time, Tencent ha

Tencent's Hunyuan DiT model has announced the full open-sourcing of its training code, including LoRA and ControlNet plugins. LoRA is a technique for fine-tuning large language models that allows training models with specific characteristics using a small amount of data, without modifying the original model or increasing its size.

The Stable Diffusion 3 model introduces the DiT (Diffusion Transformer) architecture, which significantly improves quality. Stable Diffusion 3 outperforms other text-to-image generation systems. The parameter count of Stable Diffusion 3 ranges from 800M to 8B. The SD3 architecture is developed in collaboration with core members of Sora and an assistant professor from New York University.

Good research does not fade away; it only becomes more relevant over time. In the past week, OpenAI's video generation model Sora has continued to attract attention. The team has launched a new version, SiT, which further improves quality, speed, and flexibility. The latest paper, "Scalable Diffusion Models with Transformers", has been recognized at ICCV2023, leading the advancement of technology. SiT explores a broader design space through interpolation, bringing new possibilities to the field of image generation.

RA-DIT improves the ability of language models to utilize retrieval information and provide relevant results from the retriever through a two-stage fine-tuning process. RA-DIT outperforms existing retrieval-augmented models in knowledge-intensive zero-shot and few-shot testing. The RA-DIT65B model shows improved performance in tasks that require knowledge, demonstrating the effectiveness of fine-tuning. The enhancement of retrieval capabilities brought by RA-DIT has shown good results in knowledge-intensive evaluations, confirming the effectiveness of fine-tuning.