Search AI Products and News

Frustrated by the slow processing speed of large language models on long documents? Researchers from Tsinghua University have unveiled a groundbreaking technology – the APB parallel inference framework – that dramatically accelerates processing. Benchmark tests show this technology achieves a 10x speed improvement over Flash Attention when handling ultra-long texts. With the rise of models like ChatGPT, AI's ability to process vast amounts of text (hundreds of thousands of words) has increased significantly. However, this often comes at the cost of processing speed...

The groundbreaking Transformer acceleration technology FlashAttention-3 has been launched! This is not just an upgrade; it heralds a direct increase in the inference speed of our Large Language Models (LLMs) and a direct decrease in costs! Let's talk about FlashAttention-3 first, which is a significant improvement over its predecessors: Significant increase in GPU utilization: Training and running large language models with FlashAttention-3 doubles the speed, up to 1.5 to 2 times faster, and thi