Large Model Companies Focus on 'Long Text': The Real Challenge of Context Length in Transformer Structure

Decrypting the Dark Side of the Moon o1: Long-CoT is the Key, Model Thinking Needs to 'Cast a Long Line'

Dark Side of the Moon researcher Flood Sung recently published a lengthy article, revealing for the first time the development ideas behind the k1.5 model and deeply reflecting on the technological insights brought by the OpenAI o1 model. According to Flood Sung, the importance of Long-CoT (Long Chain of Thought) was actually validated over a year ago by Tim Zhou Xinyu, co-founder of the Dark Side of the Moon. By training small models for multi-digit calculations and transforming the fine-grained calculation processes into Long Chain of Thought data for SFT (Supervised Fine-Tuning).

Doubao's Large Model Claims to Match GPT-4, Revealing Its Capability of 3 Million Long Texts for the First Time

ZhiYuan and Tencent Launch Long Text Understanding Benchmark Model LongBench v2

On December 19, 2024, at a press conference, ZhiYuan Research Institute and Tencent announced the launch of LongBench v2, a benchmark designed to evaluate the deep understanding and reasoning capabilities of large language models (LLMs) in real-world long text multi-task scenarios. The platform aims to advance long text models in understanding and reasoning, addressing the current challenges faced by large language models in practical applications.

Aliyun Bailing Launches Tongyi Qianwen Qwen2.5-Turbo Million Long Text Model

The Aliyun Bailing platform recently announced the launch of the Qwen2.5-Turbo million long text model, developed by the Tongyi Qianwen team. The Qwen2.5-Turbo model supports processing ultra-long contexts of up to 1 million tokens, equivalent to 1 million English words or 1.5 million Chinese characters.

Flow 2.0 Version Release: Beyond the Mainstream Market Model, Providing Exceptional Long Text Reading and Comprehension Skills

Recently, the AI assistant Flow officially released version 2.0, introducing the new 'Featured Content' and 'Discover' sections that provide users with more intelligent and personalized high-quality content services. These features demonstrate the practical value of artificial intelligence platforms in both life and work, especially in addressing users' confusion and needs. With the continuous maturity of large model technology, Flow offers users a richer variety of information in graphic, text, and Q&A formats through AI Native Content (AINative). The new version's 'Featured Content'

Quark Releases Intelligent Dialogue Assistant CueMe Supporting 20,000 Words of Long Text Creation

Recently, Quark officially launched its self-developed intelligent dialogue assistant, CueMe. CueMe is a multifunctional smart assistant that integrates literary creation, knowledge acquisition, and emotional companionship, aimed at providing users with comprehensive information services. Leveraging advanced artificial intelligence technology, CueMe can meet users' diverse needs in learning and work, whether it’s writing essays, reports, or creating novels and copy, CueMe offers efficient support.

Google AI Studio Launches New Prompt Library for Testing Long Text Context, Native Multimodal Capabilities, and More

Google AI Studio has released a new native prompt library that integrates features such as long-text context, multimodal inputs (images, videos, audio), and structured outputs. The aim is to provide users with a rich and diverse experience across various fields including education, programming, and travel. This library includes practical features like recipe generators, math tutors, math homework generators, treasure hunt design, unit test generators, and travel suggestions, all designed to enhance user exploration and innovation through intuitive displays and comprehensive information. Users only need to log into Google AI Studio.

Tsinghua University Launches LongWriter, a Generative AI System that Challenges Human Writers

The research team at Tsinghua University has successfully developed the 'LongWriter' artificial intelligence system, which can generate coherent texts exceeding 10,000 words, paving new avenues for long-form writing and content creation. By constructing the 'LongWriter-6k' dataset, which includes 6,000 writing samples, the research team has increased the maximum output length of the AI model from approximately 2,000 words to over 10,000 words. This innovative achievement not only enhances the AI's performance in long text generation tasks but even surpasses larger-scale models.

Zhipu AI: GLM-4-Long API Launched with Input and Output Price of 0.001 Yuan / 1,000 Tokens

Zhipu AI announces that the LLM GLM-4-Long, which supports ultra-long context lengths, is now available on the open platform bigmodel.cn. This model is designed specifically for processing long texts and can read the equivalent of two copies of 'Dream of the Red Chamber' or 125 papers in one go. It is widely applied in scenarios such as translating long documents, global analysis of financial reports, extracting key information, and building chatbots with ultra-long memory.

New Breakthrough in GPU Optimization! 'Tree Attention' Accelerates Inference of 5 Million Long Texts by 8 Times

The Transformer architecture, a star in the field of artificial intelligence, has led a revolution in natural language processing with its self-attention mechanism at its core. However, when handling long contexts, the resource consumption of self-attention calculations becomes a bottleneck. To address this issue, researchers have proposed the Tree Attention method, which decomposes the calculation tasks through tree reduction to improve efficiency. This method not only reduces communication overhead and memory usage but is also 8 times faster than existing methods in a multi-GPU environment. The introduction of Tree Attention not only enhances performance.