Meta Releases Llama 2-Long Model, Reducing Computational Demand for Long Text Processing by 40%

Meta has released the Llama2-Long model, which does not increase computational demand when processing long texts while still maintaining excellent performance. The model employs continuous pre-training, improved positional encoding, and data mixing strategies to reduce computational overhead by up to 40%. It performs outstandingly on both long and short tasks, surpassing other long-context models and has the potential to revolutionize the field of natural language processing. The model shows significant improvements in encoding, mathematical, and knowledge-intensive tasks, even surpassing GPT-3.5. The release of the Llama2-Long model provides a powerful solution for handling long texts.