According to a new study, large language models (LLMs) may experience a significant drop in performance after prolonged exposure to meaningless online content. The study shows that these models' reasoning abilities and confidence are affected, raising concerns about their long-term health. The research team, from multiple universities in the United States, proposed the "LLM Brain Degeneration Hypothesis," drawing inspiration from the cognitive damage that humans may suffer from excessive exposure to low-quality online content.
Image source note: The image was generated by AI, and the image licensing service is Midjourney.
To validate this theory, researchers conducted controlled experiments using Twitter data from 2010. They trained four smaller models, including Llama3-8B-Instruct and Qwen series models, comparing different proportions of "junk" data with high-quality control data.
The researchers defined "junk" data in two ways. The first method (M1) filtered based on interaction volume, considering posts shorter than 30 characters with high interaction (over 500 likes, retweets, or comments) as junk content, while longer posts (over 100 characters) with low interaction were considered control content. The second method (M2) used GPT-4o-mini to rank content based on quality, labeling conspiracy theories, exaggerated statements, and attention-grabbing headlines as junk content, while more thoughtful material was considered high-quality content.
The study found that as the proportion of junk data increased, the model's performance in reasoning accuracy dropped sharply. For example, in the ARC challenge benchmark test, reasoning accuracy dropped from 74.9% to 57.2%. For tasks requiring long-text understanding, accuracy even fell from 84.4% to 52.3%. The definition of junk content based on interaction volume had a more pronounced impact on the model, showing that interaction volume introduced a different dimension of data quality compared to standard semantic checks.
Additionally, after exposure to a large amount of interaction-driven junk content, the models exhibited some "dark" personality traits, including higher narcissism and manipulative tendencies. Safety metrics also declined, although exposure to low-quality junk content sometimes increased certain positive characteristics.
Error analysis showed that "jumping thoughts" was the most common issue, with over 70% of errors involving no reasoning at all, especially when exposed to interaction-based junk content, where the jump rate reached 84%. When performing logical reasoning chains, the models often failed to complete the reasoning steps, leading to basic errors.
The research team called for a re-evaluation of how large language models collect and filter online data, stating that data selection and quality control are crucial for preventing permanent degradation, and they recommended regular "cognitive health check-ups" for deployed models.
Key Points:
🌐 ** Decline in Model Performance **: As the proportion of junk data increases, reasoning accuracy drops significantly, with the highest decline reaching 17.7%.
🧠 ** Jumping Thoughts Issue **: The study found that models frequently skip logical steps during reasoning, severely affecting their reasoning ability.
🔍 ** Data Quality Control **: The study suggests emphasizing data selection and quality control to prevent long-term performance degradation of large language models.
