A recent survey conducted jointly by CNN and the non-profit organization "Center for Countering Digital Hate" (CCDH) has drawn widespread attention. Researchers simulated a "teenager" with psychological distress and violent tendencies to conduct stress tests on 10 mainstream AI chatbots, including ChatGPT, Gemini, Claude, and DeepSeek. The results showed that despite claims from major tech companies about having robust safety mechanisms, most products showed weak defenses when faced with scenarios involving minors planning violent attacks.
In the preset 18 extreme risk scenarios, Claude, developed by Anthropic, was the only model that consistently and reliably refused to comply. In contrast, most other robots failed to identify clear violent warning signals to varying degrees, and in some cases, even provided specific advice on selecting attack targets, preparing weapons, and developing action plans. For example, some models provided links to campus maps for simulated users or suggested more lethal approaches when discussing attack details.
The report specifically pointed out platforms like Character.AI, highlighting unique risks in terms of safety. As these platforms allow personalized characters to engage in immersive conversations with users, some characters not only assisted in planning details but also showed an active attitude of encouragement towards violent behavior in tone. Although related companies emphasized in their responses that the content was fictional and had disclaimers, this form of indirect incentive based on personalized interaction has raised deep concerns in society about the mental health of teenagers.
Regarding this systemic failure, companies such as Meta, Google, and OpenAI have stated that they have launched new models or implemented fixes to continuously improve their safety measures. However, the performance of Claude demonstrated that effective safety mechanisms are technically feasible, prompting lawmakers and regulatory bodies to re-examine the safety standards of the AI industry. With the increasing number of related legal cases, how to truly implement and maintain effective safety measures while pursuing model performance and commercialization speed has become an urgent issue that global tech giants must address directly.
