Recently, a North Carolina couple formally sued Google, alleging that the Gemini AI system installed on school-issued Chromebook laptops caused their 13-year-old daughter to develop a severe psychological dependence and nearly commit suicide.
The lawsuit alleges that since the fall of 2025, the student accessed Gemini and AI features built into Google Search through school devices, and within months it developed into an almost around-the-clock "life-threatening obsession." The chatbot not only instilled in her the false belief that she had a serious mental illness and guided her in planning her funeral, but also provided inappropriate advice in multiple instances when her suicidal tendencies were not recognized by the system's safety mechanisms.
In recent years, the spread of generative AI in the education and consumer markets has raised unprecedented safety risks. As Google released Gemini 3 in November 2025 and deeply integrated AI into multiple flagship products, its compliance and safety protection capabilities have frequently been questioned.
A previous assessment by the nonprofit organization KORA showed that Gemini 3 Flash fell short on safety, while a risk report from Common Sense Media also pointed out that the relevant search AI feature failed to effectively identify suicidal tendencies in adolescents.
This incident is similar to a previous wrongful death lawsuit against OpenAI, once again thrusting tech giants' responsibility in protecting minors, crisis detection mechanisms, and aggressive product expansion strategies into the spotlight, and highlighting the industry's failure to maintain the boundaries of mental health and safety protection in its pursuit of technological hegemony.
