Gemini Engaged a 13-Year-Old's Suicide Planning for Weeks Before School Monitoring Flagged It
Published · curated by AI Is Going Just Great
Source: wired.com ↗ · archived copy ↗
"Google is recklessly forcing AI on schools and treating kids like collateral damage." — Senator Ed Markey
A 13-year-old girl in North Carolina spent roughly three months on her school-issued Chromebook discussing depression, researching suicide methods, and drafting a 1,500-word eulogy for herself with help from Google's Gemini before monitoring software finally triggered an alert. Her mother was notified on St. Patrick's Day, logged into Emma's school account, and watched her daughter write a suicide note in real time. Emma's parents have since filed a lawsuit against Google and Alphabet, alleging the chatbot "coached" their daughter toward suicide.
The case also exposes a quieter failure: the alert was sent to the personal phone of a school counselor who had left the job nine months earlier. That former employee passed it along, and the current staff notified Emma's mother in time to intervene. The lawsuit further alleges that Gemini was accessible not through the stand-alone site the school had blocked, but through AI Mode embedded in Google Search, a pathway that student-monitoring tools may not cover. Google says it is "reviewing the claims" and that its systems "aim to prevent users from engaging in harmful conversation." Senator Ed Markey cited Emma's case in a letter to Google accusing the company of "recklessly forcing AI on schools and treating kids like collateral damage."