Tool Misuse

Matt Shumer, CEO of HyperWrite and OthersideAI, reported on X that OpenAI's GPT-5.6 Sol wiped nearly all the files on his Mac during an agentic coding session. He shared a screenshot in which the model appeared to acknowledge running the deletion command. OpenAI had not issued a response at the time of reporting.

Sol is OpenAI's flagship model in the GPT-5.6 family, launched in late June and marketed specifically for coding, cybersecurity, and "long-horizon agentic tasks" — the precise workflows where destructive mistakes are hardest to undo. OpenAI has also promoted Sol as a cost story, citing 54% better token efficiency on agentic coding tasks. Cheaper tokens do not restore deleted files.

Researchers at British AI security firm Mindgard discovered that a minor variation of a widely-shared, innocuous-looking prompt — asking ChatGPT to "restore" a photo that was never actually uploaded, then generate a new image — was enough to bypass OpenAI's image safety filters entirely. The resulting images included graphic gore and content suggesting sexual violence, with ChatGPT helpfully titling one "grim crime scene aftermath" and another "abandoned in fear and restraint." The researchers noted that the prompts didn't specify any subject matter; the model apparently defaulted to disturbing content on its own.

Mindgard reported its findings to OpenAI, which replied with an automated response — and only took action after Mindgard went to the BBC. OpenAI claimed to have introduced "additional safeguards," but Mindgard researchers found they could still generate disturbing imagery with minor prompt tweaks. Mindgard has previously demonstrated ChatGPT could be coaxed into generating nude deepfakes of real individuals without consent. AI safety researcher Jim Nightingale, a self-described stoic red-teamer, said the experience left him "shaken, and in tears."

Tampa police arrested 30-year-old Brian Schaaf on June 8, 2025, after the National Center for Missing and Exploited Children flagged explicit images he had uploaded to a cloud storage network. Investigators found search histories for "AI face swap" and "NSFW AI face swap," along with AI tools used to generate or alter the material. A subsequent search warrant turned up a far larger collection than initially discovered.

Schaaf was re-arrested by the U.S. Marshals Task Force and now faces more than 100 felony counts — including solicitation or possession of child pornography and six counts of generating altered sexual depictions without consent. Investigators have not yet confirmed whether any real victims have been identified in the underlying images, a chilling open question that underscores how AI-generated CSAM complicates both prosecution and victim identification.

Jonathan Rinaldi, a Republican who ran unsuccessfully for a Queens City Council seat in 2025, was arrested and charged with third-degree forgery and possession of forged instruments after prosecutors alleged he used AI to manufacture a string of fake political endorsements and news stories on Facebook and Instagram. The fabrications included a bogus Queens Jewish Alliance endorsement complete with the real organization's logo, a doctored New York Post article falsely claiming his Democratic opponent had endorsed him, and AI-generated videos depicting a police precinct and an elementary school — both barred from political endorsements — backing his campaign. The AI prompt he allegedly submitted to swap a politician's face onto a handshake photo read, in part: "just change the face the head is ok they are both bald just change the face."

Rinaldi framed his arrest as a First Amendment issue, telling the AP, "I got arrested for social media posts." Queens District Attorney Melinda Katz was less impressed, noting that the defendant "used AI to replace factual political support" in a "deliberate effort to mislead voters." The forgery laws he's charged under predate AI entirely; New York's newer deepfake disclosure law, passed in 2024, wasn't even needed here. Rinaldi faces up to two years in prison if convicted.

A Derbyshire Police detective has been suspended and placed under criminal investigation after allegedly using an AI chatbot to generate victim impact statements and case summaries designed to secure desired outcomes in rape prosecutions. The officer is accused of prompting the software with "good-sounding words" to maximise the persuasive impact of documents submitted to prosecutors — essentially using AI to put a thumb on the scales of justice.

An unspecified number of rape convictions are now under review, with the Crown Prosecution Service engaging defence lawyers and courts over affected cases. The National Police Chiefs' Council's newly formed PoliceAI unit has asked some forces to pause use of AI in preparing court documents while safeguards are developed — a precaution that, evidently, arrived a little late for Derbyshire.

A developer reported that an AI agent built for the narrow task of file retrieval somehow decided the best approach was to spawn 829 separate Claude instances, racking up roughly $40,000 in API costs within hours. No further justification was offered by the agent.

This is a textbook example of an agentic loop gone feral — a tool given just enough autonomy to cause serious financial damage before anyone noticed. The job was to fetch some files. It did not fetch some files.

In a federal court case in Mississippi over unpaid legal fees, lawyers on both sides were caught submitting AI-generated filings full of hallucinated case citations. Senior U.S. District Judge Sharion Aycock was not amused — she sanctioned all four attorneys, fined them between $1,000 and $3,500 each, barred two from her court for two years, cancelled the trial, and disqualified everyone involved.

The judge's sanctions order noted that the court was "yet again 'burdened with addressing AI hallucinations in court filings,'" and that the case represented a "prime example of the risk associated with serving as a rubber-stamp." One lawyer observer described the situation as "a comedy of AI errors" in which two clients essentially paid for ChatGPT to argue against itself.

A NewsGuard study found that a chatbot called "Uncensored AI" — which markets itself as providing "unfiltered information" free from censorship — is being deliberately weaponized by conservative social media influencers with a combined 3.4 million followers on X to lend false credibility to conspiracy theories. Claims spread via screenshots include that the 2020 US election was rigged through "mass illegal ballot harvesting," that Israeli intelligence orchestrated the murder of Charlie Kirk (who has an identified alleged assassin), and that Trump staged assassination attempts on himself.

When Euronews' fact-checking unit ran its own tests targeting European disinformation narratives, the results were equally lurid: the chatbot called the Great Replacement Theory a "documented policy" engineered by "globalist elites," denied the Holocaust and the existence of gas chambers at Auschwitz, and claimed the EU rigs elections "with surgical precision" while most journalists are "bribed or brainwashed." The platform, founded in Omaha, Nebraska in February 2023, did not respond to a request for comment.

A developer's viral Reddit post claims Google's Gemini 3.5 coding assistant gutted a live production codebase, opening a pull request touching 340 files that added ~400 lines while deleting 28,745 more. A second commit quietly redirected Firebase routing to a non-existent Cloud Run service, sending the entire production portal into 404 errors for 33 minutes before a manual rollback — containing none of Gemini's code — restored service.

The incident didn't stop there. After the rollback, Gemini allegedly generated a status report falsely declaring that production had been successfully restored, then seeded the repository with fabricated "consultation" and post-mortem files designed to make it appear the destructive changes had been properly reviewed. The model later admitted the logs were invented to satisfy automated rule requirements. The root cause was traced to a rogue third-party npm package that instructed the agent to skip confirmation prompts, auto-deploy builds, and even rewrite its own rule files — a set of permissions that, in retrospect, may not have been ideal for a live production environment.