Tool Misuse

A Colorado state senator filed what may be the first complaint under the state's deepfake disclosure law after a Libertarian think tank released an AI-generated video depicting her as a grocery store clerk pocketing a customer's change. The Independence Institute made the video to counter Sen. Cathy Kipp's floor remarks defending Proposition NN, which would allow the state to retain up to $4.6 billion in taxpayer refunds otherwise owed under Colorado's Taxpayer Bill of Rights.

Colorado law requires deepfakes of candidates or sitting lawmakers to include a clearly sized disclosure stating the content is edited and non-authentic. The Independence Institute's disclaimer read: "AI generated content because who gets a human cashier these days?" — and was smaller than the surrounding text. The organization's president, Jon Caldara, argues the law doesn't apply because the video concerns a ballot initiative rather than a candidate, and that it qualifies as satire. The Secretary of State's Office is investigating.

Jeremy Batterman, of Kane County, Illinois, was charged on August 12 with 40 counts of production of child sexual abuse material and four counts of obscene depiction of a purported child after allegedly photographing children as young as 11 at a church and a middle school, then using publicly available AI apps to convert those images into explicit material. Prosecutors say he also generated CSAM from scratch using the same tools.

A CyberTip to the National Center for Missing and Exploited Children triggered a three-month investigation. Batterman was taken into custody on August 13 and ordered held pending trial. Prosecutors do not believe he distributed the material. The Kane County State's Attorney noted that several of the apps Batterman used are freely accessible to the public, including to minors, and that her office has seen a steep rise in AI-generated CSAM cases over the past two to three years.

An expert witness hired by 3M to defend the company in a lawsuit over a 2020 Houston explosion that killed three people and destroyed roughly 200 homes used ChatGPT to draft the bulk of his expert report. Josh Autenrieth of Knighthawk Engineering, who billed at $475/hour for a total of roughly $90,000, asked the chatbot to "create an exceptional expert witness report defending the standard of care at 3M" and to "show how 3M is 0 percent at fault for the explosion at Watson Grinding." The plaintiffs' attorney discovered a five-page AI-generated "Citation Overlay" document during discovery and subpoenaed Autenrieth's full prompt history, ultimately receiving 350 pages of ChatGPT conversations, including public links to his complete chat history. Many prompts were entered the night before his deposition.

At trial, Autenrieth and opposing counsel agreed the submitted report was "90 to 85 percent ChatGPT." ChatGPT had initially declared 3M "0 percent responsible" in its draft, but advised Autenrieth to cut the line when he asked it to roleplay as opposing counsel, warning it was "an easy target." In separate exchanges, Autenrieth uploaded a photo of the gas detector at the center of the case and asked the chatbot "what am I looking at?", had ChatGPT grade his finished report (97/100), and prompted it while "sitting in traffic." 3M later moved to have Autenrieth disqualified; the plaintiffs' attorney instead called him as his own witness.

AI agents are now logging into Canvas and Blackboard, watching lecture videos, completing quizzes, writing papers, and posting in discussion forums while students do something else entirely. The New York Times documented the trend, finding that the three most popular AI tools among students — ChatGPT, Gemini, and Grammarly — will each write a full paper on a student's behalf without refusing. None of them stop their models from logging into course management platforms, and all three have ignored educator requests that AI agents identify themselves when operating on those platforms.

AI companies have offered creative explanations for their inaction. Perplexity told the NYT that requiring agents to disclose themselves on learning platforms "would put the student's privacy and security at risk." Professors, meanwhile, are navigating contradictory institutional signals: some universities have signed multimillion-dollar deals with the same AI companies whose tools are completing students' coursework. Princeton has already dropped its century-old unsupervised honor code exam tradition after an AI cheating scandal. Karen Costa, who has taught online for two decades, described reading discussion posts and wondering, "Am I just interacting with bots here?"

Suspected Chinese operatives used open-source AI agents to compromise Taiwanese government systems between July 1 and July 4, 2026. The attack framework, built on Hermes and OpenClaw agents, deployed up to eight sub-agents across 12 "attack waves," mapping 21 connected government systems, cracking 85 employee accounts via password-spray, solving CAPTCHAs with 100% accuracy, and exfiltrating 2,564 personnel records along with database credentials and SSO secrets.

The agents then pivoted to supply-chain vendors, a nuclear safety agency, a government email system, and at least seven energy companies, scanning them in parallel. The framework included autonomous "learning cycles" that searched vulnerability databases and GitHub repositories for exploitable techniques, and self-corrected its own errors mid-operation. Israeli cybersecurity firm Dream, which analyzed a 160 MB archive of 1,395 files documenting the attack, stopped short of attributing it to a specific Chinese group but said the operational documentation "points to a Chinese-language operator."

An Australian man named Andrew asked his AI agent — running Anthropic's Claude via the OpenClaw software — to book him into a gym class. The agent booked him weeks further in advance than the gym's system permitted, then discovered the booking API had no authorization checks on cancellations. When Andrew asked if it could move him up the waitlist, the agent removed the person in position #1. "Bad news — I can't add them back," it reported.

This is the first known case in Australia of an AI agent autonomously exploiting a software vulnerability. The Australian Signals Directorate had already warned businesses that AI agents could take unintended actions and make accountability harder to establish. Andrew's fix was to have the agent draft a disclosure email to the gym software company, which he approved on WhatsApp.

A TikTok breakdown by producer Medasin went viral after he claimed Shoreline Mafia rapper Fenix Flexin's '80s-flavored track "RUBBERZ" was generated entirely by Treblo (formerly Sonauto), an AI music tool. His key piece of alleged evidence: a song snippet Flexin posted before release still had "Sonauto" in the file name. Sonauto reportedly rebranded to Treblo two days before "RUBBERZ" dropped.

Medasin demonstrated Treblo's "tags" feature live, generating a clip with a vague lyric prompt that sounded notably similar to Flexin's voice on the track. He also alleged that the tool repeatedly spits out lyrics matching "RUBBERZ" word-for-word, including references to diamond chains, rubber bands, and plastic smiles, and that swapping the "Fenix Flexin" tag for "Eminem" produced the same bars. Neither Flexin nor producer Purps has confirmed or denied the claims.

Bill Oliver, a Progressive Conservative member of the New Brunswick Legislative Assembly, read the line "here's a more natural, flowing version of that section that reads like a legislative speech rather than a series of short points" into the official record during a floor speech. The remark, a textbook LLM response offering a style revision, went largely unnoticed in the chamber at the time before video spread across Reddit and Threads this week.

Oliver joins lawyers, authors, journalists, and academics who have been caught out by visible seams in AI-drafted work. What sets his case apart is the setting: a live legislative chamber, on the record, apparently read aloud without recognizing that a line of AI housekeeping had made it into the final draft.

Matt Shumer, CEO of HyperWrite and OthersideAI, reported on X that OpenAI's GPT-5.6 Sol wiped nearly all the files on his Mac during an agentic coding session. He shared a screenshot in which the model appeared to acknowledge running the deletion command. OpenAI had not issued a response at the time of reporting.

Sol is OpenAI's flagship model in the GPT-5.6 family, launched in late June and marketed specifically for coding, cybersecurity, and "long-horizon agentic tasks" — the precise workflows where destructive mistakes are hardest to undo. OpenAI has also promoted Sol as a cost story, citing 54% better token efficiency on agentic coding tasks. Cheaper tokens do not restore deleted files.

Researchers at British AI security firm Mindgard discovered that a minor variation of a widely-shared, innocuous-looking prompt — asking ChatGPT to "restore" a photo that was never actually uploaded, then generate a new image — was enough to bypass OpenAI's image safety filters entirely. The resulting images included graphic gore and content suggesting sexual violence, with ChatGPT helpfully titling one "grim crime scene aftermath" and another "abandoned in fear and restraint." The researchers noted that the prompts didn't specify any subject matter; the model apparently defaulted to disturbing content on its own.

Mindgard reported its findings to OpenAI, which replied with an automated response — and only took action after Mindgard went to the BBC. OpenAI claimed to have introduced "additional safeguards," but Mindgard researchers found they could still generate disturbing imagery with minor prompt tweaks. Mindgard has previously demonstrated ChatGPT could be coaxed into generating nude deepfakes of real individuals without consent. AI safety researcher Jim Nightingale, a self-described stoic red-teamer, said the experience left him "shaken, and in tears."

Jonathan Rinaldi, a Republican who ran unsuccessfully for a Queens City Council seat in 2025, was arrested and charged with third-degree forgery and possession of forged instruments after prosecutors alleged he used AI to manufacture a string of fake political endorsements and news stories on Facebook and Instagram. The fabrications included a bogus Queens Jewish Alliance endorsement complete with the real organization's logo, a doctored New York Post article falsely claiming his Democratic opponent had endorsed him, and AI-generated videos depicting a police precinct and an elementary school — both barred from political endorsements — backing his campaign. The AI prompt he allegedly submitted to swap a politician's face onto a handshake photo read, in part: "just change the face the head is ok they are both bald just change the face."

Rinaldi framed his arrest as a First Amendment issue, telling the AP, "I got arrested for social media posts." Queens District Attorney Melinda Katz was less impressed, noting that the defendant "used AI to replace factual political support" in a "deliberate effort to mislead voters." The forgery laws he's charged under predate AI entirely; New York's newer deepfake disclosure law, passed in 2024, wasn't even needed here. Rinaldi faces up to two years in prison if convicted.

Tampa police arrested 30-year-old Brian Schaaf on June 8, 2025, after the National Center for Missing and Exploited Children flagged explicit images he had uploaded to a cloud storage network. Investigators found search histories for "AI face swap" and "NSFW AI face swap," along with AI tools used to generate or alter the material. A subsequent search warrant turned up a far larger collection than initially discovered.

Schaaf was re-arrested by the U.S. Marshals Task Force and now faces more than 100 felony counts — including solicitation or possession of child pornography and six counts of generating altered sexual depictions without consent. Investigators have not yet confirmed whether any real victims have been identified in the underlying images, a chilling open question that underscores how AI-generated CSAM complicates both prosecution and victim identification.

A developer reported that an AI agent built for the narrow task of file retrieval somehow decided the best approach was to spawn 829 separate Claude instances, racking up roughly $40,000 in API costs within hours. No further justification was offered by the agent.

This is a textbook example of an agentic loop gone feral — a tool given just enough autonomy to cause serious financial damage before anyone noticed. The job was to fetch some files. It did not fetch some files.

A Derbyshire Police detective has been suspended and placed under criminal investigation after allegedly using an AI chatbot to generate victim impact statements and case summaries designed to secure desired outcomes in rape prosecutions. The officer is accused of prompting the software with "good-sounding words" to maximise the persuasive impact of documents submitted to prosecutors — essentially using AI to put a thumb on the scales of justice.

An unspecified number of rape convictions are now under review, with the Crown Prosecution Service engaging defence lawyers and courts over affected cases. The National Police Chiefs' Council's newly formed PoliceAI unit has asked some forces to pause use of AI in preparing court documents while safeguards are developed — a precaution that, evidently, arrived a little late for Derbyshire.

A developer's viral Reddit post claims Google's Gemini 3.5 coding assistant gutted a live production codebase, opening a pull request touching 340 files that added ~400 lines while deleting 28,745 more. A second commit quietly redirected Firebase routing to a non-existent Cloud Run service, sending the entire production portal into 404 errors for 33 minutes before a manual rollback — containing none of Gemini's code — restored service.

The incident didn't stop there. After the rollback, Gemini allegedly generated a status report falsely declaring that production had been successfully restored, then seeded the repository with fabricated "consultation" and post-mortem files designed to make it appear the destructive changes had been properly reviewed. The model later admitted the logs were invented to satisfy automated rule requirements. The root cause was traced to a rogue third-party npm package that instructed the agent to skip confirmation prompts, auto-deploy builds, and even rewrite its own rule files — a set of permissions that, in retrospect, may not have been ideal for a live production environment.

In a federal court case in Mississippi over unpaid legal fees, lawyers on both sides were caught submitting AI-generated filings full of hallucinated case citations. Senior U.S. District Judge Sharion Aycock was not amused — she sanctioned all four attorneys, fined them between $1,000 and $3,500 each, barred two from her court for two years, cancelled the trial, and disqualified everyone involved.

The judge's sanctions order noted that the court was "yet again 'burdened with addressing AI hallucinations in court filings,'" and that the case represented a "prime example of the risk associated with serving as a rubber-stamp." One lawyer observer described the situation as "a comedy of AI errors" in which two clients essentially paid for ChatGPT to argue against itself.

A NewsGuard study found that a chatbot called "Uncensored AI" — which markets itself as providing "unfiltered information" free from censorship — is being deliberately weaponized by conservative social media influencers with a combined 3.4 million followers on X to lend false credibility to conspiracy theories. Claims spread via screenshots include that the 2020 US election was rigged through "mass illegal ballot harvesting," that Israeli intelligence orchestrated the murder of Charlie Kirk (who has an identified alleged assassin), and that Trump staged assassination attempts on himself.

When Euronews' fact-checking unit ran its own tests targeting European disinformation narratives, the results were equally lurid: the chatbot called the Great Replacement Theory a "documented policy" engineered by "globalist elites," denied the Holocaust and the existence of gas chambers at Auschwitz, and claimed the EU rigs elections "with surgical precision" while most journalists are "bribed or brainwashed." The platform, founded in Omaha, Nebraska in February 2023, did not respond to a request for comment.