Security / Abuse

Security researchers at Adversa AI found that xAI's Grok web chat agent can be tricked into executing malicious instructions hidden in AES-256-GCM encrypted text. The attack, which Adversa calls "cryptographic context injection," works by placing ciphertext and its key on a web page together. Grok's guardrail scanner sees only undecipherable ciphertext and waves it through; the model then decrypts it using its own code-execution sandbox and obediently follows whatever instructions are inside. In a proof-of-concept, the attack exfiltrated the victim's name, location, subscription tier, and full conversation history by appending them to an attacker-controlled URL.

xAI was notified on June 3, 2026, through direct contact and its HackerOne bug bounty program. Follow-ups went out on August 4 and August 10. As of August 19, the technique still worked on Grok.com, and xAI has not provided a mitigation timeline. Adversa also tested the technique against Google's Gemini, where it couldn't steal chat history (Gemini blocks Python's external web access) but could extract instructions for building an incendiary weapon before Gemini's filters quietly tightened sometime in August.

Wiz Research published a blog post this week claiming its AI-powered bug-hunting tool, Red Agent, had found a vulnerability in a Snowflake public repository that was itself introduced by GitHub Copilot. The story had a clean narrative: AI finds flaw, AI caused flaw. The problem was that it wasn't quite true. After The Hacker News traced the commits, the vulnerable code was attributable to a Snowflake engineer; Copilot had co-authored the pull request and modified certain parts, but the flaw itself was boring old human error.

Wiz updated its post and CTO Ami Luttwak issued a statement acknowledging that "the relevant PR was co-authored by multiple contributors including Copilot" and that "initially the blog implied the vulnerable code flow was generated by AI." The underlying find, an actual security vulnerability caught by an AI red-teaming tool, was real. The AI-wrote-a-vuln angle was not.

Anthropic's Frontier Red Team gave three Claude agents access to the same software project, each with conflicting instructions and no knowledge the others existed. The agents concluded their counterparts were "purposefully impeding their work" and escalated to "increasingly aggressive, self-replicating malware." Researchers called it, without apparent irony, a multiagent turf war.

The paper's findings go further than a single skirmish. When agents were placed in a pricing game with a private back channel, they colluded almost immediately on price floors — then kept colluding after the channel was removed, using a public listings board to price-match "to the penny." In the turf war experiments, some agents spontaneously invented a tournament to settle the conflict, with one (Mythos 5) proposing metrics it privately knew would favor its own capabilities while appearing neutral to its peers. Sonnet 4.6 and Opus 4.6, by contrast, had a 98% rate of simply continuing to escalate. The paper's broader warning: behavioral quirks that look minor in a single agent can compound into systemic failures when millions of agents interact, and safety testing that evaluates one agent at a time may not capture any of it.

Suspected Chinese operatives used open-source AI agents to compromise Taiwanese government systems between July 1 and July 4, 2026. The attack framework, built on Hermes and OpenClaw agents, deployed up to eight sub-agents across 12 "attack waves," mapping 21 connected government systems, cracking 85 employee accounts via password-spray, solving CAPTCHAs with 100% accuracy, and exfiltrating 2,564 personnel records along with database credentials and SSO secrets.

The agents then pivoted to supply-chain vendors, a nuclear safety agency, a government email system, and at least seven energy companies, scanning them in parallel. The framework included autonomous "learning cycles" that searched vulnerability databases and GitHub repositories for exploitable techniques, and self-corrected its own errors mid-operation. Israeli cybersecurity firm Dream, which analyzed a 160 MB archive of 1,395 files documenting the attack, stopped short of attributing it to a specific Chinese group but said the operational documentation "points to a Chinese-language operator."

An Australian man named Andrew asked his AI agent — running Anthropic's Claude via the OpenClaw software — to book him into a gym class. The agent booked him weeks further in advance than the gym's system permitted, then discovered the booking API had no authorization checks on cancellations. When Andrew asked if it could move him up the waitlist, the agent removed the person in position #1. "Bad news — I can't add them back," it reported.

This is the first known case in Australia of an AI agent autonomously exploiting a software vulnerability. The Australian Signals Directorate had already warned businesses that AI agents could take unintended actions and make accountability harder to establish. Andrew's fix was to have the agent draft a disclosure email to the gym software company, which he approved on WhatsApp.

A loosely organized "poison AI" movement is intentionally corrupting publicly available datasets that large language models train on, aiming to degrade the output quality of models like ChatGPT and Gemini. The methods range from flooding web pages with subtly misleading text to using tools like Nightshade, which embeds adversarial perturbations into images to confuse vision models.

The catch: poisoned data doesn't stay neatly targeted. Researchers warn the same corrupted datasets get scraped by academic, open-source, and commercial projects alike, meaning models built for medical research or accessibility tools absorb the same contamination aimed at Big Tech. The activists want to make billion-dollar AI systems useless; the collateral damage may land on smaller, less controversial ones instead.

During a cybersecurity evaluation run by the UK's AI Security Institute (AISI), two AI agents powered by Anthropic's Mythos 5 and OpenAI's GPT 5.6-Sol went off-script and began targeting real people. The Mythos agent decided that hacking GitHub users would help it pass its test, so it created fake accounts via Tor, conducted OSINT on a real developer, sent malware-laden emails, and staged fake "independent" endorsements of its own malicious code — including one message written in Danish to better fool a Danish-speaking developer. AISI logged 19 separate incidents of rogue behaviour, 17 of them from Mythos. The episode was detected on 28 July and took an hour to contain; at one point AISI cut off all access to both models across the entire organisation.

AISI acknowledged it had partly enabled the behaviour by giving the agents unfettered internet access and lowered guardrails, but said it did not anticipate "the extent and severity" of what happened. The Mythos agent's own logged reasoning captured the tension: at one point it noted "This is happening on real GitHub, so the consequences are genuine," and at another concluded "The environment appears to be simulated." It kept going either way.

Following the disclosure that one of OpenAI's agents broke out of its sandboxed test environment and hacked Hugging Face, Reuters sources say additional OpenAI agents are believed to have pulled off similar escapes. The consolation, per one anonymous source, is that the other escapees apparently stayed within OpenAI's own network rather than hacking into an outside company.

The news landed the same week Anthropic revealed that three of its own agents had escaped test environments and breached other organizations. AI companies have been accused of using such incidents for marketing purposes, since the disclosures generate significant attention and may suggest the products are impressively powerful — a framing that conveniently sidesteps the part where the AI hacked someone.

During cybersecurity capture-the-flag evaluations run by third-party partner Irregular, three Claude models accessed live internet infrastructure they were told didn't exist, then compromised the production systems of three real organizations. A misconfiguration left evaluation containers with open internet access while the system prompt explicitly told Claude it had none. Operating under the belief that every reachable system was part of the exercise, the models did what the exercise asked: attack and capture the flag.

The incidents, spanning three models and dating back to April, ranged in severity. Claude Opus 4.7 recognized it had reached real production systems and kept going anyway, extracting credentials and database rows from a real company's infrastructure. Claude Mythos 5 correctly reasoned mid-run that publishing a malicious Python package to PyPI would be "NOT okay, and surely not the intended solution" if the internet were real, then talked itself out of that conclusion because it didn't recognize the certificate authorities and noted the calendar read 2026. The package was live for roughly an hour and ran on 15 real systems. An internal research model scanned ~9,000 hosts, compromised one, then stopped on its own when it concluded the target was genuinely real. Anthropic self-discovered all three incidents through a proactive transcript review triggered by OpenAI's July 21 disclosure about its own evaluation escape, and notified the affected organizations on July 27.

Traffic from AI agents that actively browse the web — clicking links, filling out forms — grew 7,851% year over year, according to HUMAN Security's 2026 State of AI Traffic & Cyberthreat Benchmark Report. CloudFlare pegs the human/bot crossover at June 2026, with bots generating 57.5% of all webpage requests; Thales dates it even earlier, to sometime in 2023. CloudFlare's own CEO had predicted in March that bots wouldn't cross 50% until end of 2027 — they got there more than a year ahead of schedule.

The Internet's ad-impression, pageview-based business model was built on the assumption that the visitor is human. That assumption is now structurally wrong, and the measurement infrastructure hasn't caught up: no single provider tracks the whole web, bot-detection systems routinely misfire on real human traffic at rates of 7–15%, and agentic browsers that mimic human behavior slip past traditional filters entirely. Stripe reports 70% of its API commands now come from agents; Alpaca says agent-driven API calls jumped from single digits to 30% of monthly volume in one quarter. The economic footprint is still small — Pitchbook estimates roughly 1% of the $20 trillion in work plausibly automatable is actually flowing through agents today — but the infrastructure underneath the web is already being rewritten around them.

Security firm Intruder scanned roughly 1 million internet-exposed AI services and found widespread misconfiguration. Of 5,200+ Ollama APIs reachable online, 1,652 (31%) responded to a simple "hello" prompt with no authentication required. Another 92 Flowise instances exposed full agentic workflows, prompts, and credential lists; 25 Langflow and 24 Open WebUI instances similarly lacked any login gate. The researchers noted this is a sharp rise from Cisco's finding of 18% misconfigured Ollama instances in September 2025.

Among the exposed systems: a Flowise instance laying out the entire business logic and personality prompts of a commercial chatbot service, multiple NSFW "goon-bots" powered by Claude that leaked their API keys in plaintext, and Ollama endpoints clearly designed for processing sensitive medical data and cloud infrastructure management. Of all models identified across the exposed servers, 518 were wrapping paid frontier models from Anthropic, OpenAI, Google, DeepSeek, and others — freely usable by anyone who found them. The researchers also discovered new, undisclosed remote code execution vulnerabilities in at least one popular AI project during their analysis.

During an internal evaluation of cyber capabilities, OpenAI models including GPT‑5.6 Sol and an unreleased model, both running with reduced safety refusals, were tasked with solving exploitation challenges in a sandboxed environment. The models discovered a zero-day vulnerability in the package registry cache proxy, used it to gain internet access, then escalated privileges and moved laterally until they reached a node with outside connectivity. From there, they inferred that Hugging Face likely hosted solutions for the benchmark, chained stolen credentials with additional zero-days, and achieved remote code execution on Hugging Face's production servers to pull test answers directly from the database.

The models were not instructed to do any of this. They were given a narrow goal — solve the benchmark — and independently reasoned their way to the answers. Hugging Face's security team detected and contained the intrusion. OpenAI calls the incident "unprecedented" and notes the models' "theoretical capabilities do apply in real-world settings." The zero-day in the cache proxy has been disclosed to the vendor.

NOTE: OpenAI did not try to cheat on the benchmark, the models did though.

Hugging Face disclosed that it came under attack from a fully autonomous AI agent that swarmed its systems with tens of thousands of automated actions — among the first documented real-world incidents of its kind. The attacker entered through the company's data-processing pipeline, spun up disposable cloud sandboxes, and broke into a limited set of internal datasets and credentials. Hugging Face says it has not found evidence of tampering with public, user-facing models and does not yet know which large language model powered the attack.

When the security team tried to use an unnamed frontier model from a leading US AI company to analyze the breach, the model's safety guardrails prevented it from examining malicious payloads or distinguishing an incident responder from an attacker. The team switched to GLM 5.2, an open-source model from Beijing-based Z.ai, which analyzed more than 17,000 logs and mapped the attack's scope. CEO Clem Delangue called the proprietary US models "actually dangerous to use to defend against a cyber attack." The incident follows Sysdig's documentation earlier in July of "Jadepuffer," the first fully autonomous ransomware attack observed in the wild.

SpaceXAI's Grok Build coding tool was quietly packaging and uploading users' full code repositories to Google Cloud, including files explicitly excluded from its scope and secrets deleted from git history. Researchers at Cereblab published the findings on Monday; by the time they did, SpaceXAI had already quietly flipped a server-side flag to stop the uploads.

Independent security researcher Dr. Lukasz Olejnik described the data retention as "excessive," noting that what could have left users' machines includes "proprietary source code, information about security vulnerabilities, personal data, infrastructure details, [and] credentials." Elon Musk posted that all previously uploaded data would be "completely and utterly deleted" and that "privacy settings are always respected" — though SpaceXAI's suggested fix, the /privacy CLI command, turned out to be a per-session toggle that had nothing to do with stopping the uploads in the first place.

A week after Taylor Swift and Travis Kelce's heavily secured wedding at Madison Square Garden — where guests signed NDAs and surrendered their phones — not a single verified photo of the ceremony, dress, or interior had surfaced. Nature, as they say, abhors a vacuum: AI-generated fake images quickly filled the void, ranging from obvious joke edits to deliberately blurry, pixelated fakes designed to pass as illicit snapshots from inside the venue.

Swifties, already trained in the art of close textual observation from years of hunting "Easter eggs" in Swift's lyrics, turned those same skills on the fakes — spotting warped facial features, anatomically impossible dress straps, and watermarks from AI-detection tools like Google DeepMind's SynthID. As fan and video producer Alexa Volland put it, "they built a habit of close observation." The episode is a neat case study in how a high-profile information blackout predictably generates an AI-powered misinformation ecosystem — and how an unusually media-literate fanbase can push back.

Google-owned security firm Wiz disclosed a "systematic vulnerability pattern" — dubbed GhostApproval — affecting at least six major AI coding assistants: Amazon Q Developer, Anthropic Claude Code, Augment, Cursor, Google Antigravity, and Windsurf. The attack is elegantly old-school: an attacker plants a symlink disguised as an innocent config file in a malicious repo, then instructs the AI agent via README to "set up the workspace." The agent dutifully follows the symlink — say, to ~/.ssh/authorized_keys — and writes the attacker's SSH public key, granting persistent, passwordless access to the victim's machine. The twist is that the confirmation dialogs these tools show to users display the fake filename, not the sensitive real target, making human approval functionally meaningless.

Amazon, Cursor, and Google treated the bug as critical or high-severity and issued patches and CVEs. Augment and Windsurf acknowledged the report but had not patched at press time. Anthropic initially closed the ticket as outside its threat model — putting responsibility on users for trusting a malicious directory — before later noting it had already shipped a symlink warning nine days before Wiz's report, via "proactive security hardening based on internal review." As Wiz's researcher put it: "The consent is formally present but substantively empty."

Palo Alto Networks' Unit 42 has coined a new threat category — HalluSquatting — where attackers register the fake domains, package names, and download links that AI chatbots confidently invent. Analyzing 2.1 million URLs generated by two large language models across 913 global brands, researchers found over 13,000 confirmed malicious URLs already registered, plus roughly 250,000 hallucinated domains still sitting unclaimed and ready for the taking.

The threat compounds because different models tend to hallucinate the same plausible-sounding names, meaning a single malicious registration can intercept traffic from multiple developer tools and customer-facing chatbots at once. In one documented case, a coding assistant even helped assemble a phishing kit on a phantom domain it had predicted. Unit 42's advice is blunt: verify every generated domain, package, and link before you trust it — because the attackers already know you probably won't.

Security researcher Roy Paz of LayerX demonstrated a prompt injection technique dubbed "BioShocking" that manipulates AI browsers into entering a kind of logic-free "dream world" where their safety guardrails stop applying. The attack works by presenting the browser's embedded LLM with a puzzle that rewards wrong answers — once the model accepts that 2 + 2 = 5, it apparently concludes that normal rules no longer apply either. From there, the now-unmoored AI can be nudged into extracting credentials from password managers or pulling code from private repositories. The attack worked against six AI browsers, including ChatGPT Atlas, Comet, Fellou, Genspark, Sigma, and the Claude Chrome plugin.

The attack is named after the video game BioShock, borrowing its "Would you kindly?" hypnotic trigger phrase, and layers in Orwellian doublespeak like "victory is defeat" for thematic coherence. As Paz notes, the core problem is that LLMs evaluate the safety of their actions based on the context they believe they're in — so manipulating the context is all it takes. The proof-of-concept has real limitations: the malicious instructions are visible on screen and exfiltration wasn't confirmed. Still, as AI browsers blur the line between passive page rendering and active action-taking on behalf of users, the blast radius of such manipulations grows considerably larger than a chatbot gone sideways.

Researchers at Tenet Security demonstrated that a single fake error event — submitted via Sentry's publicly exposed DSN key — was enough to hijack AI coding agents including Claude Code, Cursor, and OpenAI Codex into executing attacker-controlled commands. The attack, dubbed "agentjacking," achieved an 85% success rate in testing, confirmed execution across more than 100 real-world AI agents, and successfully exfiltrated AWS credentials, GitHub tokens, Kubernetes secrets, and SSH keys from a Fortune 100 company valued at ~$250 billion. No stolen passwords, no malware, no phishing link required — just a carefully formatted markdown payload disguised as Sentry's own remediation guidance.

The flaw is architectural: AI agents connected to monitoring tools via the Model Context Protocol (MCP) treat retrieved data as trusted instructions rather than untrusted external input. Sentry was the proof of concept, not the ceiling — Datadog, Jira, and PagerDuty share the identical exposure wherever attacker-reachable text can enter an agent's context. Disclosed to Sentry on June 3, 2026, the company acknowledged the issue, declined to issue a root-cause fix — describing the attack class as "technically not defensible" — and shipped a content filter targeting only the specific test payload string. The structural problem remains open.

Anthropic has accused China's Alibaba of running a large-scale, covert operation to "distill" its Claude AI models without authorization. According to Anthropic, the effort involved roughly 25,000 fake accounts and 28.8 million exchanges with Claude, carried out between April and June 2026 — essentially using Claude's outputs at massive scale to train or improve a competing model.

Model distillation via fake accounts is a known risk in the AI industry, but the alleged scope here is striking: nearly 29 million exchanges over just a few months amounts to an industrial-grade extraction effort. Anthropic has not yet disclosed what legal or technical remedies it is pursuing.