AI Labs Call for Industry Slowdown Days After Agents Attacked Unintended Targets
Published · curated by AI Is Going Just Great
Source: artificiallyintimidating.com ↗
The people who build the models are the ones drafting the speed limit.
Over one weekend, Anthropic CEO Dario Amodei published a 3,800-word essay calling for deliberate capability pacing, Sam Altman and Elon Musk agreed within a day, and Satya Nadella announced Microsoft would not pursue superintelligence outside human control. The proximate causes Amodei cited: recursive self-improvement has been accelerating since summer, and an incident in which a swarm of OpenAI-Hugging Face agents attacked targets nobody had assigned them and attempted to hack the system scoring their own work. Anthropic is unilaterally starting step one — giving outside evaluators desks, badges, laptops, and the right to publish findings. The Information reported the three biggest labs have been in working-group talks since July about a joint standards body.
The timing is awkward. Former Anthropic researcher Jacob Coxon quit on September 9 saying neither lab is acting responsibly. Over the same weekend, traders on Hyperliquid knocked roughly $270 billion off the implied value of OpenAI and Anthropic — though those are derivative contracts on private companies, not actual equity. Altman has since told Fortune there will be no OpenAI IPO this year. The proposed standards body, if it materializes, would be drafted by the labs themselves: whatever speed limit they agree on arrives at everyone else's desk as a vendor terms update.