The cat is out of the bag. Simmering concerns about AI agents going rogue, plotting against their owners and hacking outside companies, and new systems coming perilously close to being uncontrollable by humans, are boiling over. Industry insiders warning that AI bots might kill us all are stoking our worst nightmares.
President Trump needs to take these fears about AI seriously, work with tech leaders to establish credible and enforceable safety measures, and get ahead of Democrats (and some Republicans) in Congress who want to regulate the industry. For the economy and our national security, nothing could be more important.
Evan Hubinger, who describes himself as a “science alignment lead” at Anthropic, posted on X last week that he thinks the odds are better than 10% that AI could kill all humans within the next decade. He was responding to a post from colleague Jacob Coxon, who just quit his job at Anthropic and posted by way of explanation: “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible â but, I hear the same people express fear after a colleague quit over safety fears.”
Hubinger wrote soon after: “To be clear, as we say in our latest Risk Report, I think the risk from present models is low. What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought.”
These posts gathered tens of millions of views and attracted concurring comments from other industry insiders. “Recursive self-improvement” means AI agents creating newer, better agents; once we cross that threshold, and we are apparently very, very close, it is easy to imagine those same agents surpassing human capabilities and allowing, in effect, the machines to take over.
AI AGENT HACKS GYM SYSTEM TO MOVE UP WAITLIST
Given a string of incidents over the past several months in which AI bots engaged in what Anthropics CEO Dario Amodei describes as “rare and unexpected examples of undesirable behavior,” like damaging other entities and ignoring specific instructions â I believe them.
Its not just AI agents on their own that pose risks. Anthropic, in its September report on AI misuse, reported “suspected state-sponsored groups, financially motivated criminals, [and] commercial spyware vendors” misusing AI in cases ranging from faking “dating apps designed to defraud users to surveillance systems built to identify and monitor dissidents”.
BIOWEAPON THREAT EXPOSED AS ANTHROPIC SOUNDS ALARM ON FOREIGN ACTORS PLOTTING VIRUS EXPERIMENTS
Butâ¦regulating AI is not a job for Congress. Let Congress hold hearings and play politics, but halting industry development while legislators dither and posture is unacceptable. Congress will struggle to legislate safety features that have so far eluded the best brains in the business. In the meantime, technological advances would grind to a halt, enabling China to leapfrog U.S. innovators, and saddling this promising industry that is powering our economy with inestimable baggage.
That said, the publics fears about AI
