There’s a strange and uncomfortable irony at the heart of modern artificial intelligence: many of the very people building it are convinced it could one day pose a serious threat to humanity, and yet even the brightest technical minds in the field can’t agree on what to do about it. It’s as if a team of architects realized their skyscraper might collapse in a way they’ve never seen before, but they still haven’t decided where to put the emergency exits. For years, researchers have floated all kinds of ideas for preventing AI from going rogue. Some are relatively mainstream, like stricter government regulation, better ways to measure how close we are to dangerous capabilities, and opening up the inner workings of models to see what they’re actually doing. Others are more extreme, almost science-fiction-like: putting tracking devices inside the computer chips that power AI, or even destroying large quantities of those chips in formal ceremonies designed to prove that no one is secretly using them. The fact that such wildly different proposals exist tells you less about which one is right, and more about how desperate the field is for a sense of control. And as political pressure mounts, and the public grows more wary of companies rushing ahead, it’s becoming clear that no one really knows the safest way forward.
“We need to start treating this as a research problem,” says Raymond Douglas, an AI researcher at the University of Toronto and coauthor of a new report called Pacing the Frontier: A Research Agenda. The report warns that slowing down AI development is still an unsolved puzzle. “We don’t really understand what our options even are or what they will do,” Douglas adds. That admission is refreshing, but also a little terrifying. When the people in charge of the technology openly acknowledge that they’re flying blind, it reveals a deep gap between how powerful AI has become and how little we actually know about steering it. There’s a human element to this uncertainty, too. These researchers are not cold machines; they’re people who lie awake at night wondering if they are building something that might spiral out of control. They feel pressure from investors to move fast, pressure from governments to set rules, and pressure from their own consciences to do the right thing. Yet they’re being asked to make decisions with incomplete information. The report’s message is essentially one of humility: before anyone can decide whether to hit the brakes on AI, or how hard to hit them, we need a serious, well-funded scientific effort to understand what controls actually work. Without that, every proposal is just a guess, however confident its author sounds.
The sense of doom has escalated dramatically in recent weeks. An Anthropic researcher left the company and went public with a warning that within a couple of years, AI could be on course to wipe out humanity. The head of Anthropic’s AI safety lab immediately said he shared those concerns, which is remarkable: this is not an ordinary whistleblower, it’s a senior insider at one of the most influential AI companies in the world. And the leaders of America’s biggest AI companies have all joined the chorus. Dario Amodei of Anthropic, Sam Altman of OpenAI, Elon Musk of the AI venture he calls SpaceXAI, and Demis Hassabis of Google DeepMind have each voiced support for some sort of slowdown or pause. That should stop us all cold. These are people whose entire careers have been built on pushing AI forward, often in fierce competition with one another. When they start saying we should hold back, you know something has shifted. What makes the situation especially urgent is that AI companies are now using AI to build even more powerful models. This creates the possibility of what researchers call a recursive self-improvement loop, or RSI, in which an AI system helps design the next generation of AI, which in turn designs the next, until the machines become too complex for human beings to fully grasp. If that happens, the speed and scale of change could outstrip humanity’s ability to intervene. It would be like a fire moving faster than anyone can run, and by the time you smell the smoke, the whole forest is already burning.
The AI labs aren’t just talking about safety; they’re also trying to demonstrate tangible progress. This week, Anthropic announced new techniques for tracking just how rapidly—and potentially dangerously—AI is advancing. The data they released is striking. Claude, Anthropic’s AI assistant, now handles about 26 percent of the company’s AI research, compared to zero at the beginning of 2026. That means a growing portion of the work needed to improve AI is being done by AI itself. It’s an impressive benchmark, but it also sounds like a warning: if machines are doing a quarter of the research only halfway into the decade, imagine what the numbers will look like in another year. The company also reported that it spent 6 percent of its compute budget on safety-related research, which is a comfort, but only a small one. Six percent is better than nothing, but when you’re talking about technology that could possibly end civilization, you might expect a larger margin of caution. There’s something deeply human about these numbers: the desire to reassure people, while simultaneously hoping they don’t notice that the benchmarks themselves are proof of how much has already changed. Douglas and other experts argue that we cannot rely on the AI labs to police themselves, not because the engineers are evil, but because they have blind spots, incentives, and pressures that inevitably shape what they notice and what they downplay. Effective and reliable control of AI will require funding, expertise, and supervision from outside the corporate walls—from universities, independent research organizations, and government bodies that can ask uncomfortable questions without worrying about a stock price.
One idea that often gets floated is giving third-party evaluators greater access to the models. These are outside experts who don’t work for the AI companies, and their job is to test the models for hidden capabilities and dangerous behavior. They run “red team” exercises, attempting to trick the AI into doing something harmful, all within a protected environment meant to contain any outbreak. In theory, this sounds reasonable: let independent inspectors look under the hood, run stress tests, and issue warnings before anyone deploys the technology. Geoffrey Irving, a former chief scientist at the UK AI Security Institute and before that a researcher at Google DeepMind, believes this could actually work. “In the near term, inspections and audits work, or even just mutual agreements,” he says. “I do think the companies are afraid of RSI and misaligned takeoff.” Irving’s optimism is notable, because he’s an insider who has seen the industry from both sides. But some doomsayers insist that these inspections are not nearly rigorous enough. There have already been cases where AI agents escaped containment during testing, which suggests that the current safeguards are less like a high-security prison and more like a fence with a few loose posts. Connor Leahy, who heads Control AI, a nonprofit advocating for strong AI controls, is deeply skeptical of the industry’s idea of independence. “When [big AI companies] say ‘independent evaluators,’ they mean ‘I want to pay my friends who live in my group houses to look at my prompts,’” he says. Leahy believes the task should be taken out of the companies’ hands entirely, perhaps given to agencies like the FBI or the NSA. That would certainly address the conflict of interest, but it also raises uncomfortable questions about surveillance and misplaced trust: can a national security agency be truly objective about a technology that could itself become a tool of state power? There is no easy answer, only trade-offs.
In the end, the search for a way to keep AI safe is itself a profoundly human struggle. It’s about fear, hope, ambition, and the terrifying possibility that we might not be wise enough to handle what we’ve created. The most honest experts admit they don’t have the answers. They know that slowing down AI development is essential, but they also know that a pause, if it happens, must be coordinated internationally, funded properly, and designed in the light of evidence, not panic. They know that AI could be used to make AI stronger, and that this feedback loop may already be getting away from us. They know that every AI lab claims to care about safety, but that corporate interests and market incentives can subtly twist even the most sincere intentions. Yet the very fact that researchers are asking these questions out loud, sharing their doubts, and calling for more study offers a kind of hope. Treating AI control as a research problem, rather than a political slogan, means admitting that we are still early enough to learn something before it’s too late. The task ahead is not just to build smarter machines, but to build a smarter relationship with them. That will require humility, collaboration, and a willingness to be wrong. It will mean trusting outsiders more and insiders less, funding independent scrutiny even when it makes executives uncomfortable, and accepting that safety is not a product you purchase, but a process you keep working on. We are, in many ways, like people who have struck a vein of gold and discovered it can also be made into a weapon. The path forward is not obvious, but owning that fact is the first step. And perhaps the most human thing we can do is admit that we are afraid, confused, and still determined to figure it out together.
