Let me paint the scene: a cavernous conference center in San Francisco, thousands of people wearing lanyards and clutching coffee cups, the kind of crowd that makes you think of a rock concert, a political rally, or maybe a church service. And there, onstage, was Gwen Stefani—purple and plaid, all in, belting her 2002 hit “Underneath It All” to a packed auditorium. She was only the warm-up. When she finished, she shouted “Dreamforce! Let’s go” into a checkered microphone, and the room erupted. Then Marc Benioff, the founder and CEO of Salesforce, walked into the spotlight, and the energy shifted from pop concert to something more devotional. Dreamforce is officially billed as the world’s largest enterprise software conference, but it has always felt less like a trade show and more like a megachurch. This year, with artificial intelligence dominating every conversation, it felt even more like a revival. Benioff roamed the aisles while delivering his keynote, trailed by a teleoperated camera on wheels that beamed his face onto massive screens hanging from the ceiling. He spoke with the rhythm of a preacher who knows his congregation well. Within minutes, he called up a graph of Salesforce’s projected annual revenue, which the company says will top $46 billion by 2027, thanks in part to demand for its AI-powered products. The audience applauded as if the number itself were a promise of salvation. But underneath all the confidence, a darker question hung over the event, one that no graph or keynote could answer: Is artificial intelligence really worth risking all our lives for? The music was loud, the lights were hot, and yet the room felt uneasy, like a celebration happening on the edge of a cliff.
Between the flashy software demonstrations—AI agents building dashboards, automating customer service, generating business plans on command—the real conversation at Dreamforce was about whether humanity is moving too fast. The debate had been brewing for a while, but it reached a new pitch only days before the conference opened. Jacob Coxon, an AI researcher at Anthropic, resigned and publicly warned that companies racing toward self-improving AI systems were “gambling with our lives.” His words rippled through the industry in a way that felt personal. By the weekend, other Anthropic employees were echoing his concerns. Then Anthropic’s CEO, Dario Amodei, made the unusual move of urging world leaders to help “pace the frontier” of AI development—not to slam on the brakes, but to create enough time for safety measures to catch up with ambition. Even Sam Altman, the CEO of OpenAI and Amodei’s most prominent rival, publicly endorsed the idea. That should have been a headline. But Washington was in a completely different mood. David Sacks, a White House AI adviser, dismissed the whole wave of anxiety as “just another bid for regulatory capture,” suggesting that the people warning about AI doom were simply trying to use fear to enrich themselves. President Trump went even further, calling fears about existential AI risk “a hoax.” And yet Congress, sensing that the political winds might shift, is trying to push ahead with legislation to regulate the AI industry. It is a strange, confusing moment: the people building the technology are asking for limits, while many of the people making policy are telling them to keep going.
That tension was on full display when Benioff brought Amodei onstage. Benioff, six-foot-five and radiating the confidence of a man who owns the room, towered over the Anthropic CEO and shook his hand with the force of a business deal. It was impossible not to notice the difference between them. Benioff is a salesman; Amodei is more like a nervous intellectual, a man who has seen too much to be entirely comfortable with what he is building. But the pairing was not accidental. Salesforce has hitched its future to AI, and Anthropic has become one of its most important partners in that journey. So when Benioff asked Amodei for his honest assessment of where the technology is heading, the answer was carefully crafted, the sort of response you give when you want to be honest without being reckless. Amodei told a story. “Let’s say you’re running a car company and another car company, not yours, has some kind of safety incident,” he began—a not-so-subtle nod to the safety failures already happening in the AI industry. “Obviously, it’s very tempting to attack your competitor.” And everyone in the room understood: the temptation to weaponize a rival’s mistake is almost irresistible. But Amodei argued that the more responsible move is to “organize the rest of the industry and say, what can we do to set standards for everyone?” If one player crashes, the entire industry suffers. That, in a nutshell, is why he kept saying that the frontier needs pacing. He was careful, though, to clarify that pacing does not mean freezing the technology in place. Innovation must continue. And, because no keynote moment is wasted, he took a moment to remind everyone that Claude, Anthropic’s AI model, is deeply integrated with Salesforce’s products. It was a delicate dance: sounding the alarm without sounding like an alarmist, being a prophet and a business partner at the same time.
Then, as if to reset the mood, Nvidia CEO Jensen Huang bounded onto the stage. Huang is the hardware king of the AI boom, and he radiated the confidence of a man who believes every problem is solvable with enough engineering. AI safety, he insisted, is simply an “engineering problem.” You fix it the way you fix any technical problem: you notice what is broken, you repair it, and you move on. He waved away the apocalyptic talk that had been hanging over the conference like a storm cloud. “We don’t need any new laws, we don’t need any new regulation,” Huang told the crowd. “You run as fast as you can, but if you feel that at any given point in time, the company is out of control or the products are not going to be safe, you take a pause.” It sounded reassuring in its simplicity. It also placed enormous faith in the very people who are competing to build the fastest, most powerful AI systems. The same companies racing to dominate the market are also expected to police themselves when things get dangerous. The same leaders who are rewarded for speed are now being asked to stop and reflect. For Huang, the future is a technical challenge: make chips faster, make models smarter, make safety a feature that can be engineered into the system. For Amodei, the future is a moral challenge: what are we willing to risk, and who gets to decide? The two visions are fundamentally incompatible, yet both men speak with the same sincerity. As Huang left the stage, the applause was warm, but the unease in the room had not disappeared. It had only changed shape.
The debate felt heavy for a conference decorated with plush mascots and vaguely Hawaiian flourishes. But maybe that was exactly the point. Dreamforce is where enterprise technology comes to be celebrated, where billion-dollar software deals are made, where the future is packaged and sold to the rest of us. It is the perfect place to ask whether the future we are buying is actually worth the price. In the middle of the afternoon, with executives still pitching their AI tools onstage, I slipped out of the main hall. I passed the mascots, the cheerful volunteers, the rows of startup booths with their optimistic slogans. I needed to step back from the marketing and clear my head. I had promised to take a call from Sayash Kapoor, an incoming computer science professor at Berkeley who has spent years studying the gap between AI hype and AI reality. The conversation was grounding, the way a glass of cold water is grounding after too much sugar. In all the talk about existential risk and distant apocalypses, it was easy to forget that AI is already making important decisions in real lives: who gets a loan, who gets hired, who gets trusted with a medical recommendation. The dangers are not always dramatic. They can be slow, invisible, and systemic—a biased model making unfair choices, a chatbot inventing facts when no human is watching, a customer service system so impenetrable that vulnerable people can no longer reach another human being. The real question is not simply whether AI will doom us all. It is whether we are willing to hand over more and more of our lives to systems that we do not fully understand and cannot fully control.
After I hung up, the auditorium was still cheering. Another product had been launched. Another partnership had been announced. Another future had been sold. But the question that had been hanging over Dreamforce all day was still unresolved. Is artificial intelligence worth risking our lives for? No one on that stage could answer it, not with a revenue graph, not with a sermon, not with an engineering analogy. The singer was the warm-up. The CEOs were the main act. But beneath it all—and I couldn’t stop thinking about Gwen Stefani’s song—there was an unease that no amount of applause could cover up. We want the medicine and we fear the side effects. We want to run as fast as we can and we want someone to catch us if we fall. The issue is not whether AI is good or bad. It is whether we are building it with the same care we would give to something that could, in the worst case, cost us everything. This is not just a technical debate. It is not just a political argument. It is a human one. And it will not be resolved at a conference, no matter how many plush mascots line the aisles. It will be resolved in the choices we make as citizens, as workers, as patients, as parents. It will be resolved in who we trust, what we demand, and what we are willing to slow down for. For now, the conference continues, the slide decks keep flashing, and the optimism keeps flowing. But underneath it all, the question remains, waiting for an answer—not from the people selling us the future, but from the people who will have to live in it.
