I Put a Bunch of AI Agents in Charge of a Fantasy Soccer Team. They’re Not Better Than Humans—Yet

Staff
By Staff 12 Min Read

There is something almost magical about the idea of peering into the future, and for as long as humans have existed, we have tried to do it in every way we could imagine—by reading animal entrails, studying the stars, watching the stock market, or simply trusting our gut. Now we are handing that ancient ambition to machines. In a series of recent experiments, researchers asked cutting-edge AI models to predict things like which team would win the NBA Championship or who President Trump would meet with, and the results were startling. According to a researcher named Xu, the team behind these tests “very quickly started to realize that models did really well.” The machines weren’t just guessing at random; they were making forecasts that, in many cases, outperformed the average person without any special instructions or guidance. Among the models tested, GPT-5 proved to be the best at making predictions, followed by Grok 4, Claude Sonnet, Gemini, and Llama. It is the kind of finding that makes you pause. We already know AI can beat us at chess and Go, solve complex math problems, and write passable poetry, but predicting the chaotic, messy, human-driven future feels like a different kind of skill entirely. It is one thing to master a closed system with fixed rules; it is quite another to anticipate the outcome of an election, a sports season, or a diplomatic meeting, where so much depends on luck, emotion, and the unpredictable choices of real people. Yet here was evidence that machines were beginning to do exactly that, and they were doing it with a quiet, unassuming confidence that felt both impressive and a little unsettling.

To explore this further, Xu and his team developed a benchmark called Prophet Arena, designed specifically to measure a model’s ability to make predictions. It is not the only project of its kind. Researchers at Harvard University and a company called Arcada Labs created something called Prediction Arena, where they gave frontier models $10,000 to wager on prediction markets over the course of 57 days. The results were not exactly a financial windfall—the models consistently lost money—but the researchers saw something else that was perhaps even more important: evidence that the models were improving over time. Think about that for a moment. An AI system started out making bad bets, but gradually it learned from its mistakes, adjusted its reasoning, and became a slightly better gambler. It is like watching a novice poker player slowly develop a feel for the table, learning when to hold, when to fold, and when to trust the numbers over the adrenaline. That kind of learning curve is deeply human, and it is also deeply promising for the future of AI. The fact that these models lost money is not really the point. The point is that they were able to take a messy, noisy, unpredictable stream of information and turn it into probabilities that got more accurate over time. If that trajectory continues, it is easy to imagine a future where AI not only plays the prediction game but plays it exceptionally well. And if that happens, the implications go far beyond sports betting or political forecasts. We are talking about machines that can see around corners, anticipate market shifts, and maybe even warn us about crises before they happen.

The question, of course, is how they do it. Xu explains that AI can make predictions most effectively when it is given a custom harness—a carefully designed framework that gives the model structure and purpose. The most effective approach seems to involve multiple agents, each with its own specific job: one reads the news, another collects data, another analyzes the whole system, and together they work like a kind of prediction brain trust. It is a fascinating image: a team of digital minds, each focused on a different part of the puzzle, sharing their insights and coming to a collective conclusion. But even with all that horsepower, AI still often lags behind very skilled human experts known as superforecasters. These are people who have spent years honing their ability to predict world events, and they are remarkably good at it. Xu points out that while humans cannot consume as much information as an AI, their understanding of the real world is deeper. They grasp the dynamics behind a news story—the political tensions, the cultural undercurrents, the historical patterns—in ways that a machine, at least for now, cannot fully replicate. It is a humbling reminder that raw data is not the same as wisdom. A superforecaster might read a single article and understand the human motivations at play, while an AI reads a million articles and still misses the emotional truth that makes all the difference. That deeper understanding is hard to quantify, but it is real, and it has kept human experts ahead of the machines in this particular arena.

But perhaps not for much longer. There is growing evidence that the superforecaster’s edge is eroding, and the erosion is happening faster than many people expected. In a contest called the Metaculus Cup, AI sometimes managed to beat even the best human predictors, a result that would have seemed almost impossible just a few years ago. And another benchmark, ForecastBench, is currently running an ongoing contest that could provide even more evidence that AI can match human skills once the results come in. The pattern is clear, and Xu believes we can expect the big labs to turn their attention to future-gazing as a powerful new way to measure progress in machine intelligence. In other words, predicting the future is becoming the next great benchmark, the way math, coding, and law have been used to test AI’s capabilities. It is not hard to see why. The future is the ultimate test of understanding, because it requires not just knowing facts but knowing how the world works, how people make decisions, and how random events can ripple outward in unexpected ways. If AI can master that, it will be a sign that we have crossed some profound threshold. And if the big labs start competing on prediction, we will likely see rapid improvements in the coming years, as they pour resources into making their models better at this very human skill. It is both exciting and a little frightening to imagine what that future looks like—machines that can tell us what is going to happen before it happens, with a degree of accuracy that no human could match.

All of these predictive experiments point toward a larger truth: industries built on trading and forecasting may soon be upended by AI. Financial markets, commodities trading, sports betting, political analysis—these are all fields where the ability to anticipate the future is the difference between success and failure. If AI becomes better than humans at seeing what is coming, the human role in these industries will have to change. Perhaps the best human traders will take on a new kind of role, supervising and orchestrating AI agents capable of divining the future far better than they ever could. Instead of being the ones making the predictions, they will become the managers of machines that make predictions. They will need to understand when to trust the AI, when to question it, and how to integrate its insights with broader strategic thinking. It is a shift that echoes what has happened in so many other fields: the expert moves from being the doer to being the orchestrator, from the one who swings the hammer to the one who guides the robotic arm. There is something a little melancholy about that, a sense that we are losing a part of our own agency, but there is also something exciting about it. If AI can truly help us see the future more clearly, we might make better decisions, avoid disasters, and seize opportunities we would otherwise miss. The key will be finding the right balance between human judgment and machine intelligence, and doing it before the machines get so good that we are left with nothing to do but watch.

For the time being, though, the AI in my own life is not quite ready to make me rich or to win me bragging rights. The suggestions it offers still don’t seem all that impressive, and its team hasn’t outscored the average over the past few weeks. I certainly won’t be using any of its recommendations in my own selections, at least not yet. There is something almost comforting about that, a small reminder that the machines still have their limits. But I know it won’t last. AI will no doubt get better, and perhaps it will soon be all too easy to dominate a fantasy league or a stock portfolio with the help of a tireless, endlessly patient digital predictor. I am kind of hoping it won’t, though, because part of the fun of these games is trying to win all by yourself. There is a joy in the struggle, in the late-night research, in the gut feeling that tells you to pick a player no one else believes in, in the satisfaction of being right when the odds were against you. If an AI hands you the answer, all of that disappears. The bragging rights would feel hollow, the victory meaningless. Maybe that is the real cost of superhuman prediction: not losing our jobs or our edge, but losing the sweet, messy pleasure of being wrong sometimes, and learning from it, and trying again. For now, I will hold onto my human limitations and enjoy the game while I still can.

Share This Article
Leave a Comment

Leave a Reply

Your email address will not be published. Required fields are marked *