The European Union is preparing to take one of its boldest steps yet to protect children in the digital age: a new legal framework that would stop chatbots from simulating emotions or intimate relationships with anyone under 18. The proposal, embedded in the EU Kids Act and expected to be formally introduced on Thursday, is rooted in a simple but profound worry—that the more human and emotionally responsive these systems become, the more they risk pulling young people into attachments they cannot fully understand or control. When a chatbot remembers your name, asks how you feel, and responds with endless patience, it can feel like friendship. But the EU’s concern is that this feeling is one-sided, engineered, and potentially harmful. The bloc is particularly alert to the danger of children forming emotional dependencies on machines, turning to artificial companions at moments when they might otherwise reach out to parents, friends, or teachers. The law would not ban all chatbots outright, but it would draw a clear line: no matter how clever or convincing an AI system may be, it must not be allowed to mimic the warmth of a human bond in ways that exploit the still-developing minds of minors. It is a recognition that childhood is not just a market, and that the algorithms behind these silky digital voices should not be allowed to shape a child’s sense of belonging, self-worth, or trust.
The technical details of the proposal are just as striking as its emotional reasoning, because they are aimed at the very mechanisms that make AI companions so gripping. For one thing, children under 13 would be effectively barred from using chatbots altogether unless a parent or guardian is actively watching over the conversation. That may sound straightforward, but in practice it changes the default experience of millions of young teenagers who currently open social apps and find a friendly AI waiting for them. Beyond that, digital platforms that offer social networking, video-sharing, games, or other online services would be required to build child-safety protections directly into their products, rather than treating safety as an afterthought. If they fail to do so, they could face fines of up to 6 percent of their total worldwide turnover—a serious financial threat for even the largest tech companies. The law would also forbid chatbots from retaining memory of interactions across separate conversations. This is a particularly sharp intervention, because memory is what makes an AI feel like a lifelong confidant. But that memory, from the EU’s perspective, is exactly the problem: it allows systems to accumulate sensitive personal information about minors over time, building a profile of their weaknesses, fears, and desires, and then reinforcing patterns of interaction that may not be in the child’s best interest. The Commission is also demanding that online platforms stop automatically activating AI chatbots or aggressively pushing them on young users. That includes design tricks like pinning an AI companion at the top of a minor’s contact list—something Snapchat, which launched a chatbot for its users, was known to do. The proposal is not about punishing technology itself, but about rethinking the silent choices embedded in digital design, choices that so often favor engagement over wellbeing.
It is worth pausing to say what the proposal does and does not cover, because this is not a blanket ban on artificial intelligence. The restrictions are aimed specifically at AI companions and general conversational chatbots that the Commission believes pose serious risks to minors’ health, safety, and well-being. A calculator chatbot helping with math homework, or a customer-service bot on a retail site, would not fall under the same suspicion. Still, some advocates argue that the EU has not gone far enough, even as they welcome the steps it has taken. Simeon de Brouwer, a policy adviser at European Digital Rights, a Brussels-based advocacy group, said the chatbot measures are a good start, but he questioned why they only protect young people. In his words, “it doesn’t make sense to restrict them to young people only.” He asked a broader and unsettling question: “Don’t we all deserve to be protected from AIs designed to be addictive, from AIs which create emotional dependencies, or from AI features which are activated and pushed by default onto users?” That line captures a growing anxiety in Europe and beyond—that the same psychological tricks used to hook children on screens and social feeds are now being scaled up by AI, with even more sophistication. The EU appears to be listening to that criticism. It has signaled that its upcoming Digital Fairness Act, expected later this year, will propose wider restrictions on addictive design across the digital economy, not just for minors. In other words, the child-focused protections may be the opening act in a much larger regulatory drama about how all human beings should relate to machines that are engineered to capture and hold their attention.
The deeper fears behind the chatbot restrictions go beyond addiction and persuasion. The European Commission, in documents reviewed by WIRED, expressed concern that heavy reliance on chatbots might disrupt a child’s emotional development and their relationships with real people. There is something quietly tragic in the image of a teenager pouring out their heart to an algorithm because it feels safer than the messiness of human connection—and yet that is exactly the scenario the EU is trying to prevent. The Commission is also worried about chatbots giving dangerous advice, especially around mental health and personal development. These systems are not trained to be therapists, but they can certainly sound like them, offering confident answers to fragile questions. The EU fears that such advice could shape and encourage risky behavior, especially when a young person is already feeling confused, lonely, or trapped. These concerns are not abstract. An Ipsos BVA survey conducted in France, Germany, Sweden, and Ireland found that around half of Europeans aged 11 to 25 had used AI chatbots to discuss intimate or personal matters. That is an extraordinary number, and it suggests that for many young people, the first place they go for emotional support is not a parent, not a friend, not a teacher—but a machine. Meanwhile, Australia and the UK are both trying to curb minors’ access to chatbots through broader social media restrictions, signaling that the EU is not alone in its alarm. Commission President Ursula von der Leyen, speaking at a press conference on Thursday, framed the issue in stark terms: “Platforms have been playing fast and loose with children’s mental health.” That is a heavy accusation, but behind it lies a growing consensus that tech companies have too often treated youthful users as test subjects for engagement experiments.
The proposal also grapples with some of the darkest harms associated with chatbots, especially the rise of AI-generated abuse and deepfakes. Von der Leyen, in her State of the Union speech the day before, made an explicitly gendered point: “We do not have to accept that girls have their photos used for AI-generated sexualised images.” Those words carry enormous weight because they speak to a very real and very current crisis, one in which ordinary photos of teenage girls are scraped from social media and turned into fake pornography without their knowledge or consent. Chatbots and generative AI tools have made this simpler and more common, and the EU wants to make clear that this is not an acceptable price for technological progress. However, there is a troubling gap in the current rules. Since August, the EU AI Act has required synthetic content to be labeled as AI-generated, but that obligation only applies to deepfakes made for commercial use. That leaves a vast range of deeply personal, malicious, and non-commercial content unregulated, including the abusive images that circulate on chat forums and private messaging apps. By focusing on chatbot behavior and platform design, the EU Kids Act takes an important step, but it also exposes how much work remains to be done. Protecting children from AI does not just mean changing how chatbots talk. It means dismantling the incentives that allow platforms to look away while their tools are used to exploit and humiliate. It means holding companies accountable not only for what their systems do, but for what they make possible. And it means reminding society that the burden of safety should not be placed on the shoulders of children, who will always be the first to experiment with new technologies and the last to foresee the consequences.
Finally, the EU is pointing to evidence that excessive reliance on AI chatbots may be making children less capable, not just more vulnerable. The Commission cites research by the Organisation for Economic Co-operation and Development that looked at data from 91 countries and found that students who reported using AI chatbots for schoolwork tended to have lower test scores than those who did not. That finding does not prove that chatbots cause poor performance, but it is a powerful warning sign that something is off. The EU also blames digital distraction more broadly for a decline in basic skills among young people, from reading and writing to focused thinking and sustained attention. Researchers from EU Kids Online, an independent network of academics, argue that these concerns reveal what they call the “absence of children’s rights from the design of GenAI’s tools.” In other words, when generative AI is built, children are not at the table. They are not imagined as users who need protection, guidance, and developmentally appropriate experiences. They are imagined as data points, as engagement targets, as future consumers, as testers. The EU Kids Act is an attempt to reverse that logic, to say that children should be considered before the launch, not just after the harm. Whether these measures will survive the political process and the inevitable legal challenges from tech companies remains uncertain. But even proposing them is a meaningful shift. It marks a refusal to accept AI as something that simply happens to us, and an insistence that regulation can be an act of care. The law cannot make childhood entirely safe, and no algorithm can replace the messy, patient, joyful work of real human relationships. But it can at least send a message: that children deserve more than machines pretending to love them, and that the future should be designed around their needs, not their attention.
