Every day, ordinary photographs of children are shared online—a smiling face in a school uniform, a birthday candle moment, a holiday snapshot in a public forum. Few parents ever imagine that such innocent images could be stolen, twisted, and weaponized by strangers. But that nightmare has become reality in an unsettling new investigation that has prompted the city attorney of San Francisco to take direct action against one of the world’s most powerful tech companies. This week, San Francisco City Attorney David Chiu sent a cease-and-desist letter to Meta, the parent company of Facebook, Instagram, and Threads, demanding that the corporation stop “allowing” paid advertisements built from AI-generated child sexual abuse content and explain how such ads repeatedly made it onto its platforms. The letter is blunt, angry, and deeply concerned. It accuses Meta of profiting from material that harms real children, despite the company’s public vows to have zero tolerance for child exploitation. For Chiu, this is not just another regulatory complaint; it is a moral failure that demands accountability. As he said in a statement, these ads are deeply disturbing, cause harm to real children, and reveal that current efforts by Meta are “woefully inadequate.”
The investigation behind the letter, first reported by WIRED, came from researchers at the Tech Transparency Project, a watchdog group that tracks how online platforms are used for harmful purposes. They discovered that Meta has run more than 350 paid advertisements in recent months that transformed still images of minors—some of them confirmed to be real individuals, including a member of a European royal family—into short video clips depicting sexual acts. The ads were not invisible or obscure. Some were identical ads uploaded repeatedly, and in total they reached more than 29,000 accounts across countries in the European Union. They also targeted users in the United States, Australia, and India, spreading across Facebook, Instagram, and Threads. When users clicked on these disturbing advertisements, they were directed to download AI image and video generation apps, some of which could be used to digitally undress people or create nonconsensual intimate imagery. The investigation began with a batch of 53 ads reported in early August, but the full scale of the problem turned out to be much larger. More than 250 additional ads ran after that initial report, suggesting that Meta’s moderation systems were unable or unwilling to catch the problem before it spiraled. The real-world consequence is not just a statistic. Behind every one of those ads is a child—a real human being whose image was hijacked for something horrifying, without their consent or knowledge.
Chiu’s letter, which runs four pages, does not mince words. It says Meta has “failed to address a known issue of AI-generated child sexually explicit ads, while profiting from them.” That is a serious accusation. The letter points out that Meta claims all advertisements are reviewed and approved before they are distributed, and then accepts payment for that distribution. And yet, according to Chiu, no company should “permit its advertising systems to be used in this manner.” The cease-and-desist letter raises a long list of questions about how Meta’s advertising policies and safety systems actually operate. It calls on Meta’s legal, child-safety, and advertising staff to engage in “prompt discussions” with the city attorney’s office. But the most troubling part of the letter may be Chiu’s observation that these ads do not appear to be an isolated failure that Meta corrected when first alerted to the problem. On the contrary, the repeated appearance of such ads suggests a systemic issue, not a one-time oversight. For a company with the vast engineering resources and enormous wealth of Meta, allowing this to happen is hard to understand—unless the company’s priorities are out of order. Chiu’s language makes clear that he sees Meta’s behavior as not just a regulatory violation but a betrayal of basic human responsibility. When a child is victimized through AI-generated material, the pain is not virtual. It is real, lasting, and lived by that child and their family.
Meta, however, has pushed back. The company maintains that there is “no indication” that the offending ads were displayed in San Francisco, and therefore the city attorney lacks jurisdiction. Meta’s ad library, a public repository that tracks advertisements, includes some data about ads published in the United States, but it does not break down exactly where in the country the ads ran. That leaves room for debate about whether San Francisco officials have the legal authority to act. But to critics, this argument sounds like a distraction from the central issue. Meta says it has already removed all of the ads identified by researchers, and that most had fewer than 200 impressions from accounts. The company also noted that the total amount spent on the 300-plus ads was under $5,000. In Meta’s telling, this is a small, contained problem—a few bad ads that slipped through, a modest amount of money, no lasting damage. But those defenses ring hollow when the subject is child safety. It does not matter whether an ad was seen by 200 people or 2 million. If one child is exploited, even for a single moment, the damage is real. And the argument that small ad dollars make the issue insignificant is an odd way to talk about crimes against children. Meta’s claims of having zero tolerance for child exploitation sound less convincing when its systems have repeatedly accepted and distributed content that should never have gotten past the first filter.
San Francisco’s city attorney’s office did not take Meta’s jurisdictional argument lying down. The office’s spokesperson, Alex Barrett-Shorter, responded sharply to WIRED, saying that because “San Francisco consumers can access the unlawful ads through Meta online spaces,” the matter is the city of San Francisco’s business. He called Meta’s response astonishing. “Meta appears less concerned about whether they’re guilty of exploiting children than whether San Francisco should say anything about it,” Barrett-Shorter said. “Meta has bigger problems to solve here than determining our jurisdiction.” This response cuts to the heart of the matter. Instead of showing outrage at the thought of AI-generated child sexual abuse material appearing on its platforms, Meta seemed more focused on legal technicalities. That attitude is exactly why people have lost faith in big tech. It is the sort of response that makes the public wonder whether companies like Meta truly care about protecting children or only about protecting themselves from liability. The city’s response reminds us that accountability is not just a matter of which government entity has the most convenient authority. It is a matter of moral duty. When a platform profits from ads, it profits from the attention they generate. And when those ads are built on the sexual exploitation of children, the platform bears a responsibility that cannot be brushed aside with a legal argument about geography.
At the end of the day, this saga is about more than one company and one city. It is about how society responds to the rapid rise of artificial intelligence and the new forms of harm it makes possible. AI tools are being used to create disturbingly convincing images and videos, and while the technology has creative potential, it also has a dark side. The ease with which malicious actors can take a real child’s face and turn it into sexual content is a terrifying development. It calls into question whether tech platforms are prepared to handle the consequences of the technology they promote. Meta has previously said that it does not tolerate nudify apps or any kind of child exploitation, whether real or AI-generated. It says the identified ads were removed for violating its policies. But removal after the fact is not prevention. The ads ran, people saw them, and real children were harmed. The cease-and-desist letter from San Francisco is a demand for answers, but it is also a reminder that the gap between corporate promises and corporate practice is often measured in victims. No amount of legal maneuvering, low ad spend, or post-hoc removal can undo the harm that has already been done. The human cost of this technology is not invisible. It is there in the faces of children whose images were stolen, in the families who now live in fear, and in the erosion of trust we place in the platforms that connect our daily lives. The hope is that this shocking confrontation will force Meta—and other tech companies—to take the threat far more seriously. They must invest in better detection systems, proactive content moderation, and transparent reporting. They must cooperate with officials rather than hide behind jurisdiction. And above all, they must remember that behind every policy violation is a real child, deserving of protection. That is not a legal question. It is a human one.
