International
The Morning Dev

Only the news worth knowing

‹
 

Launch5 sources225 comments in the community

OpenAI wants to stop deepfakes from overwhelming child abuse tiplines

The Verge · By Lauren Feiner ·

OpenAI wants to stop deepfakes from overwhelming child abuse tiplines

The Brief · written from 5 sources

  • OpenAI announced a classifier it says can better detect abusive imagery, to help child sexual abuse investigators find material they can act on quickly.
  • OpenAI introduced new warning screens to stop users from accessing material that sexualizes children, and ChatGPT will ban an account that tries to access CSAM content for 24 hours as a warning.
  • The tools were announced on 2026-11-16 at a press briefing in Washington, DC, alongside practical guidance and partnerships.

Why it matters

Developers building on OpenAI services or moderating user-generated content may see new CSAM detection tooling and account enforcement, which can affect how they use image classification and moderation workflows.

Who’s affected

  • Developers and teams using OpenAI services
  • Trust and safety teams handling user-uploaded images
  • Child sexual abuse investigators and tipline staff

OpenAI is announcing a tool that it says will help child sexual abuse investigators cut through the noise to find harmful material they can act on quickly. At a Monday press briefing in Washington, DC, OpenAI announced a classifier it says can better detect abusive imagery of…

What developers are saying

Based on 225 comments · Mixed reactions

Commenters on Hacker News are split. Some welcome tools that help fight CSAM and reduce moderation burden, while others see the announcement as public relations or a possible data grab. Privacy and surveillance concerns recur, and one commenter reports an account banned after an automated classifier flagged over 100,000 images.

  • Welcome moderation tools25%
  • Surveillance and privacy fears30%
  • Skepticism of OpenAI's motives25%
  • Account ban concerns10%
  • Scale of AI-generated CSAM10%

In favour

  • + Gives moderators tools where LLM providers otherwise return content warnings
  • + Could reduce the review burden on human moderators amid rising CSAM volume

Concerns

  • − Scanning all images could send non-CSAM content to OpenAI
  • − Fear that the tools could become surveillance that scans files and reports users
  • − Seen by some as public relations with little real commitment
  • − An account ban after classifier use was reported, with no information provided to the affected party

“I'm glad they are giving tools to help fight this”

bearjaws on Hacker News

“this seems more like a data grabbing play.”

nrmitchi on Hacker News

“This is a positive development.”

eutropia on Hacker News

Timeline

  1. OpenAI announces image classifier and new ChatGPT warning screens against CSAM

  2. AI News & Artificial Intelligence | TechCrunch

    OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM

  3. OpenAI News

    Strengthening the fight against online child sexual abuse with new tools and support

  4. Techmeme

    OpenAI announces tools to combat CSAM, including an image classifier and new warning screens to prevent users from accessing material that sexualizes children (Ivan Mehta/TechCrunch)

  5. Hacker News: Front Page

    Strengthening the fight against online child sexual abuse

  6. Hacker News

    108 points · 225 comments

  7. AI | The Verge

    OpenAI wants to stop deepfakes from overwhelming child abuse tiplines

Full coverage · 5 sources

How do you feel about this?

0 Comments