🔗 Share this article

Technology companies and child protection organizations will receive authority to assess whether artificial intelligence tools can generate child exploitation images under new UK legislation.

Significant Rise in AI-Generated Harmful Content

The announcement coincided with findings from a protection monitoring body showing that reports of AI-generated child sexual abuse material have more than doubled in the past year, growing from 199 in 2024 to 426 in 2025.

Updated Regulatory Structure

Under the amendments, the government will permit approved AI developers and child safety organizations to examine AI models – the foundational systems for chatbots and image generators – and verify they have adequate safeguards to prevent them from creating depictions of child sexual abuse.

"Ultimately about stopping abuse before it occurs," stated the minister for AI and online safety, adding: "Specialists, under rigorous conditions, can now detect the risk in AI systems early."

Addressing Legal Obstacles

The changes have been introduced because it is against the law to produce and own CSAM, meaning that AI developers and others cannot create such content as part of a testing regime. Previously, authorities had to wait until AI-generated CSAM was uploaded online before dealing with it.

This law is designed to preventing that issue by enabling to halt the production of those images at source.

Legal Framework

The changes are being added by the authorities as revisions to the crime and policing bill, which is also implementing a prohibition on owning, creating or distributing AI models developed to generate exploitative content.

Practical Impact

This recently, the official visited the London headquarters of Childline and listened to a mock-up conversation to advisors featuring a account of AI-based abuse. The call depicted a adolescent requesting help after being blackmailed using a explicit deepfake of himself, created using AI.

"When I hear about young people experiencing extortion online, it is a cause of intense frustration in me and justified anger amongst parents," he stated.

Alarming Statistics

A prominent internet monitoring foundation reported that instances of AI-generated exploitation material – such as webpages that may include multiple files – had more than doubled so far this year.

Instances of the most severe content – the gravest form of exploitation – rose from 2,621 visual files to 3,086.

• Girls were overwhelmingly targeted, accounting for 94% of illegal AI depictions in 2025
• Depictions of newborns to toddlers rose from five in 2024 to 92 in 2025

Sector Reaction

The law change could "represent a crucial step to guarantee AI tools are secure before they are launched," commented the chief executive of the internet monitoring organization.

"AI tools have enabled so victims can be victimised all over again with just a simple actions, giving criminals the ability to make possibly limitless amounts of sophisticated, lifelike child sexual abuse material," she added. "Content which additionally commodifies survivors' trauma, and renders children, particularly girls, more vulnerable both online and offline."

Counseling Interaction Data

The children's helpline also released information of support interactions where AI has been mentioned. AI-related risks mentioned in the conversations comprise:

• Employing AI to evaluate weight, physique and looks
• AI assistants discouraging young people from talking to safe guardians about abuse
• Facing harassment online with AI-generated material
• Online blackmail using AI-faked images

Between April and September this year, Childline delivered 367 counselling interactions where AI, conversational AI and related terms were discussed, significantly more as many as in the same period last year.

Half of the mentions of AI in the 2025 sessions were connected with mental health and wellness, including utilizing chatbots for assistance and AI therapy apps.