AI Child Safety Expert Flags Open Problems

The surge in AI-generated child sexual abuse material (CSAM) is a multifaceted issue, with some reports indicating a 26,362% rise in photo-realistic AI videos of child sexual abuse in 2025 compared to the previous year. This increase is attributed to the growing sophistication and accessibility of AI tools that allow individuals with minimal technical skills to create harmful content at scale. The National Center for Missing and Exploited Children (NCMEC) received 4,700 reports related to AI-generated CSAM in 2023. A significant portion of this abusive content is created using images of children sourced from social media accounts. AI-powered applications can digitally alter these innocent photos to produce explicit deepfake content. This synthetic media not only creates new victims but also complicates law enforcement's efforts to identify and rescue real children depicted in traditional CSAM. To combat this, a "Safety by Design" approach is being advocated for AI development, which involves integrating child safety measures throughout the entire lifecycle of an AI model. This includes carefully curating training datasets to separate depictions of children from adult sexual content and using classifiers to identify and block the generation of potential CSAM. Organizations like Thorn are developing solutions to help AI companies mitigate the misuse of their models. For engineers in edtech, adaptive learning systems powered by AI are crucial for personalizing education. These systems utilize machine learning algorithms to analyze a student's interactions, performance, and preferences to dynamically adjust the curriculum. This creates individualized learning paths that can cater to different learning paces and styles, which is particularly beneficial for students with special needs. Knowledge tracing models are a key component of these adaptive systems, modeling a student's understanding of concepts over time to predict future performance. Deep learning approaches like Deep Knowledge Tracing (DKT) use recurrent neural networks to process sequences of student interactions and update their knowledge state. This allows the system to identify where a student is struggling and offer targeted support. To optimize the selection of educational content, multi-armed bandit (MAB) algorithms are employed. These algorithms balance the need to present content the student is likely to succeed with (exploitation) and introducing new topics to gauge their understanding (exploration). This approach helps to keep students engaged while effectively advancing their learning. Speech recognition for young learners presents unique challenges due to variations in vocal tract length, pronunciation, and vocabulary. Modern speech recognition systems utilize subword units and are increasingly tailored to the specific characteristics of children's voices to improve accuracy. This technology is vital for AI-powered reading tutors, enabling them to provide real-time feedback on pronunciation and fluency. Ensuring the ethical use of AI in education is paramount, with a focus on data privacy, transparency, and fairness. It is crucial to have clear guidelines on how student data is collected and used, and to prevent algorithmic bias that could disadvantage certain students. As AI becomes more integrated into educational tools, maintaining a balance between technological advancement and the well-being of young users is a primary concern.