Human pre-moderation is more costly to deliver than AI, but Tellmi puts impact first
Companies that prioritise profit are embracing AI moderation because it is cheap and fast. Tellmi prioritises safeguarding, so we use human pre-moderation to facilitate safe digital peer-to-peer support for children aged 11+.
Human pre-moderation is more costly to deliver, but Tellmi puts impact first. By contrast, companies like Meta have replaced fact-checkers and human moderators with AI. AI is great at certain tasks. It can outperform doctors in making emergency room decisions (Broedeur et al., 2026¹), but when it comes to interpreting complex scenarios or recognising the subtleties and idiosyncrasies of teenage behaviour, AI is too easy to game (Vowels & Marcantonio, 2025²).
Young people are smart. They use ’algospeak’ and intentional misspellings to bypass automated content moderation. That’s why Tellmi trains humans to screen and risk-assess every single post and reply before it is either published, withheld, or transferred to a counsellor for high-risk support. For example, a post that says ‘read every third letter’ to hide a message about suicidal intent is likely to be missed by AI, but will not get past an eagle-eyed Tellmi moderator.
Some digital providers operate a policy of post-moderation, in which comments are published and if a user reports something, a moderator can remove it. An experiment that explored how pre- and post-moderation strategies affected online comment sections in newspapers found that pre-moderation significantly reduced toxic content. By contrast, post-moderation increased toxicity because when toxic comments were visible, other users tended to “tone-match” their own responses. Filtering those comments before publication, fostered a positive feedback loop in which respectful interaction was the norm (Tomás-Valiente Jordá, 2025³).
Tellmi’s unique approach allows for more nuanced decision-making and a more personal approach to safety (Haime & Biddle, 2025⁴). Humans understand context. For example, a user who posts “I’m killing myself revising” is having a tough time, but the user who stops at “I’m killing myself” needs immediate help. Rapidly detecting suicidal ideation is the most critical example, but context is also crucial when young people make posts about sex and relationships.
Tellmi’s human moderators can quickly discern the validity of a perennial fascination such as ‘What is sex?’ by looking at the child’s age and their preceding posts, to understand both the user and the context, whereas with no age verification, AI will diligently explain that ‘In heterosexual vaginal intercourse, an erect penis is inserted into a vagina, and rhythmic thrusting can lead to sexual pleasure and orgasm’.
As we debate the ban on social media for under-16s, we must question the wisdom of allowing young people unfettered access to compliant AI tools that provide ‘informative’ responses to the most graphic questions. AI may be every thirteen-year-old's wet dream, but safeguarding requires boundaries, and so do young people.
References
Peter G. Brodeur et al., Performance of a large language model on the reasoning tasks of a physician. Science 392,524-527 (2026). DOI: 10.1126/science.adz4433
Vowels LM, Marcantonio T. Exploring How Large Language Models Understand Sexual Communication in Hypothetical Sexual Situations. J Sex Res. 2025 Sep 2:1-23. doi: 10.1080/00224499.2025.2547814. Epub ahead of print. PMID: 40891967.
Jordá, Tomás-Valiente. Inside the experiment: comparing pre- and post-moderation in online comment sections on reducing toxicity in digital discourse. 2025 Hertie School, Data Science Lab
Haime Z, Biddle L. Exploring Mental Health Content Moderation and Well-Being Tools on Social Media Platforms: Walkthrough Analysis. JMIR Hum Factors. 2025 May 29;12:e69817. doi: 10.2196/69817. PMID: 40440699; PMCID: PMC12163353.