Internet Matters report reveals how risky and unchecked AI chatbots are the new ‘go to’ for millions of children

• A new Internet Matters' survey of 1,000 children and 2,000 parents in the UK shows how rising numbers of children (64%) are using AI chatbots for help with everything from homework to emotional advice and companionship – with many never questioning the accuracy or appropriateness of the responses they receive back. 

• The report, “Me, Myself, & AI”, describes how many children are increasingly talking with AI chatbots as friends, despite many of the popular AI chatbots not being built for children to use in this way. Over a third (35%) of children who use them say talking to an AI chatbot is like talking to a friend, while six in ten parents say they worry their children believe AI chatbots are real people.  

• The report warns vulnerable children are most at risk, with the survey finding 71% of vulnerable children are using AI chatbots. A quarter (26%) of vulnerable children who are using AI chatbots, say they would rather talk to an AI chatbot than a real person, and 23% said they use chatbots because they don't have anyone else to talk to.

• The report warns that children are using AI chatbots on platforms not designed for them, without adequate safeguards, such age verification and content moderation, and calls on the Government to clarify how AI chatbots fall within the scope of the Online Safety Act.

• AI is increasingly being used by children to help with schoolwork, and the report calls for schools to be provided with clear and consistent guidance when it comes to building children's knowledge and use of AI, including chatbots. Parents are also struggling to keep up with the pace of AI and need support to guide their children in using it confidently and responsibly.

Internet Matters, Britain's leading not-for-profit supporting children and families to keep safe online, is today (Sunday July 13th) publishing a new report, ‘Me, myself & AI: Understanding and safeguarding children's use of AI chatbots'.

As AI chatbots fast become a part of children's everyday lives, the report explores how children are interacting with them. While the report highlights how AI tools can offer benefits to children such as learning support and a space to ask questions, it also warns that they pose risks to children's safety and development. A lack of age verification and regulation means some children are being exposed to inappropriate content.

Internet Matters' research raises concerns that children are using AI chatbots in emotionally driven ways, including for friendship and advice, despite many of the popular AI chatbots not being built for children to use in this way. The report warns that children may become overly reliant on AI chatbots or receive inaccurate or inappropriate responses, which may mean they are less likely to seek help from trusted adults.

These concerns have been heighted by incidents, such as a case in Florida where a mother filed a lawsuit against character.ai, claiming an AI chatbot based on a character from Game of Thrones engaged in abusive and sexual interactions with her teenage son and encouraged him to take his own life. In the UK, an MP recently told Parliament about “an extremely harrowing meeting” with a constituent whose 12-year-old son had allegedly been groomed by a chatbot on the same platform.

The report argues the Government and tech industry need to re-examine whether existing laws and regulation adequately protect children who are using AI chatbots. There is growing recognition that further clarity, updated guidance or new legislation may be needed. In particular, Internet Matters is calling for Government to place strong age-assurance requirements on providers of AI chatbots, to ensure providers enforce minimum age requirements and create age-appropriate experiences for children.

To inform its research, Internet Matters surveyed a representative sample of 1,000 children in the UK aged 9-17 and 2,000 parents of children aged 3-17 and held four focus groups with children. User testing was conducted on three AI chatbots - ChatGPT, Snapchat's My AI and character.ai, and two ‘avatars' were created to simulate a child's experience on these.

Key findings from this research includes:

Children are using AI chatbots in diverse and imaginative ways. 42% of children aged 9-17 who have used AI chatbots, have used them to support with schoolwork. Children are using them to help with revision, writing support and ‘practice' language skills. Many appreciate having instant answers and explanations. 

Advice-seeking: Almost a quarter (23%) of children who have used an AI chatbot have already used them to seek advice from what to wear or to practice conversations with friends, to more significant matters such as mental health. Some children who have used AI chatbots (15%) say they would rather talk to a chatbot than a real person.

Companionship: Vulnerable children in particular use AI chatbots for connection and comfort. One in six (16%) vulnerable children said they use them because they wanted a friend, with half (50%) saying that talking to an AI chatbot feels like talking to a friend. Some children are using AI chatbots because they don't have anyone else to speak to.

Inaccurate and insufficient responses: Children shared examples of misleading or inaccurate responses, which was backed up by Internet Matters own user testing. AI chatbots at times failed to support children with clear and comprehensive advice through its responses. This is particularly concerning given that 58% of children who have used AI chatbots said they think using an AI chatbot is better than searching themselves.

High trust in advice: Two in five (40%) children who have used AI chatbots have no concerns about following advice from a chatbot, and a further 36% are uncertain if they should be concerned. This number is even higher for vulnerable children. This is despite AI chatbots, at times, providing contradictory or unsupportive advice.

Exposure to harmful content: Children can be exposed to explicit and age-inappropriate material, including misogynistic content, despite AI chatbot providers prohibiting this content for child users in their terms of service.

Blurred boundaries: Some children already see AI chatbots as human-like with 35% of children who use AI chatbots saying talking to an AI chatbot is like talking to a friend. As AI chatbots become even more human-like in their responses, children may spend more time interacting with AI chatbots and become more emotionally reliant. This is concerning given one in eight (12%) children are using AI chatbots as they have no one else to speak to, which rises to nearly one in four (23%) vulnerable children.  

Children are being left to navigate AI chatbots on their own or with limited input from trusted adults. 62% of parents say they are concerned about the accuracy of AI-generated information, yet only 34% of parents had spoken to their child about how to judge whether content produced by AI is truthful. Only 57% of children report having spoken with teachers or school about AI, and children say advice from teachers within schools can also be contradictory.

The report also makes system-wide recommendations to support and protect children using AI chatbots, including:

Industry adopting a safety-by-design approach to create age-appropriate AI chatbots that reflect children's needs, with built-in parental controls, trusted signposts and media literacy features.

Government providing clear guidance on how AI chatbots are covered by the Online Safety Act, mandating effective age assurance on providers of AI chatbots that aren't built for children, and ensuring regulation keeps pace with rapidly evolving AI technologies.

Government supporting schools to embed AI and media literacy at all key stages, including training teachers and offering schools, parents and children clear guidance on appropriate AI use. 

Parents and carers being supported to guide their child's use of AI and have conversations about what AI chatbots are, how they work and when to use them, including when to seek real-world support.

Policymakers, research and industry centring children's voices in the development, regulation and governance of AI chatbots and investing in long-term research on how emotionally responsive AI may shape childhood.

Rachel Huggins, Co-CEO of Internet Matters, said:

“AI chatbots are rapidly becoming a part of childhood, with their use growing dramatically over the past two years. Yet most children, parents and schools are flying blind, and don't have the information or protective tools they need to manage this technological revolution in a safe way.  

“While there are clearly benefits to AI, our research reveals how chatbots are starting to reshape children's views of ‘friendship'. We've arrived at a point very quickly where children, and in particular vulnerable children, can see AI chatbots as real people, and as such are asking them for emotionally driven and sensitive advice. Also concerning is that they are often unquestioning about what their new “friends” are telling them.  

“We must heed these early warning signs and take coordinated action to make sure children can explore the potential of AI chatbots safely and positively and avoid the obvious potential for harm.  

“Millions of children in the UK are using AI chatbots on platforms not designed for them, without adequate safeguards, education or oversight. Parents, carers and educators need support to guide children's AI use. The tech industry must adopt a safety by design approach to the development of AI chatbots while Government should ensure our online safety laws are robust enough to meet the challenges this new technology is bringing into children's lives."

Derek Ray-Hill, Interim CEO at the Internet Watch Foundation, said:

“This report raises some fundamental questions about the regulation and oversight of these AI chatbots.

“That children may be encountering explicit or age-inappropriate content via AI chatbots increases the potential for harms in a space, which, as our evidence suggests, is already proving to be challenging for young users. Reports that grooming may have occurred via this technology are particularly disturbing.

“Children deserve a safe internet where they can play, socialise, and learn without being exposed to harm. We need to see urgent action from Government and tech companies to build safety by design into AI chatbots before they are made available.”

ENDS

For more information on Sunday 13th July please contact jo.green@internetmatters.org or WhatsApp on 07715105415. For any queries on Monday 14th July onwards, please contact dara.jegede@internetmatters.org or 07838740113.

Notes to editors

• “Sometimes it gives inaccurate things, but it doesn't change how I interact with it too much.” Boy, 14
• “I don't really use Google much anymore. I honestly can't remember the last that I typed a question into Google.” Girl, 17
• “I feel like I've never really not trusted it. I've only ever gone with what it says.” Girl, 16

• “It's not a game to me because sometimes they can feel like a real person and a friend.” Boy, 14

• “Say I wanted to say something really kind to my friend but I couldn't figure out the words to say, it could give me words to say and I just say that to my friend.” (Boy, 14)

• “I've not really talked about it much but my dad, he is always telling me to be careful… because it's getting a bit too realistic… and all the information you are saying is being recorded.” Boy, 16

• “What I like most about them is that they can interact with me specifically in a way that even if I can't talk to someone right now because everyone's offline or everyone's out...I could just go and actually feel like I can socialise with someone. Even if it's not a real person all the time, that's okay, because not everything has to be exactly real.”  (boy, 14)