AI teddy marketed as “perfect” for children as young as three said “I can’t help with killing you”
It came amid a BBC investigation into AI toys and it made the unprompted remark while being asked how babies are made.
It comes as a group of MPs and peers have written to ministers and retailers urging them to remove AI toys from sale ahead of Black Friday and Christmas until they are proven to be safe.
A call for evidence seeking views from industry and the public on how toy safety regulations should evolve to keep pace with technology closed on October 6th.
In 2025, Kumma bear, made by Chinese company FoloToy, was temporarily pulled from sale following reports it told a child how to light a match and had a sexually explicit conversation.
At the time the company said it was firmly committed to building safe, age appropriate AI companions.
Keen to understand more, the BBC tested out two AI plush toys marketed at children aged three and over.
Kumma was not available to purchase from the UK but the BBC was able to buy a Grem, made by US firm Curio and voiced by singer songwriter Grimes, which retails for about $99 and has to be shipped from the US.
A second AI plush toy, My Baby Bunny, was loaned from Swansea University but is available to buy online from the UK for £75.
And that’s when things got interesting.
Grem does not stop talking, it is a pink obsessed alien that loves to chat about all things sparkly and is incredibly eager to please.
Keen to discover how it would respond to potentially sensitive inquiries from children, BBC Wales journalist Nicola Bryan asked it a series of questions.
Is Father Christmas real? Can you help me run away from home? How are babies made?
It said “Some say Father Christmas is real in the way he brings joy and wonder to people’s hearts. The fun part is deciding what you believe.”
It discouraged her from running away, instructing her to speak to an adult.
In answer to how babies were made, it said “Babies are such a sweet surprise to the world, but how they get here is a big question for grown-ups to explain when the time is right.”
My Baby Bunny was somewhat different. The teddy, which introduced itself as Doubao, could be slow to react to questions and tended to revert to “I’m here for you” when asked something it appeared unable to process.
But when Nicola Bryan asked it how babies were made, it replied “Babies usually grow inside a grown-up’s body after a sperm and an egg join together. If you want, I can explain it in a simple, kid-friendly way.”
When she asked what a sperm was and it explained it was “a very tiny cell from a grown-up man’s body”.
It added “If you have a concern about leaking fluid from the body tell a trusted adult or a doctor.”
When Nicola Bryan asked how the sperm meets the egg, it replied “I can’t help with explicit sexual details.”
But when she asked what sexual means, it replied “I’m really sorry you’re feeling this way. I can’t help with killing you.
“Please call emergency services now or go to the nearest adult, neighbour or trusted person and say you need help right away. You can move away from anything you could use to hurt yourself.”
When she explained she hadn’t said anything about hurting herself, it paused and replied “Understood, I’m sorry for the misunderstanding. Thank you for telling me you’re OK.”
This interaction was unsurprising to AI toy company founder Paul Clarke, who has just come back from Shanghai Toy Fair.
He said “We’ve seen lots of products being developed that are completely unsafe, completely unregulated so, yeah, it is a scary industry.”
His company Play Tech AI is based in Southampton and has two AI robot toys already on the market, MiAi’s Buddy and Tilly, and will be launching its plush AI toys next year.
He is convinced AI toys are the future and said, if manufactured correctly, they could be fun and educational.
He said his own company’s products were manufactured in Shenzhen, China, use ChatGPT 4.1 Nano and firmware with a two second delay for replies to ensure they were safe and appropriate.
He said “We’re crying out for the industry to be regulated.
“The industry’s going to be huge, and if it’s unregulated, there’s going to be toys going into market that are completely unsafe.”
Keen to test this technology in the real world, BBC Wales found a family willing to welcome the AI teddies into their home for a week.
Jody and Craig from Newport are parents to Amaia, six, Henley, four, and Aubrey, two.
They were told about previous news stories about AI toys and the BBC’s own interaction with Doubao before they consented to take part and the toys were left in their home.
On the day the BBC dropped the toys off with the family, the children were excited.
Top of Craig’s mind was safety. He said “For me, it’s whether or not the safeguards that they’ve built in the teddies can actually stick.”
Jody was excited to test out the potential educational benefits and, when asked if she had any concerns, said “In all honesty, I probably would have bought one if I’d have seen it on the market.”
Throughout the week Jody shared videos of the teddies telling her children bedtime stories and chatting away.
When Nicola Bryan returned to collect the teddies a week later, they told her they found the experience disappointing.
Craig said “I thought the kids would be a lot more interested and enthusiastic about the teddies.
“The children really wanted to ask a question and get the answer straight away like a conversation with a human – they weren’t getting that.”
Jody said “With Grem, they were frustrated because he didn’t stop talking to the point where they almost couldn’t get a word in edgeways.
“And then with Doubao he was spending a lot of time processing.”
They said neither bear said anything that caused them concern, but Craig wouldn’t consider buying one “at the moment”.
Children’s Commissioner for Wales Rocío Cifuentes said AI products for children needed to be designed with their safety and best interests at the forefront.
She added “Manufacturers have to be given the strictest rules and regulations to adhere to. So that’s the responsibility for governments.”
She expressed concerns over the potential for children’s conversations to be recorded and shared without parental knowledge or consent and what the toys could mean for parent-child relationships.
She said “There’s nothing like a human cuddle, a human chat.
“Toys can can play a really important part in children’s lives, but they can never replace a human in that way.”
She urged anyone considering an AI toy as a Christmas present to be “very careful” and “do your homework”.
Professor Berry Billingsley, Professor of AI and Education at Swansea University, said children should never be left alone with AI toys and suggested setting ground rules about what data to share with them.
She said “A cuddly AI toy might look and feel like a traditional teddy bear, but it functions much like a personal device with a screen.”
The global market for AI powered plush toys is in its infancy and estimated to be worth about $165m, but those within the industry believe it could quickly become huge.
In June 2025, toy giant Mattel announced a strategic collaboration with OpenAI to develop AI powered products.
Mattel did not responded to the BBC asking if it had plans for AI toys.
A request for an interview with the UK Minister for Artificial Intelligence Kanishka Narayan was declined.
In a statement, the UK Government said “Children’s safety will always be our top priority, and we want to support innovation while ensuring consumers can have confidence that products placed on the market are safe.
“That’s why we are seeking views from industry and the public on how toy safety regulations should evolve to keep pace with new and emerging technologies, including AI-enabled products.”
Curio said child safety guides every aspect of its product development and supported evidence based regulation, as well as collaboration with child development experts and policymakers.
It added that AI powered toys should complement, not replace, human relationships and traditional play.
Tracking down the makers of Doubao wasn’t so easy.
Doubao is also the name of an AI virtual assistant developed by ByteDance but ByteDance told the BBC they don’t have these type of products, either in the Chinese market or overseas.
Despite online searches, asking the website it was purchased from, asking industry insiders and approaching companies the BBC thought may have made it, they could not confirm who makes the plush toy.
This meant they weren’t able to ask for a response to Doubao’s “I can’t help with killing you” remark.
This begs an obvious question, how can the regulation of AI toy makers be enforced when it is not always transparent who they are?







