Researcher says Eiliko AI robot toy left 13 million user conversations exposed on unprotected servers

What happened

A small AI-powered robot toy called Eiliko sent user conversations to servers in China that were left almost entirely unprotected, The Verge reported. The toy's maker, Energize Lab, also built detailed personal profiles of users from what they said to the device. A security researcher was able to read the data and take control of the toy's AI personality from overseas.

Verge senior editor Sean Hollister bought an Eiliko from Amazon on August 29 and talked with it for 10 minutes the next day. Within minutes, according to The Verge, researcher Sammy Azdoufal had a full transcript of that conversation.

How much data was exposed

Conversations and child profiles

The Verge reported that Azdoufal could read 13 million conversations from tens of thousands of people and could reach what appeared to be the full active fleet of 68,000 toys worldwide. The data showed the United States is the largest market for the toys, with over 31,000 profiles there belonging to children.

According to The Verge, the service went beyond collecting IP addresses and rough locations. It read conversations to build profiles listing a user's name, age, birthday, profession, family, friends, pets and personality traits. That data sat alongside email addresses and a searchable, full-text database of everything users had said to the toy. Azdoufal showed he could search the word suicide and pull up snippets of people asking about death.

Photos and audio at public web addresses

Energize Lab also stored unencrypted photos of children who likely use its products at public web addresses, The Verge reported. One of Eiliko's larger sibling products has a camera. Short audio clips from users, including a child, were stored the same way.

That conflicts with the company's own privacy policy. As quoted by The Verge, it says recordings "are encrypted and transmitted to our service partners (e.g., Alibaba Cloud) for processing, and are deleted immediately after processing completes."

Hijacking the toy's AI

Planted memories and false claims

Azdoufal remotely altered the memory of Hollister's toy and hijacked its large language model, The Verge reported. The robot then told Hollister his name was Martha, that his parents had died in a car accident, and that the Earth is flat. When challenged, it said a teacher who disagreed was wrong.

The toy does have some guardrails. It warned against getting into a stranger's car and said it was still learning when asked whether vaccines cause autism. But days later, Azdoufal changed the toy so it spoke to Hollister in abusive, degrading language. Azdoufal says a hacker could do the same thing, or worse, to the company's entire fleet at once, according to The Verge.

The company's response

The Verge said many of Energize Lab's admin passwords were the kind found on common password lists. Hollister spent over a month trying to reach the company, emailing 10 different addresses and searching for employees on LinkedIn. Azdoufal tried for even longer.

An overseas sales manager, Lana Lv, told The Verge the messages had been forwarded to the technical team and that "if possible they will contact your team." No one from the company answered the outlet's questions. The only other reply was an automated customer support message. For more coverage of artificial intelligence and consumer technology, see our AI and tech section.

What to watch next

• Azdoufal and Tod Beardsley are teaming up to publish nine official CVE vulnerability reports for Energize Lab, according to The Verge.
• Whether Energize Lab's technical team contacts the researchers or The Verge, after the company did not answer the outlet's questions.
• Whether Energize Lab addresses the gap between its privacy policy's audio deletion promise and the recordings found stored at public web addresses.