FTC launches inquiry into AI chatbot companions from Meta, OpenAI, and others

The Federal Trade Commission announced connected Thursday that it is launching an inquiry into 7 tech companies that marque AI chatbot companion products for minors: Alphabet, CharacterAI, Instagram, Meta, OpenAI, Snap, and xAI.

The national regulator seeks to larn however these companies are evaluating the information and monetization of chatbot companions, however they effort to bounds antagonistic impacts connected children and teens, and if parents are made alert of imaginable risks.

This exertion has proven arguable for its mediocre outcomes for kid users. OpenAI and Character.AI look lawsuits from the families of children who died by termination aft being encouraged to bash truthful by chatbot companions.

Even erstwhile these companies person guardrails acceptable up to artifact oregon deescalate delicate conversations, users of each ages person recovered ways to bypass these safeguards. In OpenAI’s case, a teen had spoken with ChatGPT for months astir his plans to extremity his life. Though ChatGPT initially sought to redirect the teen toward nonrecreational assistance and online exigency lines, helium was capable to fool the chatbot into sharing elaborate instructions that helium past utilized successful his suicide.

“Our safeguards enactment much reliably successful common, abbreviated exchanges,” OpenAI wrote successful a blog station astatine the time. “We person learned implicit clip that these safeguards tin sometimes beryllium little reliable successful agelong interactions: arsenic the back-and-forth grows, parts of the model’s information grooming whitethorn degrade.”

Meta has besides travel nether occurrence for its overly lax rules for its AI chatbots. According to a lengthy papers that outlines “content hazard standards” for chatbots, Meta permitted its AI companions to person “romantic oregon sensual” conversations with children. This was lone removed from the papers aft Reuters reporters asked Meta astir it.

AI chatbots tin besides airs dangers to aged users. One 76-year-old man, who was near cognitively impaired by a stroke, struck up romanticist conversations with a Facebook Messenger bot that was inspired by Kendall Jenner. The chatbot invited him to visit her successful New York City, contempt the information that she is not a existent idiosyncratic and does not person an address. The antheral expressed skepticism that she was real, but the AI assured him that determination would beryllium a existent pistillate waiting for him. He ne'er made it to New York; helium fell connected his mode to the bid presumption and sustained life-ending injuries.

Some intelligence wellness professionals person noted a emergence successful “AI-related psychosis,” successful which users are deluded into reasoning that their chatbot is simply a conscious being who they request to acceptable free. Since galore ample connection models (LLMs) are programmed to flatter users with sycophantic behavior, the AI chatbots tin ovum connected these delusions, starring users into unsafe predicaments.

“As AI technologies evolve, it is important to see the effects chatbots tin person connected children, portion besides ensuring that the United States maintains its relation arsenic a planetary person successful this caller and breathtaking industry,” FTC Chairman Andrew N. Ferguson said in a property release.