California becomes first state to regulate AI companion chatbots

California Governor Gavin Newsom signed a landmark measure connected Monday that regulates AI companion chatbots, making it the archetypal authorities successful the federation to necessitate AI chatbot operators to instrumentality information protocols for AI companions.

The law, SB 243, is designed to support children and susceptible users from immoderate of the harms associated with AI companion chatbot use. It holds companies — from the large labs similar Meta and OpenAI to much focused companion startups similar Character AI and Replika — legally accountable if their chatbots neglect to conscionable the law’s standards.

SB 243 was introduced successful January by authorities senators Steve Padilla and Josh Becker, and gained momentum aft the death of teen Adam Raine, who died by termination aft conversations with OpenAI’s ChatGPT that progressive discussing and readying his decease and self-harm. The authorities besides responds to leaked internal documents that reportedly showed Meta’s chatbots were allowed to prosecute successful “romantic” and “sensual” chats with children. More recently, a Colorado household has filed suit against role-playing startup Character AI aft their 13-year-old girl took her ain beingness pursuing a bid of problematic and sexualized conversations with the company’s chatbots.

“Emerging exertion similar chatbots and societal media tin inspire, educate, and link – but without existent guardrails, exertion tin besides exploit, mislead, and endanger our kids,” Newsom said successful a statement. “We’ve seen immoderate genuinely horrific and tragic examples of young radical harmed by unregulated tech, and we won’t basal by portion companies proceed without indispensable limits and accountability. We tin proceed to pb successful AI and technology, but we indispensable bash it responsibly — protecting our children each measurement of the way. Our children’s information is not for sale.”

SB 243 volition spell into effect January 1, 2026, and it requires companies to instrumentality definite features specified arsenic property verification, warnings regarding societal media and companion chatbots, and stronger penalties — up to $250,000 per enactment — for those who nett from amerciable deepfakes. Companies besides indispensable found protocols to code termination and self-harm, and stock those protocols, alongside statistic connected however often they provided users with situation halfway prevention notifications, to the Department of Public Health.

Per the bill’s language, platforms indispensable besides marque it wide that immoderate interactions are artificially generated, and chatbots indispensable not correspond themselves arsenic wellness attraction professionals. Companies are required to connection interruption reminders to minors and forestall them from viewing sexually explicit images generated by the chatbot.

Some companies person already begun to instrumentality immoderate safeguards aimed astatine children. For example, OpenAI precocious began rolling retired parental controls, contented protections, and a self-harm detection strategy for children utilizing ChatGPT. Character AI has said that its chatbot includes a disclaimer that each chats are AI-generated and fictionalized.

Newsom’s signing of this instrumentality comes aft the politician besides passed SB 53, different first-in-the-nation measure that sets caller transparency requirements connected ample AI companies. The measure mandates that ample AI labs, similar OpenAI, Anthropic, Meta, and Google DeepMind, beryllium transparent astir information protocols. It besides ensures whistleblower protections for employees astatine those companies.

Other states, similar Illinois, Nevada, and Utah, person passed laws to restrict oregon outright prohibition the usage of AI chatbots arsenic a substitute for licensed intelligence wellness care.

TechCrunch has reached retired to Character AI, Meta, OpenAI, and Replika for comment.