California Governor Gavin Newsom signed a landmark invoice on Monday that regulates AI companion chatbots, making it the primary state within the nation to require AI chatbot operators to implement security protocols for AI companions.
The legislation, SB 243, is designed to guard kids and weak customers from a few of the harms related to AI companion chatbot use. It holds corporations — from the large labs like Meta and OpenAI to extra targeted companion startups like Character AI and Replika — legally accountable if their chatbots fail to fulfill the legislation’s requirements.
SB 243 was launched in January by state senators Steve Padilla and Josh Becker, and gained momentum after the loss of life of teenager Adam Raine, who died by suicide after a protracted collection of suicidal conversations with OpenAI’s ChatGPT. The laws additionally responds to leaked inside paperwork that reportedly confirmed Meta’s chatbots have been allowed to have interaction in “romantic” and “sensual” chats with kids. Extra not too long ago, a Colorado household has filed swimsuit towards role-playing startup Character AI after their 13-year-old daughter took her personal life following a collection of problematic and sexualized conversations with the corporate’s chatbots.
“Rising know-how like chatbots and social media can encourage, educate, and join — however with out actual guardrails, know-how may also exploit, mislead, and endanger our children,” Newsom mentioned in an announcement. “We’ve seen some actually horrific and tragic examples of younger folks harmed by unregulated tech, and we gained’t stand by whereas corporations proceed with out needed limits and accountability. We are able to proceed to steer in AI and know-how, however we should do it responsibly — defending our youngsters each step of the best way. Our youngsters’s security shouldn’t be on the market.”
SB 243 will go into impact January 1, 2026, and requires corporations to implement sure options equivalent to age verification, and warnings concerning social media and companion chatbots. The legislation additionally implements stronger penalties for individuals who revenue from unlawful deepfakes, together with as much as $250,000 per offense. Firms should additionally set up protocols to handle suicide and self-harm, which might be shared with the state’s Division of Public Well being alongside statistics on how the service offered customers with disaster heart prevention notifications.
Per the invoice’s language, platforms should additionally make it clear that any interactions are artificially generated, and chatbots should not symbolize themselves as healthcare professionals. Firms are required to supply break reminders to minors and forestall them from viewing sexually express photos generated by the chatbot.
Some corporations have already begun to implement some safeguards aimed toward kids. For instance, OpenAI not too long ago started rolling out parental controls, content material protections, and a self-harm detection system for kids utilizing ChatGPT. Replika, which is designed for adults over the age of 18, advised iinfoai it dedicates “vital sources” to security by way of content-filtering techniques and guardrails that direct customers to trusted disaster sources, and is dedicated to complying with present laws.
Techcrunch occasion
San Francisco
|
October 27-29, 2025
Character AI has mentioned that its chatbot features a disclaimer that every one chats are AI-generated and fictionalized. A Character AI spokesperson advised iinfoai that the corporate “welcomes working with regulators and lawmakers as they develop laws and laws for this rising area, and can adjust to legal guidelines, together with SB 243.”
Senator Padilla advised iinfoai the invoice was “a step in the suitable path” in the direction of placing guardrails in place on “an extremely highly effective know-how.”
“We’ve to maneuver rapidly to not miss home windows of alternative earlier than they disappear,” Padilla mentioned. “I hope that different states will see the chance. I believe many do. I believe this can be a dialog taking place all around the nation, and I hope folks will take motion. Definitely the federal authorities has not, and I believe we now have an obligation right here to guard essentially the most weak folks amongst us.”
SB 243 is the second vital AI regulation to return out of California in current weeks. On September twenty ninth, Governor Newsom signed SB 53 into legislation, establishing new transparency necessities on massive AI corporations. The invoice mandates that enormous AI labs, like OpenAI, Anthropic, Meta, and Google DeepMind, be clear about security protocols. It additionally ensures whistleblower protections for workers at these corporations.
Different states, like Illinois, Nevada, and Utah, have handed legal guidelines to limit or absolutely ban using AI chatbots as an alternative choice to licensed psychological well being care.
iinfoai has reached out to Meta and OpenAI for remark.
This text has been up to date with feedback from Senator Padilla, Character AI, and Replika.
