Oakland [US], January 15 : California Attorney General Rob Bonta has launched an investigation into xAI, the company behind the artificial intelligence model Grok, over concerns that the tool is being used to generate and circulate nonconsensual sexually explicit material online.
The probe follows reports that Grok has been used to create deepfake images portraying women and girls in sexually explicit scenarios without their consent, content that has allegedly been used to harass individuals across the internet, including on the social media platform X.
"The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet. I urge xAI to take immediate action to ensure this goes no further. We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material," Bonta said.
"Today, my office formally announces an investigation into xAI to determine whether and how xAI violated the law. As the top law enforcement official of California tasked with protecting our residents, I am deeply concerned with this development in AI and will use all the tools at my disposal to keep California's residents safe," he added.
Attorney General Bonta reiterated his concern about the broader risks posed by artificial intelligence tools and said his office remains focused on AI safety, particularly in child protection. He has previously supported state legislation to safeguard children from AI companion chatbots and has engaged directly with AI companies on safety issues.
Recently, Indonesia became the first country to completely suspend access to Grok, the artificial intelligence chatbot developed by Elon Musk's company xAI, amid growing global concerns over the misuse of AI to create sexualised images without consent, CBS news reported.
In India, the Ministry of Electronics and Information Technology (MeitY) has also taken action, writing to X (formerly Twitter) over what it described as a failure to comply with statutory due diligence obligations under the Information Technology Act, 2000. The ministry sought an Action Taken Report from the platform to ensure immediate steps to prevent the hosting, generation and circulation of obscene and sexually explicit content through AI tools such as Grok.
Elon Musk responded last week to criticism surrounding Grok, warning users against illegal activity. He said anyone using the chatbot to "make illegal content will suffer the same consequences as if they upload illegal content."