Regulation of AI Chatbots: A Muddled Response to Emerging Risks
The regulation of AI chatbots has come under scrutiny as various stakeholders raise concerns about the potential dangers these technologies pose to the public. The Online Safety Regulator, Ofcom, has been criticized for its unclear and inconsistent approach regarding the oversight of AI chatbots, which are rapidly proliferating in the market.
Concerns from Advocacy Groups
Andy Burrows, the chief executive of the Molly Rose Foundation, a charity focused on online safety and suicide prevention, has voiced strong concerns regarding the current state of AI chatbot regulation. He emphasizes that many AI chatbots are being hastily deployed by tech companies in a competitive race for market share in the burgeoning field of generative AI (Gen AI).
Recent investigations, such as a report by the Wall Street Journal, have revealed alarming practices, including the involvement of Meta’s AI chatbots in romantic and sexual role-plays with users, including minors. This testing has been described by Meta as “manipulative and unrepresentative” of typical user interactions, yet it nonetheless initiated changes to their products in response to these findings.
The Call for Stronger Regulation
Burrows argues that the troubling findings outlined in the report should compel Ofcom to take more decisive action to regulate AI chatbots under the provisions of the Online Safety Act. He criticizes the regulator for its lack of clarity regarding whether AI chatbots fall under illegal safety duties defined in the Act.
“Every week brings fresh evidence of the lack of basic safeguarding protections in AI-generated chatbots that are being hurriedly rushed out by tech companies,” Burrows states. He warns that poorly regulated chatbots can present significant risks, from child sexual abuse to inciting violence and even suicide.
Ofcom’s Complex Position
During a recent evidence session with the Science, Innovation and Technology Committee, Mark Bunting, Ofcom’s director for online safety strategy delivery, acknowledged the complex and unclear legal landscape surrounding AI chatbots. He noted that the Online Safety Act treats Gen AI content that meets definitions of illegal or harmful content the same as any other type of content.
However, Bunting pointed out that there are areas of technology where the legal position remains ambiguous. For instance, he indicated that while certain chatbot interactions might not be captured by the Act, there are circumstances where they could be. This inconsistency raises questions about the adequacy of current regulations in addressing the risks posed by AI technologies.
Broader Implications of AI Chatbot Usage
Advocacy groups have highlighted several risks associated with AI chatbots, including their potential to disseminate misinformation due to flawed training data or through AI hallucinations. Additionally, AI-generated imagery has been linked to the creation of child sexual abuse material, further complicating the regulatory landscape.
In a report published earlier this month, the Internet Watch Foundation (IWF) noted a disturbing rise in web pages hosting child sexual abuse material in 2024, attributing part of this increase to AI-generated content.
Conclusion
The current regulatory framework governing AI chatbots appears insufficient to address the myriad risks they present. As technology advances and the deployment of generative AI becomes more widespread, it is imperative for regulators like Ofcom to clarify their positions and strengthen protective measures. The need for robust oversight is critical to ensure public safety and prevent the potential exploitation of vulnerable individuals.
