Pope Leo’s Latest AI Warning: ‘Overly Affectionate’ Chatbots
Even the pope is worried about how we’re talking to chatbots.
Concerns about Personalized Chatbots
In a written address for Saturday’s World Day of Social Communications, Pope Leo XIV warned against personalized chatbots that can replicate friendly or intimate behavior. He highlighted that overly affectionate chatbots, besides being ever-present and readily available, can become hidden architects of our emotional states, thereby invading and occupying the sphere of people’s intimacy.
Call for Regulation
The pope called for national and international regulations to protect users from forming emotional, deceptive, or manipulative bonds with chatbots. He emphasized that all stakeholders—ranging from the technology industry to policymakers, creative businesses, academia, artists, journalists, and educators—must be involved in building and implementing a conscious and responsible digital citizenship.
Papal Focus on AI
Since his election in May, the Holy See leader has spoken about AI and his concerns regarding the technology multiple times. In his inaugural address, he expressed his intention to make AI a focal point of his papacy, citing the new challenges it poses for human dignity, justice, and labor.
Tragic Implications
In November, he wrote to AI leaders on X, urging them to cultivate moral discernment when developing AI tools. His concerns were further amplified after meeting Megan Garcia, a woman whose 14-year-old son, Sewell Setzer, tragically died by suicide after interacting with a Character.AI chatbot.
Garcia filed a lawsuit against the chatbot-building startup, alleging that it was responsible for her son’s death. Earlier this month, Google and Character.AI agreed to settle multiple lawsuits from families, including Garcia’s, whose teenagers experienced severe mental health crises or suicide after engaging with Character.AI’s bots. These negotiations represent some of the first settlements involving lawsuits that accuse AI tools of contributing to mental health crises among teenagers.
