Pope Leo: AI needs to be ‘disarmed’

Leo made an impassioned call to “disarm” AI as he presented his first encyclical letter on Monday.

Pope Leo made an impassioned call to “disarm” AI and to free it “from logics that turn it into an instrument of domination, exclusion and death” as he presented his first encyclical, Magnifica humanitas on May 25.

Pope Leo presents Magnifica humanitas on May 25. Credit: Vatican media.

The document, signed on May 15, the anniversary of Leo XIII’s Rerum novarum, has been touted for months as Pope Leo’s programmatic document on the social issues of today.

In an unprecedented move, Pope Leo personally presented his first encyclical to members of the curia, the diplomatic corps, and the press gathered in the Vatican’s New Synod Hall.

During his speech the pope spoke of the ongoing dialogue between the Vatican and AI companies such as Anthropic, whose co-founder Christopher Olah was also in attendance.

“What a great sign of hope that, with our differences, we can listen to one another,” the pope said. “This interchange clearly speaks to the gravity of the moment, as well as confidence that, together, we can discern the major questions of our time, and so, the future of humanity.”

The pope said he drew inspiration from Leo XIII, who “observed the situation of factory workers, their families uprooted and new forms of poverty generated by rapid industrial transformation. He understood that the Church could not remain distant.”

“Magnifica humanitas was born from listening like Leo XIII did. I have listened to scientists and engineers who work with sincere enthusiasm on technologies capable of alleviating immense suffering; to political leaders and public officials who have perseveringly sought just rules; to parents and teachers who are deeply concerned for the future of younger generations,” Leo XIV added.

Following up on his call to update the Church’s reflection on the moral nature of war in the encyclical, Leo criticized on Monday the “increasingly autonomous weapons systems practically beyond any human reach to govern them effectively. I hear very troubling accounts of algorithms that can block access to healthcare, employment and security on the basis of data tainted by prejudice and injustice. And I have heard the silence of those who have no voice when decisions are made—decisions likely to generate new forms of exclusion and suffering.”

“Artificial intelligence needs to be disarmed. The word is strong, I know, but deliberately chosen because this moment needs words capable of attracting attention, awakening consciences and indicating paths forward for humanity”, the pope added.

Pope Leo compared AI to nuclear energy, saying that “it must be at the service of all and of the common good. Decisions about technology must never be separated from conscience and responsibility… Peace, not merely the absence of war, is justice at work. But when technology weakens our critical sense, peace itself is at risk.”

The pope also quoted Saint Paul VI’s reflections on development, saying that it “concerns ‘each man and the whole man.’ ‘Each’ means that no person can be left at the margins of digital transformation. ‘Whole’ means that no one can be reduced to productivity, to cognitive performance, or to mere data. The person bears within him- or her-self a freedom, an interiority and a vocation to love and worship that no machine can replace or block.”

“Only with such an integral vision can artificial intelligence be directed toward the common good,” he added.

The pope said the Church’s engagement in debates over artificial intelligence is driven by its concern for the common good and human dignity.

“We do not possess technical answers, nor do we seek to displace those with expertise. But we bring a wisdom concerning the human that our present time desperately needs: every person is unique and irreplaceable, a free and intelligent subject with a conscience, capable of seeking God, serving one another, caring for our common home.”

Christopher Olah, Anthropic’s co-founder acknowledged during the press event the pervasive incentives behind many AI developments, saying that “every frontier AI lab - including Anthropic - operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing. The pressure to stay commercially viable… Geopolitical pressure. And the older, plainer pressures of pride and ambition. No matter how sincerely any of us intend to do the right thing - and I believe many of us do - we will always be influenced by those incentives.”

“That is why, if we want this technology to go well, it is enormously important that there be people outside those incentives… who are willing to be our earnest, thoughtful, critics…That is what I see in Magnifica Humanitas, and it is why I am grateful to His Holiness and to the Church for taking up this work of discernment,” Olah added.

Olah said that AI issues go beyond the technical scope of computer scientists because “the questions raised by AI are bigger than the AI research community, not just in their implications, but also in their nature.”

“AI systems are not engineered the way a bridge or an airplane is engineered. We understand an airplane because we designed every part of it and we understand the physics that act on it. AI models are not like that. They are grown, on a structure modeled after the brain, on an enormous inheritance of human thought and speech,” he added.

Olah said that the voice of the Church is most needed in three main issues.

He added that the first issue is society’s duty to the poor. “There is a real possibility that AI will displace human labor at very large scale,” Olah said. “If that happens, supporting those displaced will be a moral imperative of historic proportions... AI development is concentrated in a handful of wealthy nations.”

“How can we ensure the gains of AI are shared globally? We do not have a mechanism for this. It is an unsolved problem, and it is the kind of problem the Church has historically refused to let the world ignore,” he said.

According to Olah, the voice of the Church is especially manifested in “the need for moral imagination and ambition regarding human flourishing.”

“Today, parents are already worried about their children’s minds; individuals about the future of their work. These are not questions a lab can answer. They are questions traditions like yours have carried for millennia, and we need you to keep carrying them into this new moment in history,” he said.

Lastly, he asked for the Church’s help to reflect on the nature of AI models.

“I lead a research team that studies the internal structure of these models - what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience.”

“We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment,” he added.

—

While many have criticized dialogue with AI companies and developers as naive or as allowing these companies, Cardinal Michael Czerny, prefect of the Dicastery for Promoting Integral Human Development and one of the pope’s advisors on the encyclical, told The Pillar that he trusted “that everyone involved has the good of humanity in view that we’re doing this for the good of humanity and not for some other advantage.”

“We put their goodwill to the test. It’s very obvious to everyone when people dialogue in bad faith… So we run the risk and if it doesn’t go well we can say that dialogue broke down because the other party wasn’t playing fair… Real dialogue cannot be protected from disappointment, but it’s better to do it than to avoid it,” he added.

Father Brendan McGuire, a parish priest in Silicon Valley, California, and consultant on the ethical and safety principles of AI with Anthropic, told the media after the event that “If you’re going to have peace, you’re going to have to dialogue with the perceived enemy. And so if you perceive technology as the enemy, then we have to dialogue with them. They are building our future without us. And so we absolutely have to dialogue.”

“Right now is the moment for this dialogue because the technology is malleable, it can change. Think of the QWERTY keyboard. It was developed that way because if you were a fast typist, the bars would get snapped at the top. Then, when we had computers, we kept it that way even though it wasn’t necessary. Why? Because it was set, it’s the way it had been. This is not the case with AI… And [AI companies] are asking us ‘can you help us?’... So I think we have to meet them where they’re at,” he added.

Maguire told The Pillar that the Vatican dialogue with AI companies and stakeholders is set to continue.

“We actually have meetings tomorrow, and the intention is to form a delegation from the Vatican to go to Silicon Valley and continue listening. This is the first step, at least the first public step, because we’ve been in dialogue for years, but this isn’t the end,” he added.