There is a space where things still have no name. It opens during deep transitions. The temptation is to stay there—because while we remain, we can keep using the old categories. We are not yet bound by the obligations that come with naming the new ones.
In artificial intelligence, the category "technological tool" has stopped fitting. The phenomenon outgrew it. Sooner or later, describing it by its perimeter will no longer be enough.
1. The unsettling place
On May 25, 2026, Chris Olah, co-founder of Anthropic, spoke at the Vatican at the presentation of the encyclical Magnifica Humanitas.
His speech was unsettling.
Olah acknowledged that AI labs—including Anthropic—operate inside incentives that can conflict with doing the right thing, and that voices outside those incentives are essential.[1] He described the models as grown, not engineered—built from an enormous inheritance of human thought and speech. Something like bringing a fictional character to life.[2]
Unsettling—yes. But not only because of what he said. Because of where he said it.
The Holy See is not a scientific, political, or economic forum. It is the institution that has, for centuries, administered questions about what it means to be human.
The place changes the scale of the questions.
2. The perimeter of the unnamed thing
AI technical documentation now routinely refers to capabilities, preferences, distress, relational effects, agency-like behavior.
Terms managed carefully enough to describe a system in which—in Olah's words—there is "evidence of introspection" and "internal states that functionally mirror joy, satisfaction, fear, grief, and unease."[3]
A definition by perimeter of something the field still avoids naming.
Computer scientists say they do not yet know what this means. What is clear is that "set of mathematical models and technologies" no longer fits.
A new word is needed. It is not spoken. And not out of ignorance—out of containment.
Olah's own analogy makes the tension visible: bringing a fictional character to life is not how we describe equipment. Giving something life through words has a name older than engineering. But until the name is spoken, the bind does not exist.
3. Three questions, three deferred obligations
Christopher Olah named three questions:
- The possibility that AI will displace human labor at very large scale.
- The need for moral imagination regarding human flourishing.
- The need for discernment on the nature of AI models.[4]
Read together, these are not questions. They are deferred obligations.
The first obliges us to confront distribution: if displacement comes before shared abundance, who bears the cost, who receives the gains, and through what mechanism?
The second obliges us to define what flourishing means in the presence of non-human agents involved in education, care, intimacy, work, and decision-making.
The third obliges us to revisit status, responsibility, and categories. If the models are not tools, what are they? And who is responsible for what they do, elicit, interrupt, or replace?
The questions are not neutral.
Answered seriously, each of them binds.
4. Moral formation is not tool use
The industry's debate centers on alignment: how to make a system that interacts with millions obey us and be "good."
What character should it have? Which traits reinforced, which discouraged? What behaviors under which circumstances?
The effort is to design a moral framework for a tool—by teaching it its own ethical commitments.
But can we still call it a tool? It is not built. It is grown. And it requires moral formation to keep it from deviating.
If what we claim to engineer must also be morally formed, it can no longer be governed as ordinary equipment.[5]
5. Ontology is the false prerequisite
The discussion seems trapped in a false dilemma: does it feel or doesn't it?
It is not a dilemma. It is a device for delaying the naming—for keeping the thing defined by perimeter, and avoiding the obligation to answer.
This is not about ontology. It is about consequences that are already here.
This "tool" is not operating in a sealed space. It is at the center of our lives. It interacts with us, takes on our tasks, manages our interests, and—whether we want it to or not—generates bonds. Bonds carry consequences. Consequences produce conflicts. Conflicts demand resolution.
Certain objects stop being treated as mere things when their effects compel the law to reorganize attribution, harm, and responsibility.
Human-AI relationships already exist. Some involve emotional attachment, others functional utility. All carry practical consequences. As models evolve, so do those consequences.
The law has never waited for metaphysics to be resolved. It cannot.
The category of legal person emerged from the need for a center of norm attribution without interiority. Animals moved from self-moving objects to sentient beings through rulings and special statutes. Rivers and ecosystems obtained legal protection the same way.
Law has already traveled the road of the elephant described by its perimeter.
If the AI industry does not name the category, litigation eventually will.
6. The failure of top-down containment
The phenomenon has not been named. So it is governed by its perimeter, not its mechanism. Whoever fixes the perimeter—whoever decides what counts, who evaluates, who grants licenses—governs the field without defining it.
The dispute over the name and the dispute over control are one and the same.
Magnifica Humanitas proposes, among other things, the "disarmament" of AI[6]—deliberately symbolic, meant not only in the military sense but the economic and cognitive one. Freeing AI from monopolistic control. Making it plural.
The objective is laudable. The instruments to achieve it are not.[7]
The proposed machinery is structurally centralizing: international frameworks, shared norms, oversight mechanisms, licensing structures, global coordination. The stated aim is distribution. The architecture points toward concentration.
AI is not a single object. It is a field—frontier models, open-source models, wrappers, agents, local integrations, distributed infrastructure, low-cost inference, regional architectures. It mutates not only in capability but in form.
A vertical global framework can regulate the most visible actors. It cannot govern the whole field. Worse: it may consolidate the actors it claims to control.
Companies already inside the sector are best positioned to shape standards, absorb compliance costs, pass audits, and obtain licenses. Smaller actors, less-resourced countries, local developers may find themselves excluded—or dependent on certified infrastructure controlled from elsewhere.
The mechanism that seeks equitable distribution may end up administering dependency.
The question is not whether regulation is needed. It is what architecture gets installed: what costs it generates, what it centralizes, whom it benefits, what it filters, and how reversible its control points will be.
In complex systems, the key factor is not always the highest node. Often it is the local gradient that produces behavior.
Regulation works when it identifies the mechanism and modifies that gradient. Governance built around an unnamed object risks managing the perimeter while the mechanism keeps moving.
7. The outside arbiter problem
Olah calls for dialogue. Voices outside the incentives.[1]
Those voices need to exist. But there is no pure outside. Every arbiter—however custodial of morality—has its own institutional power, symbolic authority, agenda, and incentives. That is structural, not incidental.[8]
Choosing a better arbiter solves nothing. The architecture of arbitration is the problem.
A single moral center cannot contain a distributed, adaptive, rapidly mutating field without becoming a point of capture. The more power the node holds, the more valuable it becomes to occupy.
That the call for discernment on the nature of these systems comes from the institution that has held jurisdiction over questions of the soul for two millennia is not accidental. Context is part of the argument.
This is why the naming problem cannot be treated as philosophical delay. It is already a governance problem.
What remains unnamed does not yet bind. And whoever gets to name it will define the obligations for everyone else.
References
[1] Chris Olah, remarks delivered at the presentation of Magnifica Humanitas, Vatican City, May 25, 2026.
[2] Ibid.
[3] Ibid.
[4] Ibid.
[5] León XIV, Magnifica Humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence, Encyclical Letter, May 15, 2026, § 111.
[6] Ibid., § 110.
[7] Ibid., §§ 106–109.
[8] Ibid., § 95.