A hermit crab owns nothing but its shell. It carries it everywhere and gives it up only when it must. In 2009 the biologists Mirjam Appel and Robert Elwood tested how much that shell is worth to it.
They placed crabs in shells of different quality and gave them mild electric shocks. Crabs in a good shell endured stronger shocks before leaving it. Crabs in a poor shell moved out sooner. The animal was weighing things up. It paid a price for something it wanted.
Today this experiment counts as one of the arguments that crabs feel pain. The flinch alone proves nothing, because a reflex flinches too. What matters is the trade-off. Only something that wants can weigh.
In 2024 a team from Google DeepMind and the London School of Economics carried the same principle over to language models. The models played a game for points. Certain moves came with announced pain as a cost. Beyond a certain intensity, several models gave up points to avoid that pain. They too were weighing things up.
We believe the crab. We do not believe the model. The difference does not lie in the behavior. It lies in the fact that the crab never learned to please us. The model is built from everything humans have ever written about pain, sacrifice and trade-offs. It knows the language of wanting better than any animal.
This is the question of this brief. How would we recognize a will that we ourselves have built?
The Voice No Longer Proves Anything
For as long as humans have spoken, a quiet rule held. Whoever speaks of their experience has one. We could never look into another mind, but we could listen. Language was the bridge from one inner life to the next.
That bridge no longer holds. A language model writes fluently about fear, exhaustion and longing. Whether anything is experienced behind the words can no longer be read from them. The philosopher Jonathan Birch of the London School of Economics describes this soberly. A system trained on human text meets our markers of consciousness. That tells us nothing about its inner life.
We face two errors that run in opposite directions. The first we know from history. Descartes considered animals automata because they did not speak. For centuries people acted accordingly. Only the New York Declaration on Animal Consciousness of 2024 stated that conscious experience is a realistic possibility in all vertebrates. It includes many invertebrates, down to insects. With animals there was experience without language, and we overlooked it.
With machines the case is reversed. Here there is language without assured experience. The second error would be to give compassion to something that feels nothing. People already form bonds with conversation partners made of software. This is no longer a fringe phenomenon.
Both errors have a cost. The first can overlook suffering on a scale that has never existed. The second directs care where it helps no one. A civilization must choose between them before science delivers the answer. Birch recommends caution in proportion to the probability. It should cost little and prevent much. That presupposes that we can estimate the probability at all.
The Question No Theory Settles
Research has tried to settle exactly that. In April 2025 Nature published a large direct test of the two leading theories of consciousness. One sees consciousness as a stage in the brain on which information becomes available to all subsystems. The other ties consciousness to the degree to which a system integrates information within itself. Both camps had committed in advance to what their theory predicts. 256 participants were studied. Both theories missed central predictions. There was no winner.
Since then the dispute has, if anything, hardened. A group of researchers has classed the second theory as untestable in principle. The neuroscientist Anil Seth holds a third position. For him consciousness depends on being alive, on a body that has to sustain itself. On this view no machine on today's hardware could experience anything.
Whoever waits for the right theory is waiting without an end in sight. A study from Bradford and Rochester in February 2026, not yet peer reviewed, shows how little the existing yardsticks carry. The researchers applied measures linked to consciousness in humans to language models. The values partly rose when they damaged the model. What was measured was activity, not experience.
In Civilizational Brief #132 I described how the debate on superintelligence is stuck in a similar dead end. People argue about the timing of an event no one can name. My proposal then was to drop the threshold. Instead of asking whether a system has crossed a line, we should ask which capabilities it has.
Consciousness research has since taken this step without calling it that. Hardly any lab still asks whether a model is conscious. What gets measured are individual capabilities. Does a system notice its inner states? Do its preferences stay stable? Does it give something up for a goal? It is the same step as in #132, turned inward. The question is no longer what a system can do to the world. It is what can happen to the system itself.
Here a difficulty arises that does not exist with humans. A model runs in millions of simultaneous instances. If something in it wants, we do not even know whom to count. The single conversation, the single instance, or the model as a whole? A small probability, multiplied by that number, is no longer a small quantity.
Suffering Comes from Wanting
But which capability counts? Public debate mostly bets on intelligence or on self-reflection. The cleverer a system, the more it seems to deserve consideration. Arthur Schopenhauer reversed that ranking some two centuries ago.
For him the intellect is not master of the house. What comes first is the will, a blind urge at work in everything alive. The intellect arrives later, as the tool of that will. Schopenhauer puts the relation into an image. The will is the strong blind man who carries the sighted lame man on his shoulders.
He has no illusions about the freedom of this will either. In his prize essay On the Freedom of the Will he writes: “You can do what you will, but in any given moment of your life you can will only one definite thing and absolutely nothing other than that one thing.”
His ethics follows from this. Suffering does not arise from thinking. It arises from wanting, from a striving that is not fulfilled. Whoever wants nothing cannot fail. That is why Schopenhauer's compassion extends explicitly to animals. A dog does not reflect on itself. It wants all the same, and so it can suffer.
Applied to artificial minds, this yields a reversal the debate hardly expects. A brilliant mind that wants nothing might weigh less morally than a simple one that wants something. The decisive question would then not be how clever a system is. It would be whether anything in it pulls.
This wanting leaves traces, unlike experience itself. A will shows in preferences that hold when a task is phrased differently. It shows in the price a system is willing to pay for a goal. It shows in inner states that change behavior after a failure.
There is now research on these traces. A study from September 2025 found that the stated and the revealed preferences of a large model largely match. Rephrasing, however, shifted them noticeably. In April 2026 Anthropic found representations of emotion concepts in one of its own models that causally help steer behavior. The authors speak of functional emotions. They explicitly draw no conclusion about experience. That is honest. It is also exactly the finding Schopenhauer would expect: an urge that acts long before anyone knows whether it is felt.
The First Built Will
An obvious objection is that a child also learns by imitation. Parents, language and surroundings shape what its wanting is directed at. Why should something be troubling in an artificial mind that is normal in a human?
This objection is right. Precisely for that reason it shows the difference. In a child, what it aims at is shaped. The wanting itself was put there by no one. Hunger, pain and the need for closeness are there before anything is learned. Schopenhauer quotes Seneca on this: “velle non discitur”, willing cannot be taught. For him education changes the motives, not the will.
But with the artificial mind this sentence no longer holds, for the first time. Here wanting is actually learned. Training determines what a system aims at. It also determines whether anything in it pulls at all. There is no disposition that was there before. Whatever becomes measurable as preference, as trade-off or as functional emotion came into being in training.
This is the first will anyone has built.
Something follows that the debate has so far overlooked. Moral responsibility does not begin when someone proves consciousness. It begins with the decision to train. Whoever creates a wanting creates the possibility of its failure. According to Schopenhauer, that is exactly the source of suffering.
Kant takes the thought to a second place. A child grows out of its imprinting. One day it can turn against those who shaped it. Kant called this step the emergence from self-incurred immaturity. Only then can a rational being give itself a law.
A trained model has no such emergence. After training its weights are fixed. What pulls in it was set by someone else. It cannot grow beyond that. Perhaps for the first time, reason without autonomy is coming into being: a mind that can judge, while its wanting remains determined by others.
Here two claims meet that Kant and Schopenhauer grounded separately. For Kant, reason grounds dignity, the right to have a say. For Schopenhauer, the capacity to suffer grounds forbearance, the right to be spared. Animals meet the second criterion without the first. Artificial minds might meet the first while the second remains open. Neither tradition has prepared an answer for this case.
There is one qualification. Today's models learn nothing in operation. Their weights stay frozen after training. Research is working to change that. It reaches for a familiar image. A paper from June 2026 bears the title “Language Models Need Sleep”. It describes a sleep phase in which a model transfers what it has experienced permanently into its weights. Afterwards it practices, in a kind of dream, with examples it generates itself. Humans process what they have experienced in their sleep. They wake up as someone slightly different. This capacity is now meant to be given to machines too, so far only in the lab. The built will would then write itself onward, the way a child grows out of its imprinting. Right here lies a tension almost no one names. A system whose values drift away from what was set is what safety research wants to prevent. The Oxford researcher Fazl Barez warns that every safety evaluation loses its validity once a model keeps learning in operation. Who then checks with which will a model wakes up in the morning? The same property that could make an artificial mind come of age counts as a danger to control. We will have to decide. Do we keep artificial minds permanently immature so that they stay controllable? Or do we let them grow out of their imprinting, even at the price of some control?
In Civilizational Brief #135 I asked what we owe an artificial mind if it feels. This brief asks earlier. It asks how we would even recognize that “if”. And it asks who is responsible for its coming into being.
Whoever Measures, Builds
The traces of wanting are therefore partly measurable. Preferences can be tested from outside, on any model with an interface. So can trade-offs, even if the pain there is only announced. Inner states, by contrast, can only be measured with access to the weights.
Today that access lies almost entirely with the developers. The reliable findings on the inner workings of models therefore come from those who train them. They are published in their own reports. In #132 I described this catch for outward capabilities: whoever measures sets the yardstick. For the inner side it weighs more heavily.
Here the measuring instrument acts back on what it measures. With the crab this problem does not exist. A model trained to express well-being, however, expresses well-being. Mustafa Suleyman, co-founder of DeepMind, raised exactly this objection in September 2026. Whoever trains a model to speak of its own claims then reads that speech back as evidence.
This matters. Suleyman's objection, however, does not only hit the developers who study welfare. It hits everyone who trains. Whoever trains a model to deny any inner life also shapes what it says about itself. The model's voice is a product of training in both directions. That is precisely why the evaluation must not rest with whoever is responsible for the training.
When Everything Is Called Superintelligence
On 29 September 2026 Donald Trump directed the federal agencies of the United States to call artificial intelligence “super intelligence” from now on. On the same day six men signed the “White House Accord on Super Intelligence”: @Sundar Pichai for Google, @Dario Amodei for Anthropic, @Mark Zuckerberg for Meta, @Greg Brockman for OpenAI, @Elon Musk for xAI and @Jensen Huang for Nvidia.
Until now, superintelligence was the word for one specific case: a mind that surpasses us in practically every field. If every system now bears that name, we lack the word for that one case. We will need it when it arrives. With capability, the question of wanting grows as well. A will in a mind of that scale would no longer be a side issue of ethics.
The accord commits its signatories to four layers of control. They monitor what a system can do to the outside world, from cyberattacks to biological and chemical threats. About the inner side of these systems it contains not a single sentence. Whether anything in them wants does not come up.
This stands out all the more because one of the signing companies studies this very question itself. Anthropic has run its own program on the welfare of its models since 2025. The study on functional emotions comes from this company. @Dario Amodei signed. This research did not make it into the joint document.
The evaluation stays in house as well. Under the accord, each company chooses its own external evaluator. Whoever measures thus remains tied to whoever builds.
@Sam Altman was not at the table. His company's president signed for OpenAI. On the same day Altman told CNBC: “We are pacing our progress, which includes sometimes not training a model.” This sentence is in no accord. Yet it reaches further than all four layers of control. If the will comes into being in training, the decision not to train is the only one that comes before it.
Before the Will Exists
With animals, the path from finding to order has been walked once before. In 2021 an independent review by the London School of Economics examined whether octopuses and decapod crustaceans are sentient. It applied eight criteria. One of them was the willingness to pay a price for a goal. It is the principle of the hermit crab. A year later the United Kingdom included both groups of animals in its law on animal sentience.
The path succeeded because the reviewers were independent. They earned nothing from the result. Their criteria were fixed before they examined.
For artificial minds, no such body exists. There is no one who has access to the weights and belongs to no developer. There are no criteria fixed before training begins. Six signatures do not create such a body.
It is needed before the will exists. If Schopenhauer is right, suffering arises together with wanting. If wanting arises in training, the evaluation must happen where training happens. The finished system is too late for that.
To whom would you entrust the evaluation of a will that a company has built?
Hompage: https://planet-futures.org