The Undelegable

Sometime this month, most of us asked an AI to explain something. A concept we half-followed, a clause in a contract, a paper we had only skimmed. ChatGPT or Claude explained it well — clear, patient, better ordered than the source — and we read the reply and felt the small click of having understood. An hour later it was gone. The reply is still in the history; scroll up and there it sits. What left was the understanding. We held a perfect explanation the whole time and kept almost none of it, and for a moment we could not tell that anything had gone.

That is a new kind of memory, and the whole argument is folded inside it. So concede everything to the AI first. Ask it to summarise the report, cut a book to its load-bearing claims, draft the memo, and it returns something faster and cleaner than our own hands would manage. The meeting no longer needs a note-taker, because Granola or Copilot has caught every word and mailed the summary before anyone stands up. The harder work is going the same way: agents book the travel, reconcile the sheet, and answer the thread while we watch. Measured by the page produced, the contest is finished. The pen lost. If the point of writing something down is to end up with the thing written down, there is no reason left to do it by hand, and a person still filling a notebook invites one reasonable question about why they bother.

Let the sentimental defences go too. The tactile pleasure of paper, the smell of ink, the romance of the analogue — none of it survives contact with a tool that does the job better. If putting pen to paper is worth defending, it has to be defended on the AI’s own terms: output, speed, result. On those terms it is indefensible. The defence has to begin elsewhere, by asking whether the output was ever the thing.

Two things happen when we write, and we have spent a long time treating them as one. A record gets made: a sentence exists that did not a moment ago, holding a piece of information. And something happens in the writer: a loose, half-formed thought is forced, by the demand to become a sentence, into a shape it did not have. The first is a product. The second is a change of state in a mind. They arrive together so reliably that we mistake them for one event. They are two events, and the whole argument lives in the gap between them.

Information is a commodity, and its defining trait is that it moves without loss. A fact copied to a second page is the same fact; a number sent around the world arrives intact; what one mind holds, in this sense, another can be told. Understanding behaves differently. It cannot be copied out of one head into another. You can hand someone the conclusion and watch them possess it while grasping nothing. Gilbert Ryle marked the line in 1949: knowing that something is the case is a possession, knowing how is a capacity, and no stack of the first ever amounts to the second. Memorise every proposition about swimming and you will still go under. Michael Polanyi supplied the rest in 1966, in the sentence that should sit over this whole subject — we know more than we can tell. What a mind understands runs past what it can state, so it runs past what can be transmitted, so it runs past what can be automated, because automation works on exactly the part that can be stated.

That is the part an AI works on. A language model manipulates the tellable: tokens, weights, the statistical shape of everything already written. John Searle saw the consequence forty years before the models arrived. Put a man who reads no Chinese in a sealed room with a rulebook; Chinese characters come under the door, he matches them by the rules and passes new ones back, fluently enough that the people outside are certain they are corresponding with a native speaker. He understands none of it. He is moving shapes. Searle aimed this at the machine’s mind, and it cuts the other way just as cleanly: a system can produce every outward sign of understanding by operating on symbols, with the understanding wholly absent, and from the outside the seam does not show. What the model passes back under the door is the output of the room. It is information, always, even when the information describes understanding perfectly.

✦   ✦   ✦

Here a certain reader has stopped listening, because the argument is old and went badly last time. Around 370 BCE, in the Phaedrus, Plato has Socrates tell the story of writing’s invention. The god Theuth offers his new craft to King Thamus and calls it a pharmakon — the one word for a cure and a poison — that will make people wiser and sharpen their memory. Thamus declines the gift. Writing will do the reverse of the promise: people will stop remembering and start looking things up, leaning on external marks instead of the memory within. It gives, he says, “the appearance of wisdom, not its reality.” It is a recipe for reminding, and reminding is a poor cousin of remembering.

Now the reader has me, because everything I have said about AI, Thamus said about the alphabet. The notebook I am defending is the very device he refused. It puts memory outside the skull; it lets you stop holding a thing because the page will hold it; it swaps a mark for a mental act. On my own logic the pen is the first dose of the disease. And the trap shuts completely at the realisation that we know Socrates worried about this only because Plato wrote it down — the case against writing has survived twenty-four centuries by being written, and without writing would have died with him. The defence of the unautomated hand seems to fold into the oldest self-refutation in philosophy.

It folds only if writing and AI automate the same thing, and they do not. Thamus was right about what he saw, and what he saw was the export of storage. Writing moves the record out of the head and lets memory rest on a shelf, and his complaint holds: we do remember less, because the page remembers for us. But look at what writing left in place. To get the thought onto the page, you still had to have the thought. The sentence never wrote itself; the synthesis — the forcing of the loose thing into a shape — happened in the writer every time, as the toll for putting anything down at all. Writing exported the product of thinking and left the producing where it stood. AI exports the other half: it performs the synthesis and returns the result, and the shape that once had to be paid for in effort now arrives finished, from outside. That reverses the direction writing ran in. The pen and the model are early and late points on no single line; on the axis that matters they face opposite ways, one compelling the synthesis, the other supplying it. Thamus’s objection, granted in full, turns and works for us.

✦   ✦   ✦

Push it further, because storage and synthesis are not the top of the stair. Above them stands the thing we understand least and lean on most: origination. It is the arrival of a thought the inputs did not contain — something past retrieval, past the reshuffling of old parts. The confident view says no such thing exists, that creativity is recombination, every new idea a fresh arrangement of existing pieces, and that a system able to hold and permute more pieces than any human is therefore our better at it. The view has the models on its side, spinning out more novel combinations in an afternoon than a person manages in a decade.

Its rebuttal is older than it looks. In 1843 Ada Lovelace, writing the first serious account of a general-purpose computer, fixed its ceiling: the engine “has no pretensions to originate anything. It can do whatever we know how to order it to perform.” A hundred years on, Alan Turing named this “Lady Lovelace’s Objection” and answered it the way it is still answered today — machines take us by surprise constantly, so plainly they do more than we order. The answer sounds final and is not. Surprise is a report on the observer. It measures how far the combinatorics have outrun your ability to follow them, and says nothing about origination inside the system. A tumbling die will surprise you. The die is doing arithmetic you were too slow to do, and no quantity of it becomes creation.

The harder claim is the one recombination cannot digest. Recombination runs on information; origination runs on understanding. To make a new arrangement of parts you need only the parts and a rule for arranging them, and a model has both to spare. To make the one arrangement that matters — to pick, out of an astronomical field of possible combinations, the single one worth having — takes something else. It takes judgement, and judgement is understanding in use: the felt sense that this combination is true or right or beautiful and those ten million are noise. That sense is not stored in the parts. It exists only in a mind that built its understanding the slow way. Recombination runs on information; origination runs on understanding. A model can recombine forever and originate nothing, because the origination was always in the choosing, and the choosing is the part that will not transfer. Nor can it lend the judgement out. It can hand you a combination it has no way to weigh, and you can weigh it only as far as you built the understanding the weighing needs — the very understanding the model offered to spare you.

This is where the danger leaves the seminar room. The danger is a specific kind of success. AI produces a flawless counterfeit of what understanding produces, and produces it with nothing behind it. The summary is accurate, the analysis clean, the argument well-built; every outward mark of a mind that has seized the material is present, the seizing is not, and the marks are the only thing anyone can see. Searle’s room has been mass-produced and handed out, and it turns out the room fools its own occupant. You take delivery of a comprehension you never performed and land in a state without precedent: informed and uncomprehending at once, holding the appearance of wisdom in the exact socket where its reality belongs, unable from the inside to feel that the socket is empty.

The first measurements of this are arriving, and they point where you would guess. In 2025 a group at MIT’s Media Lab led by Nataliya Kosmyna wired up fifty-four people writing essays — some with a language model, some with a search engine, some with nothing but their heads. The model group showed the weakest, sparsest brain activity, and asked minutes later, many could not quote a line of the essay that carried their name. The authors called what builds up “cognitive debt”: a productivity gain now, an understanding you never acquired, the bill deferred. The study is small and already contested, and it hardly settles the matter, because Thamus called the result in advance. He feared one man who would seem wise without being wise. What we have built is the industrial edition — the appearance of wisdom, produced faster and cheaper and glossier than the reality, so the counterfeit beats the real thing on every visible measure at the moment of choosing. And because understanding is the sense that would notice understanding missing, its absence is the one loss it cannot report. A mind can be emptied of the capacity and feel only the relief of time saved. A culture can fill to the brim with the outputs of thought while the thinking drains out of it, and every surface — every deck, every summary, every fluent reply — will read as proof that all is well.

So the claim can be set down exactly, and it is narrower and harder than the mood so far suggests. Use the tools. Most of what we write is storage, and storage should go to whatever keeps it best; there is no merit in a person doing by hand what a model does without error, and a note held by software is as good as a note held by you, because a record’s worth sits entirely in the record. The claim fastens on one act with one strange property. The building of a mind’s own understanding is the single act whose whole value is destroyed by succeeding at handing it off. Delegate the storage and the thing stored stays yours. Delegate the synthesis and there is nothing to collect, because an understanding assembled somewhere else was never going to be yours. The trade that works for every commodity breaks here, and breaks by definition, because the good was never the output. The good was the building, and building is not a thing another party can do in your place.

Call the act undelegable, if the word earns its keep — delegating it breaks no rule, it cancels the thing. The distinction is the one between lifting and strength, and the analogy is exact. You can pay someone to lift for you: the box goes upstairs, the value was the box upstairs, and the delegation succeeds completely. You cannot pay someone to be strong for you. Strength is what settles into the body that does the lifting, and a proxy who lifts in your place leaves you precisely as weak, and a little weaker, for the session you skipped. Understanding is the strength; the finished page is only the box. AI is the most capable proxy ever built, it will carry any load you name, and every load it carries for you is one your mind does not. The stronger the proxy, the weaker the principal — and the more spectacular the proxy, the harder that law is to credit, because the boxes keep arriving upstairs and the results have never looked better.

✦   ✦   ✦

Which brings us back to the word we started with: obsolete. Obsolescence is a relation between a means and an end. A thing goes obsolete when a better means reaches the same end, and the end sits outside the thing — the candle goes obsolete because the bulb makes the light, and light was the whole point of the candle. The building of understanding has no end outside itself. Its output was never the point. The building was the point, and its only beneficiary was the one doing the building, which leaves nothing for a superior means to be superior at, because here the means and the end are one event looked at twice. An act shaped like that cannot be rendered obsolete. It can only be dropped.

That is the exact service the technology has done us, and it is worth saying without theatre. It has changed a single word. The cost of thinking used to force the act on us, and the hand was necessary; the cost is gone now, and the hand is optional, an act left standing on its own to be chosen or not. Everything rests on a decision that never had to be taken before, because there was never a way to avoid taking it. The offer, at last and in earnest, is to do our thinking for us. The offer is real. The one thing it cannot tell us, since the answer is the very faculty the offer proposes to retire, is who we will be once we have accepted.