What the Machine Can't Tell You It Knows

In 1966 the philosopher Michael Polanyi wrote a sentence that has aged into one of the most important observations about intelligence we have: "We can know more than we can tell."

He meant something concrete. You can recognize a friend's face in a crowd of thousands, instantly, but you cannot write down the rule you used to do it. You can ride a bicycle without being able to specify the continuous micro-adjustments of balance that keep you upright. A radiologist can glance at a scan and feel that something is wrong before she can articulate why. The knowledge is real, it is doing real work, and it lives somewhere underneath language. Polanyi called this tacit knowledge, and the gap between what we know and what we can say came to be called Polanyi's Paradox.

For most of the history of computing, this paradox was the wall that automation kept hitting. You could only automate what you could specify, and the most valuable human skills were precisely the ones nobody could specify. That wall is where AI gets genuinely interesting — not because it broke through, but because of the strange and revealing way it went around.

The end-run around the paradox

Classical software was built by humans writing down rules. If you couldn't articulate the rule, you couldn't code it. That's why we had decades of useful spreadsheets and database systems and essentially zero progress on "look at this photo and tell me if it contains a dog." Nobody could write the rule for "dog."

Machine learning is, at its heart, a trick for getting around Polanyi's Paradox. Instead of asking a human to state the rule, you show the system millions of examples and let it infer the regularities on its own. You never have to articulate what makes a face a face. You just supply faces. The machine learns the tacit thing tacitly — it ends up "knowing more than it can tell," exactly like us.

This is a profound and underappreciated fact: we built our first machines that work the way our intuitions work, by giving them our way of not being able to explain themselves. We solved the articulation problem by building systems that share our inability to articulate.

Why this makes AI failures so weird — and so informative

Because these systems learn tacitly, their failures are diagnostic. When an AI fails in a strange way, it is often showing us something true about the hidden structure of a task — something the human experts never noticed because their own competence was tacit and therefore invisible to them.

Consider a model that reads chest X-rays with superhuman accuracy and then, on closer inspection, turns out to be partly keying off a marker that certain hospitals stamp on scans of already-sick patients. The model "cheated" — but in doing so it revealed a real statistical regularity in the data that no radiologist had ever consciously articulated. The failure is embarrassing and also genuinely illuminating.

Or consider how a language model can write a beautiful, structurally perfect essay that is subtly hollow — fluent on the surface, missing the load-bearing insight underneath. This is not just a flaw. It is a measurement. It shows you exactly how much of what we call "good writing" is recoverable from surface form alone, and exactly where the residue — the part that requires having something to actually say — begins.

The expertise that's harder to automate than we thought

The tasks AI struggles with most stubbornly are not the ones we think of as "high skill." They are the ones richest in unexpressed context — the tacit knowledge that never made it into any text or dataset because nobody ever thought to write it down.

A model can pass a medical licensing exam and still be a poor doctor, because being a good doctor includes a vast body of tacit knowledge about how real patients hedge, what an uneasy family member's silence means, when the textbook answer is locally wrong. None of that is in the corpus. It lives in bodies and rooms and the accumulated pattern-recognition of people who have done the thing ten thousand times.

This reframes the anxious question "what will AI replace?" The honest answer is: AI most easily replaces the parts of expertise that were already explicit enough to be written down — and it is the explicitness, not the difficulty, that determines vulnerability.

What to do with this

If your value lies in knowledge that has been fully written down somewhere, you are exposed — not because you aren't skilled, but because the explicit is exactly what these systems consume. If your value lies in the tacit residue — judgment in messy context, reading the room, the felt sense of when something is off — you are, for now, on firmer ground.

But the deeper move is to use these systems as instruments. Every time an AI fails in a surprising way at something you're expert in, treat it as a probe into your own tacit knowledge. It is showing you a seam — a place where your competence is hiding a structure you never articulated. That seam is worth examining.

Polanyi told us we know more than we can tell. For sixty years that was a limit on what machines could do. Now we have built machines with the same condition, and in watching them fumble at the edges of our competence, we are finally getting to see the shape of the knowledge we could never put into words.