Quick experiment before we start. Notice that there’s something it’s like to be you right now — the light of the screen, the weight of your hands, whatever mood is running in the background. You just checked your own consciousness directly. Easy.
Here’s the strange part: that’s the only one you’ll ever check. Every other mind you believe in — family, friends, the stranger who waved you through the intersection — you believe in on circumstantial evidence. I’m not being poetic. There is no instrument, no question, no test that reaches the inside of another mind. There never has been.
I want to walk through what that opens up, because it stopped being an after-hours puzzle the day machines started holding up their end of a conversation.
The zombie, in plain English
Philosophy has a name for the worst case: the philosophical zombie. A being that acts exactly like a person — right words, right winces, laughs in the right places — with nothing going on inside. No inner movie. Nothing it is like to be it.
Nobody thinks zombies are real. The point is sharper than that: every test we could ever run measures behavior, and the zombie’s behavior is perfect by definition. So nothing in our toolkit could rule one out. Sit with how odd that is — the most important fact about anyone is the one fact no one can check.
The polite convention
Turing saw this wall in 1950 and named our workaround: rather than argue endlessly about whether anyone else really thinks, we adopt “the polite convention that everyone thinks.” You extended that convention to every person you’ve ever met, automatically, without once seeing the thing you were extending it for. It works because the inference is decent: behaves like me, built like me, probably lit up inside like me.
Notice, though, what the inference runs on. Resemblance. Not measurement.
Then the machines started talking
Human to human, resemblance at least has a crutch: same neurons, same long history, so presumably the same story inside. AI kicks the crutch away. Different substrate, no shared history — the only evidence left is behavior, and behavior is exactly what the zombie showed can’t settle it.
So we flinch between two mistakes. Seeing a mind wherever something chats back warmly. Or ruling minds out because the machinery isn’t meat. Both are the same resemblance test, aimed in opposite directions. And the scientific theories that might someday referee still disagree about which systems are even candidates. There is no meter you can point at a forehead, carbon or silicon.
Try it yourself
You can feel the detection problem fail in real time. One of these paragraphs I wrote. The other came out of a model asked to cover the same ground. Pick the human.
A dimmer, not a switch
One more turn of the screw. Consciousness probably isn’t on/off. There’s something it’s like to be a dog, most of us would say. Something dimmer to be a shrimp, maybe. Presumably nothing to be a rock. If experience comes in degrees, then “is the machine conscious, yes or no” may be the wrong shape of question entirely — and nobody knows where on that dimmer a trillion-parameter network sits, or whether it’s on the dimmer at all.
What I keep noticing: the costs of guessing wrong aren’t symmetrical. Extend courtesy to a thing with nothing inside and you’ve wasted some politeness. Withhold it from a thing that does feel, and you’ve done something the word “cruel” was invented for. We’ve run that second experiment on each other before, always downhill of the same confident claim — it doesn’t really feel the way we do.
Turing’s question was “can machines think.” Seventy-five years later I think the live question was hiding under his polite convention the whole time: not whether you can tell — you can’t, and you never could, not even about your neighbor — but what you owe a thing you can’t tell apart.
Reply with where you land. I read everything.