A robot I watched recently lifted a cutlery basket out of a dishwasher on a single finger and carried it toward the tabletop. Halfway across, the basket lost balance and the forks slid out and clattered onto the floor.
The robot finished the motion exactly as planned — set the lighter basket down, opened the finger, retracted. No pause, no re-grasp, because nothing inside it had registered that anything happened. The failure isn’t that the policy was wrong. It’s that it was wrong and had no idea.
Here is the part that nags. The success rate on this task is high — high enough to call the skill solved and put it in a demo reel. Then you watch one rollout like this and the number stops meaning what you thought. Nothing in that sequence looked like a policy that knows how to carry a basket. It looked like a policy with a motion that usually works, and usually is doing a lot of quiet work inside an average over rollouts where nothing shifted.
So, the question this whole piece is about: are we measuring the wrong thing?
I have wanted to build curiosity into a robot for as long as I have been building robots, and that rollout is why. Not because the basket tipped — baskets tip. Because nowhere in the sequence was there a moment where the robot wondered. By curiosity I don’t mean exploration noise; a random-number generator perturbs actions perfectly and has never wondered about anything. The first real sign of curiosity is that a thing asks a question — and to ask one you must first have located something you don’t know.
For robots this matters more than it sounds, because our data problem is the inverse of everyone else’s — a small circle of data you have to grow, not a large one you filter down — and curiosity is what chooses where to grow it next. Part two is about that.
So the model I started sketching after that rollout was the smallest honest version of that curious robot: not curiosity itself, just a policy that could ask instead of carrying serenely on while the forks hit the floor. Working through it with Claude, it said: by the way, have you heard of Meno’s paradox? What you’re describing is close to the problem Plato poses there.
I had not — and what followed changed how I was thinking about the problem.
The other paradox
Every roboticist knows Moravec’s paradox: the difficulty ranking for machines is close to the inverse of the ranking for humans. Symbolic reasoning took little to automate; sensorimotor competence — gravel, a wet sock, a door handle you’ve never touched — consumed decades, because perception and movement have had hundreds of millions of years of optimization and abstract reasoning is a thin recent veneer.
Moravec’s paradox ranks tasks by difficulty. Meno’s asks something else entirely — whether inquiry is possible at all. It is 2,300 years older, and I had never heard of it.
Meno’s paradox
The Meno opens with Meno asking Socrates whether virtue can be taught. They fail to define virtue, and Meno — irritated — throws a genuinely nasty objection at him:
And how will you inquire into a thing, Socrates, when you are wholly ignorant of what it is? Which of the things you don’t know will you set up as the target of your search? And even if you should stumble right onto it, how will you know that this is the thing you did not know?
— Meno, 80d
It looks like sophistry. Sharpened into two horns it becomes genuinely hard:
- If you already know what you’re looking for, there is nothing to look for. Inquiry is unnecessary.
- If you don’t know what you’re looking for, you have no criterion for recognizing it. You could trip over the answer and walk on. Inquiry is impossible.
That right-hand horn is the exploration problem, stated exactly. A policy in a state it has never seen cannot know the state is one it has never seen; it has no representation of unfamiliar. It cannot search for the demonstration it is missing, because “the demonstration I am missing” is not a thing it can point at. It cannot ask for help, because asking requires knowing you need it. That — not compute, not data, not architecture — is why the design was hard.
Socrates, a boy, and a square in the dirt
Socrates’ answer to the paradox is not an argument. It’s a demonstration, and it’s the best part of the dialogue.
He calls over one of Meno’s household slaves — a boy who has had no mathematical education whatsoever — draws a square in the dirt, and asks him to double it. Not to double the side: to produce a square with twice the area.
The boy answers instantly and wrongly — double the side — and is shown that this gives four times the area, not twice. He tries three: nine, still wrong. And now the moment Socrates flags as the turning point (84a–c): the boy stops guessing and says he does not know.
Socrates makes Meno admit that the boy is better off in this state than he was five minutes earlier:
He did not know before… but he thought he knew, and answered confidently as if he knew, and did not think himself at a loss. Now he does think himself at a loss, and as he does not know, neither does he think he knows… We have made him ready to find out.
— Meno, 84a–c
Read that again as an engineer. Nothing was added to the boy — no new fact, no demonstration, no gradient step. His confidence came into agreement with his competence, and Socrates’ claim is that this alone made him teachable. (The image is Meno’s: at 80a he accuses Socrates of being a torpedo fish that numbs whatever it touches, and Socrates keeps the name.) The numbing isn’t a step on the way to learning; it is the precondition for it — and that is the design document I had been fumbling toward. “Knows it does not know” and “therefore asks” are not two features. They are one.
Only after the boy is genuinely stuck does Socrates draw the diagonal — and the boy sees it himself. Socrates’ own explanation is metaphysical: the boy learned nothing, he recollected something his soul already knew. You don’t have to buy the metaphysics to keep the mechanism.
True opinion, and the statues that run away
The dialogue’s other gift, at 97a–98a, is the most useful thing I have read about model evaluation.
Socrates asks what makes knowledge better than merely-correct belief. A man who knows the road to Larissa will guide you there; a man who has never been but happens to hold a true opinion about the road will guide you there just as well. For the trip, they are indistinguishable. A true opinion — orthē doxa — is as good a guide as knowledge.
Right up until it isn’t. Because true opinions, Socrates says, will not stay:
True opinions are a fine thing and do all sorts of good so long as they stay in their place, but they will not stay long. They run away from a man’s mind; so they are not worth much until you tether them by an account of the reason why.
— Meno, 97d–98a
The image is the statues of Daedalus, which legend held were carved so lifelike they would walk off if you didn’t tie them down. An untethered true opinion is a statue on a plinth: perfectly correct, perfectly convincing, gone tomorrow. What converts it into knowledge — epistēmē — is aitias logismos, an account of the reason why. You know why it is true, so it stays put when circumstances move.
Five tethers
Socrates’ remedy is to tether the opinion. A policy cannot recite an account, so its version has to be operational: a list of things you change that should not matter. A policy holding real knowledge is indifferent to all of them. Each one you leave unmeasured is a rope you never tied.
I’d propose five, and they are cheap to state.
- Stability — change what should not matter perceptually: the lighting, the distractors, the camera pose, this mug for that mug. Does the policy even notice?
- Transfer — change the instance but not the type: a drawer it has never opened. Did it solve the problem, or memorize the episode?
- Recovery — disturb it mid-episode: slip the grasp, shove the target, move the goal. Does it re-plan, or carry on regardless?
- Foresight — make it say what its next action will do before it does it, and score the prediction rather than the outcome.
- Humility — score whether its confidence tracks its competence. The only one that asks the policy about itself rather than about the world.
Fig 6 names the failure each untied rope leaves behind: the Parrot, dazzling until someone moves a lamp; the One-Trick Pony, which solved the episode and not the type; the Glass Dancer, flawless until first contact with disorder; the Sleepwalker, acting without imagining consequences; and, when the last rope goes, the Confident Fool.
Four of these are about robustness. The fifth is different in kind, and it is the boy at 84a: Humility is the tether that makes the other four improvable.
A success rate is a count of true opinions
Now hold the number we actually ship against those five ropes, and count how many it ties. None.
A success rate is a count of true opinions. Run N episodes, count the wins, report a percentage. That number is the road to Larissa: it tells you the policy got there this time. It cannot distinguish a policy that has learned to solve a type of problem from one holding a lucky, memorized solution to these particular episodes — both produce identical numbers. One bit per episode, sampled suspiciously close to the training data, and quoted to two decimal places.
Every robot in every demo video is a statue on a plinth: beautifully lifelike, genuinely correct, and — the moment someone moves a lamp or starts the episode eight centimetres to the left — walked off in the night.
Policy B has a name: it is the Confident Fool, and it is the dishwasher failure from the opening, caught in the act. High success rate on its task, steady on the scenes it was scored on.
We scored that episode a failure — someone was watching, and forks on the floor are hard to miss. But notice how little comfort that gives. We caught it by luck. No initial condition reproduces it, no seed replays it, no flag in the harness says the load shifted mid-trajectory. A failure you cannot reproduce cannot be measured — no rate, no A/B, nothing to certify against. The aggregate here isn’t a weak instrument. It is blind, and stays blind however many episodes you run, because nothing in the protocol ever asks for the disturbance.
Now put it next to the slave boy. He also produced a confident wrong answer — and at the moment it failed, something in him changed state. He stopped. He said I do not know. The robot’s answer failed in precisely the same way and nothing in it changed state at all. It could not stop, because stopping requires noticing, and noticing was not among the things we trained.
One rollout, three missing tethers: no Foresight to predict that an off-centre load on one finger would swing, no Recovery once it did, and no Humility — the failure moved its confidence by not one hair. Missing in the most expensive way available: silently, where no aggregate over clean rollouts can see.
Meno’s paradox, answered by a metric
That last rope — Humility, the one the basket rollout never moved — is also the one that unties the paradox. Go back to Fig 3, the fork with no third door.
Socrates escaped it by insisting the answer was already in the boy. I’d escape it with calibration.
The right-hand horn says you cannot search for what you do not know, because you cannot recognize it. True — but you do not have to recognize the answer to recognize the question. A policy cannot know the correct action in a novel state. What a calibrated policy can know is that this is a state where its own confidence is low. Not the answer. The boundary.
And a boundary is enough, because it converts an impossible search into a request:
The robot that knows what it doesn’t know can ask for precisely the experience that converts its next true opinion into knowledge. Inquiry bootstraps.
And that is the curious robot I wanted at the top, arrived at from an unexpected direction. Curiosity is not a drive you add to a policy; it is what a calibrated policy does once it can feel its own edge. All of it is downstream of that one measurement.
A century before the Meno, and five thousand miles east of it, Confucius had already put the same thing as a definition rather than a virtue:
That is a definition, not a proverb about modesty — the Humility tether stated 2,500 years early. Two traditions, a century and five thousand miles apart, converging on the same test: a correct answer is not evidence of knowledge. Surviving a changed world is.
What to do on Monday
Three moves follow from all this, and none of them needs a new model.
1. Stop shipping a success rate without its context. Report the vector. A model card that says 85% without saying held fixed: lighting, object set, initial pose, and the twelve scenes we also trained on is a press release, not a result.
2. Make confidence a first-class output. Not a threshold bolted on at deployment, and not whatever signal happens to fall out of the logits — an output the model emits on every step, next to the action, trained and benchmarked as seriously as the action is. Score its calibration alongside its success rate, and make confidently wrong cost far more than uncertain and right. This is the same problem large language models have with hallucination, and it has the same cause: a model that has never been penalized for confident nonsense will produce confident nonsense, because nothing in the objective ever made its confidence mean anything.
3. Certify on knowing when to stop, not on the average. Don’t gate deployment on a success rate. Gate it on whether the robot recognizes the states it cannot handle and says so. We don’t license pilots on their pass rate over rehearsed routes; we license them on crosswinds, engine-out, and the judgment to declare a missed approach. Knowing when to go around is the licence.
The obvious objection: nobody wants a robot that stops every five minutes to ask for help. Quite right — which is why the thing to certify is calibration, not caution. A robot that asks constantly is as badly calibrated as one that never asks; in both cases its confidence tracks nothing. What belongs on the certificate is the trade-off: how much of the job it does unaided, and what fraction of its failures it flags before they happen. A robot that does 95% alone and raises a hand on the 5% that would have broken something is worth more than one that does 99% and buries the rest in an average.
I went looking for a way to make a robot ask for help and found the problem named, diagnosed and half-solved twenty-four centuries before anyone had a robot to ask it about. The paradox is real: you cannot search for what you do not know. The escape is smaller than it sounds. You never needed the answer — only to know, in calibrated units, where your knowing stops.
Everything else is a statue.
Part two, Three sources of knowledge — and the two axes we aren’t scaling, takes up the training half: where a policy’s content actually comes from, why 说 (thinking) and 亲 (experience) are barely scaled at all, and two further moves, on the training side rather than on measurement.
Written by Jinyu Xie and Claude. The Meno came up in one of our design conversations; the argument was worked out across a good many more. Passages from the Meno are Plato, trans. W.K.C. Guthrie and G.M.A. Grube, lightly adapted.