The Dunning–Kruger effect, and what the data can and cannot show

An everyday example

Someone who has baked bread three times cuts into a loaf and is pleased: it rose, it browned, it tastes like bread. A baker looks at the same slice and sees a tight, uneven crumb and a crust that set too early. The novice is not being vain. The knowledge that would let them see the faults is the same knowledge they would need to avoid them, and they do not have it yet.

That is the idea Justin Kruger and David Dunning put forward: in many skills, the ability to do the task and the ability to judge how well it was done are one and the same, so the people with the least of the skill are also the least equipped to notice what they are getting wrong. They called it a dual burden.

The classic experiment

Kruger and Dunning's paper, published in 1999, reported four studies with Cornell University undergraduates. Participants took a test in one of three areas: judging which jokes are funny, logical reasoning, and English grammar. Afterwards they estimated how their ability and their test score compared with those of their fellow students, as a percentile, and how many questions they thought they had answered correctly.

In every area, the students who scored in the bottom quarter overestimated themselves the most. According to the paper's abstract, their scores put them around the 12th percentile on average, while they estimated themselves to be around the 62nd. The top quarter showed the reverse, smaller error: they tended to underestimate where they stood relative to others.

Two further results carry the argument. In the grammar study, participants were later shown other students' answers to grade. The top scorers then raised their estimates of their own standing, having seen how others did; the bottom scorers did not. In the fourth study, some participants received a short training session on the logic problems. After the training, the low scorers became better at recognising which of their own answers were wrong and lowered their estimates. Kruger and Dunning read this as evidence that skill and insight into skill go together.

The curve that is not in the paper

The picture most people associate with the effect is a curve of confidence against experience: a sharp early peak of confidence, a fall into a valley of despair, and a slow climb. That curve does not appear in the 1999 paper. The paper's figures plot, for each quarter of test scores, the average actual percentile and the average estimated percentile. It measured people once, not over time, and it did not measure experience at all.

The distinction matters because the popular curve claims something the studies never tested: that beginners are more confident than experts. In the 1999 data the lowest scorers were not more confident than the highest scorers. Their self-estimates were somewhat lower. They were simply much further from the truth.

Does it replicate?

Replication grade: Mixed: the pattern is reliable; its explanation is disputed

The basic pattern, with low scorers overestimating their percentile standing far more than high scorers, has been reproduced many times, in classrooms and outside the laboratory. Ehrlinger, Johnson, Banner, Dunning and Kruger reported it in real exam settings and when participants were paid for accurate estimates, which argues against the idea that low scorers were simply not trying.

What is disputed is why it appears. Krueger and Mueller argued in 2002 that two ordinary ingredients produce the same picture: almost everyone rates themselves above average, and any measurement with noise in it makes extreme groups look more extreme than they are, so estimates drift back toward the middle. Burson, Larrick and Klayman showed that the pattern depends on task difficulty: on hard tasks, people at every skill level underestimate their standing, and the best performers can be the least accurate. Nuhfer and colleagues showed that random numbers, arranged the way the original figures were, produce a look very similar to the famous chart.

Gignac and Zajenkowski tested the claim with methods suited to data from individuals and concluded, in the words of their title, that the effect is mostly a statistical artefact. Jansen, Rafferty and Griffiths fitted a model of how people update their self-assessments and found that low performers were less sensitive to evidence about their own performance, which supports a real, if smaller, effect. Dunning has argued that the effect survives these corrections. A fair summary: people who do badly on a test usually do not realise how badly, but the size of the gap attributable to a special lack of insight, rather than to noise and the general habit of rating oneself above average, remains an open question.

How noise alone can draw the famous chart

The diagram on this page is a simulation, not real data. It invents 400 people whose self-estimates contain only two ingredients: a small amount of genuine self-knowledge and a general tendency to rate oneself above the middle. There is no extra blindness at the bottom. Grouped into quarters by score and plotted the way the 1999 figures were, the simulated people reproduce the familiar shape: the bottom quarter overestimates by a wide margin and the top quarter slightly underestimates. That is why the shape of the chart alone cannot settle the argument.

Try it: predict, then check

Say how many of five short puzzles you expect to get right, solve them, then say how many you think you got. The page puts the three numbers side by side.

How to catch it

The effect is about judging your own work, so the fixes are about getting an outside standard to judge it against.

• Before you rate yourself, write down what a good result would look like. If you cannot describe it, your rating has nothing to be measured against.
• Predict a number, such as a score, a time or a word count, before you find out the real one, and keep the pairs. Your record is a better guide than your feeling.
• Compare your work with an answer key, a worked example or someone more skilled, and look for differences rather than for agreement.
• Treat the thought 'this is easy' as a hypothesis when you are new to something, and test it on a hard case.
• Notice when doing a task and judging it rely on the same skill; that is where self-assessment is weakest.
• If you are experienced, remember the reverse error: you may be underestimating how unusual your skill is.