The base-rate fallacy, and how a rare case gets lost

An everyday example

A workplace screening tool flags an employee as a fraud risk. The tool is described as 90% accurate, and that number alone feels damning. What is missing is how rare fraud actually is among employees to begin with: if only one in a thousand employees is actually committing fraud, a 90% accurate tool run across the whole workforce will still flag many more honest employees by mistake than it catches actual cases, simply because there are so many more honest employees for it to make mistakes on.

The accuracy figure describes the tool. It does not, by itself, answer the question anyone actually wants answered: given a flag, how likely is this particular person to really be a fraud risk. That answer needs the base rate too.

The classic experiment

Kahneman and Tversky's 1973 paper gave participants short personality sketches, such as one describing a man as enjoying maths puzzles, having little interest in people, and being described by neighbours as somewhat withdrawn. Participants were told the sketch was drawn at random from a pool of descriptions, and, critically, were told what the pool was made of: in one condition, 70 engineers and 30 lawyers; in another, 30 engineers and 70 lawyers. Asked to judge the probability that the described person was an engineer, participants gave very similar answers in both conditions, judging almost entirely by how much the sketch matched their idea of a typical engineer and barely adjusting for whether engineers made up 30% or 70% of the pool they were drawn from. In a further condition, participants were given no personality sketch at all, just told the pool's composition, and asked to guess the probability a randomly drawn person was an engineer; here, participants did use the base rate correctly, since it was the only information available. The base rate was not being ignored because people could not use it. It was being crowded out by specific, if largely uninformative, detail the moment any was on offer.

When is a base rate actually used?

Maya Bar-Hillel's 1980 paper looked across this kind of study and argued the neglect is not blanket. Base rates, she found, tend to be used more when they feel causally or directly relevant to the specific case at hand, and tend to be ignored when they feel like a separate, merely statistical fact sitting alongside the case rather than bearing on it. A base rate framed as a cause, such as a known local risk factor that would plausibly apply to this specific person, gets weighed differently from a base rate framed as an abstract proportion of an unrelated pool, even when the two carry the same statistical information. The lawyer-engineer sketches sit closer to the second kind: the stated proportions read as a fact about the pool, not as a fact that obviously bears on this one described individual, which is part of why they were so easy to set aside.

Does it replicate?

Replication grade: Mixed: the core finding replicates; how general and how normative the effect is remains debated

The basic pattern, personality or case detail dominating a stated base rate, has been reproduced many times since 1973, across professions, students and, in some studies, working doctors and statisticians reasoning about diagnostic tests. Jonathan Koehler's 1996 target article, published with a long set of expert commentaries attached, pushed back on how the finding is usually summarised. Koehler argued that base rates are used correctly far more often than a blanket "people ignore base rates" claim suggests, and that many classic demonstrations, including versions of the lawyer-engineer task, use base rates that are ambiguous, not clearly sampled at random, or not obviously relevant to the individual case, exactly the conditions Bar-Hillel had already flagged as ones where neglect is more likely. Koehler also questioned how often the textbook normative answer, weight the base rate this heavily, is really the objectively correct one for real-world versions of these problems, where the individuating evidence and the base rate are not always as cleanly independent as a classroom vignette makes them look.

The honest summary sits between the two positions: the core finding, that specific case detail can crowd out a relevant base rate, replicates reliably under the conditions Kahneman and Tversky used. Koehler's reconsideration does not erase that; it narrows the claim, showing base-rate neglect is not a fixed, universal blind spot but something that depends heavily on how the base rate is presented and how relevant it is made to feel.

Why the rare group still gets swamped

The diagram on this page works out a worked, invented example as an icon array of 100 cases, so the base rate's effect on the final answer can be seen directly rather than taken on faith.

Try it: the Bayesian intuition test

Six base-rate problems about a check that sometimes raises false alarms, half given as counts and half as percentages, each followed by an icon array of the same numbers as cases.

How to catch it

Base rates are easiest to lose exactly when a case comes with a vivid story attached, which is most of the time.

• Before weighing any specific detail, ask: out of everyone this description could apply to, how common is the thing I am judging the probability of.
• Convert percentages into a natural count out of a stated population; it is far easier to see a rare group get swamped by false alarms in a count than in a percentage.
• Ask whether the base rate feels like a fact about this specific case or a fact about an unrelated pool; if it feels unrelated, that is exactly when it is most likely to be set aside, correctly or not.
• When a screening or diagnostic result is described only by its accuracy, ask for the rate of the thing being screened for in the relevant population before drawing a conclusion.
• Remember that specific, vivid detail is not the same as strong evidence; a detail can match a stereotype well and still be uninformative about the true odds.