Insensitivity to sample size
Insensitivity to sample size is judging a result without considering how many cases it is based on. Small groups swing to extremes far more often than large ones, yet we treat their results as equally reliable.
In everyday life
People are equally impressed when a tiny school and a huge school both post unusually high test scores, though the tiny school's result is far more likely to be a fluke.
You pick a restaurant because it has a perfect five-star average, without noticing it has only three reviews, while a nearby place rated 4.6 has two thousand.
Why your mind does this
Our ancestors survived by spotting patterns fast: rustling grass might mean a predator, and dark clouds meant rain. Filling gaps with a plausible story lets us act on thin information instead of freezing. A brain that sometimes sees a pattern that isn't there loses less than one that misses a real threat.
Its family, in 30 seconds: Stories & patterns
How to spot it
Ask how many cases a result is based on before deciding how impressed to be.
How to counter it
Look for the count behind every average or percentage, and give results from small counts much less weight.