I just happened across an excellent illustration of why you need to go all the way through statistics to the hard numbers rather than stopping at initial impressions. This is in a presentation on newborn screening for a specific disorder. Check out these numbers:
(probability of correctly identified patient): 100.00%
(probability that healthy baby has normal result): 99.94%
False Positive Rate: 0.06%
Looks great, right? No false negatives, and only 0.06% false positives. So if you have a baby who tests positive, what are the odds that baby actually has the condition?
Here are some more numbers to help you, since the above information is not enough to tell you the answer:
Total screened: 78,017
Number of affected newborns: 41
Those numbers will lead you to these numbers:
False negatives: 0
False positives: 47
Positive predictive value
(proportion of babies with positive result who are truly affected):
So here’s a screening test where there are no false negatives and only 0.06% false positives, and if a baby tests positive there’s still a less than 50% chance the baby has the condition.
But which of the above numbers do you reckon would get play in a newspaper article? And if the last number wasn’t explicitly stated, how many people do you reckon would work it out?
The other thing that remains to be considered is, of course, what is done on the basis of a positive result, and what effect that would have on an unaffected infant with a false positive.