Another Dementia Test Oversold 24
An anonymous reader writes: Many prominent news organizations, including the BBC, are reporting on a study (PDF) that claims a new blood test is 87% accurate in predicting which patients will develop cognitive impairment. It's hailed as a major step forward in efforts to fight dementia and Alzheimer's disease. Unfortunately, reality isn't quite so impressive. An article at MedPage Today explains all of the statistical facts that the mainstream press glosses over: "Only about 10% of patients of patients with MCI convert to clinical dementia per year. With nearly 30% of positive results false (remember, the specificity was 71%) as well as 15% of negative results false, most of the positive results in such a group will be false. Yes, it's time once again for a tutorial in positive predictive values. If we have 100 MCI patients and a 10% conversion rate, then 10 of them will develop dementia. These are the true positives. There will be 90 true negatives — the ones who don't convert. But with a specificity of 71%, the test will falsely identify 29% of the 90 true negatives, or 26, as positive. Meanwhile, with a false negative rate of 15%, only nine (rounding up from 8.5) of the 10 true positives will be correctly identified. ... It's easy to get a high negative predictive value when the annual event rate is 10%. If I simply predict that no one will convert, I'll be right 90% of the time."
Re:Basic statistics (Score:5, Interesting)
Or that they have learned, passed the test, and forgotten. There was quite an interesting read on exactly this subject on the BBC website this week, Do doctors understand test results? [bbc.com]:
In one session, almost half the group of 160 gynaecologists responded that the woman's chance of having cancer was nine in 10. Only 21% said that the figure was one in 10 - which is the correct answer. That's a worse result than if the doctors had been answering at random.