Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
Medicine AI Stats

88% Of Medical 'Second Opinions' Give A Different Diagnosis - And So Do Some AI (mayoclinic.org) 74

First, "A new study finds that nearly 9 in 10 people who go for a second opinion after seeing a doctor are likely to leave with a refined or new diagnosis from what they were first told," according to an article shared by Slashdot reader schwit1: Researchers at the Mayo Clinic examined 286 patient records of individuals who had decided to consult a second opinion, hoping to determine whether being referred to a second specialist impacted one's likelihood of receiving an accurate diagnosis. The study, conducted using records of patients referred to the Mayo Clinic's General Internal Medicine Division over a two-year period, ultimately found that when consulting a second opinion, the physician only confirmed the original diagnosis 12 percent of the time. Among those with updated diagnoses, 66% received a refined or redefined diagnosis, while 21% were diagnosed with something completely different than what their first physician concluded.
But in a related story, Slashdot reader sciencehabit writes that four machine-learning algorithms all performed better than currently-used algorithm of the American College of Cardiology, according to newly-published research, which concludes that "machine-learning significantly improves accuracy of cardiovascular risk prediction, increasing the number of patients identified who could benefit from preventive treatment, while avoiding unnecessary treatment of others."

"I can't stress enough how important it is," one Stanford vascular surgeon told Science magazine, "and how much I really hope that doctors start to embrace the use of artificial intelligence to assist us in care of patients."
This discussion has been archived. No new comments can be posted.

88% Of Medical 'Second Opinions' Give A Different Diagnosis - And So Do Some AI

Comments Filter:
  • by scsirob ( 246572 ) on Sunday April 16, 2017 @11:36AM (#54244821)

    So what the study finds is that doctors as well as technology disagree. There's nothing indicating that the second or third opinion is correct.

    • So matters are even worse than the articles suggest. Of course, the more sophisticated and complex medicine becomes, the more scope there is for errors, and the harder it becomes for doctors and surgeons to be sure of their diagnoses.

      • by ColdWetDog ( 752185 ) on Sunday April 16, 2017 @12:03PM (#54244909) Homepage

        Pretty much this. Cardiovascular risk is one of the best studied disease states known. Which is probably why they studied it. Even then, the 'AI' algorithms only improved risk stratification around 5% - nothing to sneeze at but hardly earth shattering.

        OK, now, for extra credit lets risk stratify middle age hypertensive diabetics who are depressed.

        Like the typical 'real world' patient. I'd just love some help here but the underlying data just doesn't support it. Which is kinda surprising since we've been studying these folks for a while. Simple medical problems are simple. Typical medical problems are not.

        • Simple medical problems are simple. Typical medical problems are not.

          Oh I don't know, I've always had great success treating patients using a course of leeches. A leech on the ear for ear ache, a leech on the bottom for constipation, just pop a couple down your codpiece before your go to bed.

          Signed: Dr. Hoffmann of Stuttgart.

          • by K10W ( 1705114 )
            I know you're joking of course but they still use leeches in mainstream medicine, used to be niche but they made a wider use comeback in the 90's. They have obvious benefits such as restore bloodflow in reconstructive surgeries where there would be loss of tissue from hypoxia etc but there is less obvious benefits. Not just when they're stimulating the bloodflow directly but after they're removed there is continued effects from the antithrombins they inject (Hirudin and something, I forget the other compoun
      • by ShanghaiBill ( 739463 ) on Sunday April 16, 2017 @04:38PM (#54245875)

        So matters are even worse than the articles suggest.

        Not necessarily. The study is an example of "selection bias". They only looked at cases where the patient requested a 2nd opinion. So these were cases where the patient already had a reason to question the 1st opinion. It is unlikely that the error (or "different diagnosis") rate would be as high for other cases.

        Before I go to a doctor, or take a family member, I do some online research to inform myself. If the doctor's diagnosis is significantly different from what I concluded, I ask for an explanation. If the doctor's "explanation" is that he is the expert and I should just trust him, then that usually means he is full of crap and doesn't know what he is doing.

        • It's worse than that. The second doctor likely knows it's a second opinion. And you likely picked a doctor that might not give you the same diagnosis (otherwise, why waste the money to likely hear the same thing again?). Tons of biases here. My question is always: do journalists repeating this know the these things, which are pretty basic, and just don't care because it fills pages and gets clicks? If so, there is something evil about taking advantage of Average Joe's lack of statistical/logical knowledge t

      • As usual, journalists and /. submitters knowingly or unknowingly use statistics and data to imply things they don't actually imply. Here, we are trying to suggest there is a high error rate in medical diagnoses. But the rate of different diagnosis given you seek a second opinion is not at all the same as the base rate of a different diagnosis if randomly selected individuals (or everyone) sought a second opinion. You seek a second opinion when you don't like the first opinion, or when it's so serious and yo

    • It is an important consideration that those that seek a second opinion often have reason to. It can be that the first doctor is uncertain, or the patient senses it, or the diagnosis doesn't seem right. It is not uncommon for doctors to recommend a second opinion when they are not sure or to make certain before treatment is started.

      The next questions is, how often is the first doctor's diagnosis correct?
      • I think you're on the right track....I have not RTFA, but the synopsis raises some concern about interpreting the conclusion. The data was taken from "General Internal Medicine Division", which are, typically, first line or Primary Care Physicians (PCPs). Often they will make a diagnosis based on their generalized knowledge of a medical problem and often refer them to a sub-specialist (i.e. you have trouble breathing with exertion, you find you need to sleep on multiple pillows, your ankles are swollen an

        • by jedidiah ( 1196 )

          So? Mayo is disagreeing with a bunch of GPs? No big surprise there. There are rare conditions that your average GP has never heard of whereas Mayo might have one of the top 5 experts in the whole world that treat that obscure condition.

    • There's more going on here than questioning the reliability of a doctor's opinion, or AI's statical improvement. One example of what I mean: Isn't it strange that some diagnoses seems to concur with the contours of state and county lines? Medical diagnoses has political overtones beyond those of the Affordable Care Act (ACA), aka, Obamacare. What the doctor tells you, whether she/he knows it or not, reflects these overtones.

      Money and Medical Care are intertwined to a point where the patient's health doesn

      • Good comments above!

        1) "Isn't it strange that some diagnoses seems to concur with the contours of state and county lines? Medical diagnoses has political overtones beyond those of the Affordable Care Act (ACA), aka, Obamacare. What the doctor tells you, whether she/he knows it or not, reflects these overtones."

        2) "Money and Medical Care are intertwined to a point where the patient's health doesn't matter."

        Copied the comments to help emphasize their importance. My research shows the same issues.
    • There's nothing indicating that the second or third opinion is correct.

      Or will lead to a different outcome. The article says that in 21% of cases the diagnosis was completely changed, but not what the outcome of the treatment was. For a non-medical analogy, consider you have a rotting deck that needs fixing. You ask five builders in for a quote and get give different ways of addressing the problem. Most of them will probably end up fixing your deck, but they're all slightly different. Does this mean any of them are right or wrong?

      An example from the medical field is blood

  • WebMD? (Score:3, Insightful)

    by houstonbofh ( 602064 ) on Sunday April 16, 2017 @11:45AM (#54244841)
    So, WebMD really is the better option... ;)
    • Nope. Google is your friend.

      Trust the computer.

    • by AmiMoJo ( 196126 )

      So, WebMD really is the better option... ;)

      Maybe. Really, I'm not joking.

      Doctors are presented with limited evidence. Basically some test results and what you tell them. They will have areas they are more familiar with, and areas they are less familiar with. You on the other hand know a lot more about your lifestyle, your medical history (because the notes are never complete or even accurate, try reading them some time) and most importantly your lived experience.

      I've had incorrect diagnosis in the past. When the right one was eventually found it jus

      • People lie. To themselves, to their doctors, to the computer. They'll put in their symptoms, get a bad diagnosis and start shopping for something a little better by tweaking their symptoms, etc.

        The miracle is that anyone gets a proper diagnosis ever.

      • This isn't to defend bad doctors -- of which there are many -- or anything, but you are not a doctor, you don't possess medical expertise which even generalists have, and to be frank, your "lived experience," while not totally worthless, is more than likely not nearly as good an indicator for medical diagnoses as you seem to think it is. Much of the time, your recollection of what you lived is going to be incomplete, biased, or straight-up counterfactual (note that I didn't imply lying... you can be quite w

      • Unfortunately, the training for doctors still focusses far more on memorising a large subset of known conditions and not on effectively using expert systems to refine a diagnosis.
  • Of course (Score:5, Insightful)

    by Gibgezr ( 2025238 ) on Sunday April 16, 2017 @12:02PM (#54244903)

    I would expect that many people who ask for a second opinion have a reason to ask for a second opinion: in fact, the article even mentions the situation where the first doctor recommended they do so. What would be more telling would be if they did a large study and gave EVERYONE second opinions, and then told us how many differed.
    This smells like another case of "lies, damned lies, and statistics", which is designed to make the Mayo Clinic look good.

    • Re:Of course (Score:4, Interesting)

      by Registered Coward v2 ( 447531 ) on Sunday April 16, 2017 @12:27PM (#54245001)

      I would expect that many people who ask for a second opinion have a reason to ask for a second opinion: in fact, the article even mentions the situation where the first doctor recommended they do so. What would be more telling would be if they did a large study and gave EVERYONE second opinions, and then told us how many differed. This smells like another case of "lies, damned lies, and statistics", which is designed to make the Mayo Clinic look good.

      Correct. A sample size of 286 is pretty small and without knowing details on the data it's hard to draw from conclusions, unless you want to put out a press release. As for the /. header, 88% didn't give a different diagnosis 21% did, with some 67% getting a refined or redefined diagnosis. I would guess that the second opinion MD, knowing what the first said, would focus in on those aspects of the diagnosis to confirm or refine it; medicine is an inexact science, I'd be more worried about 90% agreement than the numbers from this study. More to the point, what was the actual conditions of the patients and what was the accuracy of diagnosis once the condition was positively identified.

      Mayo later points out that this study supports getting second opinions even though they cost money. Duh, an outfit that makes money off of MD visits supports more visits? I'm shocked, shocked. Next thing you know there'll be gambling at Rick's...

      What was interesting is the conclusion "There were no significant differences between provider types;" i.e. PA's and NPs did just as well as MDs; so using them as the primary entry point in healthcare may be one good way to lower costs while maintaining the quality of care.

      As for AI, it certainly is good to use it to aid in diagnosis, as it can learn about specific conditions and continue to build a database to draw from, far beyond an MD's ability to see a broad range of patients to help refine his or her diagnostic capability. AI is good at drawing conclusions from large datasets but not good at recognizing other symptoms that only manifest themselves in person, such as odors, odd way of walking or speaking, that can clue an MD into looking further.

    • Re:Of course (Score:5, Interesting)

      by wisnoskij ( 1206448 ) on Sunday April 16, 2017 @01:54PM (#54245371) Homepage

      I was thinking more of the opposite. A doctor who is asked for a second opinion knows the patient does not want the same diagnosis. He knows that the patient is shopping around for the "best" diagnosis. He knows that the only way he is likely to be able to start treatment is to give a different diagnosis. I would say that medical diagnosis are complicated things that patients are not likely to be able to gauge correctly. If a patient is better at diagnosis than a doctor, and better able to tell if the correct one has been given, why even use doctors?

      • by Xenna ( 37238 )

        So I guess I'm lucky that I got the same diagnosis/treatment advice from the second specialist...

    • It's a textbook case of self-selection bias.
      Does anyone have a second opinion on that point?
    • I would agree with this sentiment. Yet, would also like to point out another caveat...

      88% discrepancy in any study, on any pool of subjects, seems worthy of a closer investigation.
      There are, in fact, way too may misdiagnosed cases coming from our supposedly high quality medical profession.
      I have, in fact, encountered a few shitty doctors here in the USA. One of them at the Mayo Clinic in AZ!

      How about improving our medical system: Require second opinions for those relevant cases.
      Then, after the diagno
  • I'm not supposed that to doctors will give different diagnoses, but they also have to work with what they're given. A patient may give different details, especially if they've just spoken to a GP - they're likely to preempt questions that had to be asked previously.

    This goes with every profession. A colleague of mine was being investigated by HMRC (tax) and was complaining that his accountant was still telling him it was ok to claim fuel, but the tax man wasn't. The problem was he didn't understand that by

  • If only 12% of the time second opinions confirm the original diagnosis and 21% of the time they find something completely different, then we should obviously skip the initial diagnosis and just go for the second opinion.
    • If only 21% of the time the first diagnosis was thrown out then sticking with the original therapy was probably right in 80% of the cases. That's not too bad.
      Then for that 21% you have to consider a differential diagnosis is list of possible diagnoses listed somewhat in the order of plausibility. To what extent the doctor was just giving the next item in the list without really contradicting the first one? Was the second diagnosis better? Some patients just shop for a doctor to confirm a diagnosis. I'm temp

  • Another thing to keep in mind is that a chart is not some purely dispassionate recording of standardized metrics. Even before you get to the diagnosis, you are already looking at pre-processed information, and not raw data.

    Although there are some parameters recorded for most patients regardless of the issues at hand (such as vital signs, or maybe listening to the patient's heart/lungs), other history and data is selected -- much of the history, review of systems, and physical exam is performed as a way of

    • > In any case, a doctor who is asked for a second opinion gets the luxury of having both the first doctor's records (hopefully), as well as a data point occurring later in time.

      Thank you for posting, as this would not have occurred to me if you hadn't.

      Without RTFA (because I'm lazy right now), I'd love to see them use the expert system in parallel to doctors, complete with initial and follow-up visits to see what the difference is when the same doctor gets that second data point.

      I would still expect the

    • Guppy's post above shows the benefit of more time and treatment data, looping back to a doctor. This is what makes keeping good records so important for each doctor visit.

      Different doctors, and different kinds of drs, have different diagnostic models and skills, but could use all the data. DItto therapeutics.
  • by Impy the Impiuos Imp ( 442658 ) on Sunday April 16, 2017 @01:10PM (#54245159) Journal

    concludes that "machine-learning significantly improves accuracy of cardiovascular risk prediction

    Computer: Subject has 728% increase in cardiovascular heart disease risk due to obesity, sedentary lifestyle, nuggets, and left-handed masturbation to furry Dragon Ball drawings.

    Impressed Doctor: God damn.

  • One doctor saying you have cancer while another says it is the flu is definitely 'different'. If both say you have lung cancer, but the second says it is a slight variation of the cancer diagnosed by the first doctor, does that meet the definition of 'different' for this study?
  • by AndyKron ( 937105 ) on Sunday April 16, 2017 @02:21PM (#54245453)
    Maybe the people who went for a 2nd diagnosis had reasons to do so.
  • I used to work at a hospital and I think I can explain the basis for this. Doctors are clueless, refuse to use technology, they rush everything, and they're incredibly lazy. I don't mean some of them either. 100% of them.
    • Re: (Score:2, Insightful)

      by Anonymous Coward

      "I used to work at a hospital and I think I can explain the basis for this. Doctors are clueless, refuse to use technology, they rush everything, and they're incredibly lazy. I don't mean some of them either. 100% of them."

      Right....'Used to work at a hospital,' now more clueful than 100% of doctors, by his own estimate. With rippling abs I'll bet, too. Thank god we can all bask in this magnificent being's radiance! "My name is slashmydots, King of Kings; Look on my Works, ye Mighty, and despair!"

      What an ass

  • by Chris Katko ( 2923353 ) on Sunday April 16, 2017 @04:07PM (#54245753)

    It should be read as: of people who were UNSATISFIED with their first doctor to the point they got a second opinion, were correct in their belief.

    Not that 88% of doctors visits are wrongly identified. 99.999% of people who get a cold, and get some antibiotics, and feel better in a couple of days, aren't seeking a second doctor.

    • by kanweg ( 771128 )

      But they should get another doctor! The cold isn't cured by antibiotics (which don't kill virusses; only bacteria).

      Bert

      • by Anonymous Coward

        A lot of the problem there is the idiotic patient.

        I had a co worker telling me all about her horrible cold and how she had to fight her horrible doctor to get antibiotics for it, isnt that horrible.
        I tried explaining that antibiotics wont kill the cold virus, and she immediately got defensive saying she needs them, and its the only thing that helps her get over her cold. I remember thinking I hope the Dr just gave her a placebo.

        I have family members dealing with antibiotic resistant strains of bacteria and

  • by Anonymous Coward on Sunday April 16, 2017 @09:51PM (#54246833)

    I needed FIVE medical opinions to get to the correct diagnosis of my condition, and I had to go to Mexico to get it.

    Thank God that the doctor in Mexico correctly diagnosed my lung cancer, because FOUR American doctors kept on putting me off, two of them accusing me of being psychosomatic and telling me "nothing is wrong with me," and the other two incorrectly diagnosing me of, get this, allergies and asthma, respectively... because coughing up blood and having a constant metallic taste in the mouth are definitely characterizing symptoms of allergies and asthma.

    The good news is that I have civil suits against four well-insured American doctors and an insurance company now that will guarantee my family's financial well-being once I am gone, which will be in 3-9 months from now, especially given that, had my cancer been correctly diagnosed 3 years ago, it would have been operable and I likely would have had a good chance of survival.

  • be careful. just because two doctors conclude the same or similar doesn't make it right. doctors frequently just back each other up following your chart notes in terms of diagnosis and are afraid to change diagnosis. you may not have a condition at all and still receive multiple false diagnosis. it's a really big problem if say you need medical care and you're in a prison or state hospital and you need care but they deny diagnosis; and if you don't need care for mental illness but they frame you to look men

  • You're ugly too!

  • This sounds an awful lot like a Bayesian Trap, also called a Base Rate Bias. See here for a decent explanation.
    https://www.youtube.com/watch?... [youtube.com]

    weylin

It is not best to swap horses while crossing the river. -- Abraham Lincoln

Working...