Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
AI IBM Medicine Technology

IBM Pitched Its Watson Supercomputer as a Revolution in Cancer Care. It's Nowhere Close (statnews.com) 108

Posted by msmash from the reality-check dept.
IBM began selling Watson to recommend the best cancer treatments to doctors around the world three years ago. But is it really doing its job? Not so much. An investigation by Stat found that the supercomputer isn't living up to the lofty expectations IBM created for it. It is still struggling with the basic step of learning about different forms of cancer. Only a few dozen hospitals have adopted the system, which is a long way from IBM's goal of establishing dominance in a multibillion-dollar market. And at foreign hospitals, physicians complained its advice is biased toward American patients and methods of care. From the report: The interviews suggest that IBM, in its rush to bolster flagging revenue, unleashed a product without fully assessing the challenges of deploying it in hospitals globally. While it has emphatically marketed Watson for cancer care, IBM hasn't published any scientific papers demonstrating how the technology affects physicians and patients. As a result, its flaws are getting exposed on the front lines of care by doctors and researchers who say that the system, while promising in some respects, remains undeveloped. [...] Perhaps the most stunning overreach is in the company's claim that Watson for Oncology, through artificial intelligence, can sift through reams of data to generate new insights and identify, as an IBM sales rep put it, "even new approaches" to cancer care. STAT found that the system doesn't create new knowledge and is artificially intelligent only in the most rudimentary sense of the term.
This discussion has been archived. No new comments can be posted.

IBM Pitched Its Watson Supercomputer as a Revolution in Cancer Care. It's Nowhere Close More

IBM Pitched Its Watson Supercomputer as a Revolution in Cancer Care. It's Nowhere Close

Comments Filter:

  • Greed rules all. (Score:2, Flamebait)

    by geekmux ( 1040042 )

    I'm curious, after a few dozen hospitals adopted Watson, did the "revolution" it created cause a negative impact to the massive amounts of revenue created by maintaining the status quo within the Cancer Industrial Complex?

    If so, you have your answer as to why adoption would die off faster than someone mentioning a cure...

    • Re:Greed rules all. (Score:4, Insightful)

      by jedidiah ( 1196 ) on Wednesday September 06, 2017 @11:16AM (#55147807) Homepage

      Why do you think it would do that? Being better at getting the right diagnosis quickly isn't going to make things any cheaper. If you think otherwise then you're probably just a deranged pot head or vegan kidding themselves.

      A lot of cancers are rare and difficult to deal with. Your random guy at "General Hospital" is going to have no clue. He won't even know well enough to throw the $10K per month med at the patient.

      PubMed on steroids could actually be quite useful for the average doctor who's not at a world leading treatment center like Mayo.

      • Re: (Score:1)

        by Anonymous Coward

        You really have no idea how much money is squandered annually on tests [latimes.com] that are performed by rote and have little chance of revealing the underlying issues.

        • Re: (Score:1)

          by jedidiah ( 1196 )

          > You really have no idea

          Not only do I have an "idea", I also know what those tests really cost because I actually pay attention to my medical bills.

          On the other hand, I quite literally know of patients KILLED for lack of robust testing. This strange deranged "too much testing" mentality is probably getting a lot of people killed.

          This mentality especially makes no sense in socialized medicine where ALL of the associated costs have already been paid for and really there is no good reason to not have a dia

      • Re:Greed rules all. (Score:5, Insightful)

        by ColdWetDog ( 752185 ) on Wednesday September 06, 2017 @11:30AM (#55147883) Homepage

        Except Mayo is not a 'world leading cancer treatment center'. It's one of literally thousands of places with excellent oncology teams.

        Who look at the exact same data as Watson.

        And come up with pretty much exactly the same result. Sans Watson.

        The clueless doc at General Hospital doesn't even figure into this. If a patient has a complex / rare / difficult cancer they get referred to a regional cancer center. A lot of cancer treatments are pretty straightforward due to the large number of trials that have been done over the years. The databases have existed for decades and obviously are getting better and more complete over time. The real killer, so to speak, for Watson is that it could never really beat the industry standard 'tumor board' composed of various meat space biologic computers. Perhaps one day. When AI is actually a bit more than a marketing term.

        • Re: (Score:2, Troll)

          by jedidiah ( 1196 )

          > The clueless doc at General Hospital doesn't even figure into this. If a patient has a complex / rare / difficult cancer they get referred to a regional cancer center.

          Perhaps in your ideal TV world it does. In the real world, this quite often does not happen.

          The problem with a rare cancer is KNOWING to look for it. If you don't KNOW what you're looking for, then it's hard to find it. THAT is the problem that better knowledge tools can help solve.

          Quite often doctors don't know to send you out to the "on

      • Why do you think it would do that? Being better at getting the right diagnosis quickly isn't going to make things any cheaper. If you think otherwise then you're probably just a deranged pot head or vegan kidding themselves.

        You've got to be fucking kidding me if you think reducing the amount of tests and getting more efficient at diagnosing cancer does not impact revenue. Talk about delusional...

        • Re: (Score:2)

          by jedidiah ( 1196 )

          What tests do you think they can skimp on? I know someone who died from a very survivable cancer because they skimped on testing.

          You simply don't understand how expensive cancer treatment is. I am not talking about the diagnostic tests leading up to the treatment, but the treatment itself. Even the most expensive and esoteric diagnostic tests are CHEAP compared to the cost of actual treatment.

          You don't really understand anything about this subject. All you know are some deranged conspiracy theories you've h

  • I've seen better from high school students (Score:5, Interesting)

    by TimothyHollins ( 4720957 ) on Wednesday September 06, 2017 @11:06AM (#55147769)

    And at foreign hospitals, physicians complained its advice is biased toward American patients and methods of care.

    Are you seriously telling me that they sold a multi-million dollar machine and didn't even include a goddamn machine learning step to adapt to local variations? Aren't the IBM guys supposed to be experts? Or at least guys that know how to pick up a fucking phone and dial an expert?

    This is the kind of rookie mistake I see in my undergrads...
    I sure hope I'm reading this wrong, because it sounds like people might die from maltreatment over this.

    At its heart, Watson for Oncology uses the cloud-based supercomputer to digest massive amounts of data — from doctor’s notes to medical studies to clinical guidelines. But its treatment recommendations are not based on its own insights from these data. Instead, they are based exclusively on training by human overseers, who laboriously feed Watson information about how patients with specific characteristics should be treated.

    Ahh I guess I was wrong. There is no machine learning at all yet.

    In the case of Watson for Oncology, those human operators are a couple dozen physicians at a single, though highly respected, U.S. hospital: Memorial Sloan Kettering Cancer Center in New York. Doctors there are empowered to input their own recommendations into Watson, even when the evidence supporting those recommendations is thin.

    But hey, looks like the dying part could be correct. I only hope those doctors know what the variations across the world requires, because they will be giving recommendations both for japanese highschool girls and African village elders without even knowing it, and I don't think those groups have the same contextual issues.

    • Re: (Score:3, Interesting)

      by CraigDavidIsTheShit ( 5075993 )
      Watson is literally a big decision tree from the bits i've seen/worked on

      • Re:I've seen better from high school students (Score:4, Interesting)

        by Anonymous Coward on Wednesday September 06, 2017 @11:19AM (#55147831)
        I took an IBM Watson course at my University (Partnered with IBM), and their current head researcher was our Prof for the 6 week course. From what I remember it was essentially ~34 modules attempting various methods of Brute Forcing a solution, with several modules grading their outputs. But the actual modules from what I remember were infinitely basic and it was just their multitude that actually lead to improvements in results. I remember walking away from that course having learned little about how Machine Learning is done at the upper echelons of computing, but having a 1 year license to actually use the BlueMix platform. We even had to create a project using the BlueMix API, but it only allowed us to use a pre-imported corpus submitted by the TAs, so we didn't even get to take a whack at classifying training sets. Anyway, I was very unimpressed, and I was the only First-year Bachelor student in that masters course. Even I felt it was all a bit basic.
        • Yep sounds about right, from what I experienced at IBM it was just throw more hardware at this so we out-perform our competitors Also inside IBM nobody would use bluemix..
          • One of the things that I learned upon talking to IBM developers is that they have to pay standard amounts to use IBM products. I work for a relatively small software company, but anything that we do is available to anybody else in the company. We have a lot of stuff that is narrowly focused for a customer and could not be re-used, but we have libraries, apps, frameworks, and various tools (for devops, for example) that we use freely. IBM doesn't do that; you want to use Watson? You have to pay for it,

      • Re: (Score:2)

        by jedidiah ( 1196 )

        Well, that's a bit worthless now isn't it.

        Even as an American cancer patient, I don't want other methods from abroad to be ignored. HELL, even in the US there are differences of opinions regarding treatment. I even get different ideas from two oncologists that work together in the same office. A system like this should be able to sort through all that stuff and find most successful approach for an individual patient.

        • While that's true, I would bet most of the complaints are about recommendations involving expensive equipment that some smaller hospitals overseas don't have. If you have that available, it's likely the best option. If it's not, then it's a huge waste of time.

        • Search on "oncologists would not have chemotherapy".

          Boosting the bodies own defenses against cancer in various ways (including nutrition, intermittent fasting, immune-system tuning, etc.) is another approach at least generally without negative side effects -- wonder if Watson has been fed enough alternative data to recommend it (especially for prevention)?

          Example: https://www.drfuhrman.com/lear... [drfuhrman.com]
          "Cancer screening is promoted as preventive health, and while this may detect early forms of cancer so it can be

    • At its heart, Watson for Oncology uses the cloud-based supercomputer to digest massive amounts of data — from doctor’s notes to medical studies to clinical guidelines. But its treatment recommendations are not based on its own insights from these data. Instead, they are based exclusively on training by human overseers, who laboriously feed Watson information about how patients with specific characteristics should be treated.

      To be fair, this is close to how automated driving is being handled as well. A much more complex problem than diagnosing cancer.

    • Re: (Score:3)

      by hey! ( 33014 )

      So supervised learning doesn't count as "machine learning"? It sound like a fairly generic classification setup.

    • Re: (Score:3)

      by pesho ( 843750 )

      Ahh I guess I was wrong. There is no machine learning at all yet.

      In the case of Watson for Oncology, those human operators are a couple dozen physicians at a single, though highly respected, U.S. hospital: Memorial Sloan Kettering Cancer Center in New York. Doctors there are empowered to input their own recommendations into Watson, even when the evidence supporting those recommendations is thin.

      That's all you need to know from the article. The value of machine learning is the ability to find subtle trends by processing in unbiased fission large and diverse data sets. Instead they fed it a limited data set that was strongly biased by the fact that it is based on 12 oncologist who coordinate their decision (Tumor boards, where all oncologist will gather and review each case is the standard practice in US hospitals). Then there is the "standard of care" approach which is a must for US hospitals - onc

      • Re: (Score:2)

        by jedidiah ( 1196 )

        Except the whole "standard of care" idea implodes when you treat cancer. Seemingly identical patients with the same condition and treatment quite often exhibit wildly differing results. You really can't take a "one size fits" all approach.

        Cancer seems to be one area in particular that takes all of the apparent progress we have made and make it seem like we really don't know anything at all yet.

  • Despite the hype, computers don't learn anything. They run programs in the same way they always have. The term "AI" is just hype to attract venture capital and avoid doing real work.

    • Well, come on, there's a big difference between a program where the human sifts through the data and codes in rules of some form, and a program that sifts through the data and comes up with the rules (explicitly or implicitly).

      I'm a huge skeptic about the _effectiveness_ of much-vaunted AI revolution in general based on everything I've done in machine learning and everything I've read and seen; and the Watson revolution specifically because of and what my friend who helped build Watson has told me, but you'

    • That's not entirely true. There are algorithms to handle machine learning but most of them require a trial-and-error approach. Computers can burn through thousands of choices and outcomes and pick the most effective.

      The downside to this type of learning in medicine is that you have to kill thousands of people before you get the "best" treatment.

    • Re: (Score:2)

      by hord ( 5016115 )

      They essentially "learn" the same way your brain does by connecting, inspecting, and re-configuring weighted pathways. If they don't learn I'm not sure what you are doing.

  • I know it's AI Doom and Gloom, but Winter is Coming

  • The people at IBM thought it was about "caring about people who are cancer (astrological sign)".

  • FTA: https://www.statnews.com/2017/... [statnews.com]

    Pilar Ossorio, a professor of law and bioethics at University of Wisconsin Law School, said Watson should be subject to tighter regulation because of its role in treating patients. “As an ethical matter, and as a scientific matter, you should have to prove that there’s safety and efficacy before you can just go do this,” she said.

    Norden dismissed the suggestion IBM should have been required to conduct a clinical trial before commercializing Wats

    • Wait a minute...I certainly hope they test parachutes in the appropriate manner! It may not be a randomized trial, but that's because parachutes are different from diagnosing and treating medicine.

    • “Has there ever been a randomized trial of parachutes for paratroopers?”

      Without even looking, I'd say "yes, there was" -- just not with live human beings, probably human-shaped dummies that weighed the same as their human counterparts. Idiot doesn't know how rigorously the military expects it's contractors to test and prove things before accepting them.

  • Sorry to be cynical, but... (Score:5, Insightful)

    by hyades1 ( 1149581 ) <hyades1@hotmail.com> on Wednesday September 06, 2017 @11:44AM (#55147963)

    When captains of industry are talking about cancer treatment in terms of "establishing dominance in a multibillion-dollar market", does any rational person believe we're going to have an actual cure for cancer any time soon?

    • Re: (Score:2)

      by swb ( 14022 )

      What's wrong here is that healthcare is seen as a growth market, as if the sources of healthcare funding were infinite and there was room for continuous long-term revenue growth.

      If anyone really believed we had a way to solve the health care access problems, they probably wouldn't be investing in health care as a business sector because it would likely be a market facing at best flat costs if not declining overall spending.

    • Sure I do. I imagine the first company to cure cancer will get a huge bump in stock price in the short term, which the CEO will then cash out on as he switches to, I dunno, making cars. Along the way, he'll be lauded as a hero.

      There aren't many times when I think the stock market's short term profit obsession is good, but this is one of them.

      • I hope you're right. I am afraid, though, that this is one of the times where corporations will look at long-term trillions over short-term billions.

  • Does Watson give explanations (Score:3)

    by Shotgun ( 30919 ) on Wednesday September 06, 2017 @11:45AM (#55147969)

    For something this critical, I would want the AI to explain itself. If an "expert" tells me that I need to that I need to take a pill for my cholesterol (intentionally choosing something rather minor), I will first ask her why this medicine and how does it work. I will expect to get a cogent explanation before paying for the drug. If I'm being asked to pay thousands of dollars for a cancer treatment, I'd expect someone to explain how the medicine works, and show me how it has been successful in other cases.

    There are AI solutions that will show how they arrived at a recommendation. Intel has some AI that uses the feature as a selling point. Why would anyone just say, "Hmm? The computer says to give 'em hypocholoroacetiminophin. Wonder why? Where's my needle.", without getting an explanation?

    • In this particular case, that's the problem: The machine is incapable of 'thinking'. It's not 'conscious'. There is no 'personality'. You can't talk to it. It can't relate to you in any way whatsoever. It's all just 'information'. There are no machines that actually 'think'. We won't have those for a long, long time to come -- if at all, ever.
    • In ML that's called "Feature Importance". Some algos can do it. For example, if you're running a decision tree based model such as Gradient Boosting, you can determine which particular pieces of data were considered important in a prediction.
      But I'd be willing to bet that's not what Watson is using. IBM is more concerned with making headlines than functional products. So they'll of course go right for the Neural Nets (Works like the human brain!1!) . Which is of course a black box that requires a huge

  • Comment removed (Score:5, Interesting)

    by account_deleted ( 4530225 ) on Wednesday September 06, 2017 @11:53AM (#55148013)
    Comment removed based on user account deletion

    • Re: (Score:1, Insightful)

      by Anonymous Coward

      Can you please spell out the whole words instead of the failed abbreviations.

      • NLP = Natural Language Processing or maybe Neural-Linguistic Programming (is there a difference? I don't know)
        UIMA = Unstructured Information Management Architecture
        SPSS is statistical analysis software https://en.wikipedia.org/wiki/... [wikipedia.org]

        originally stood for Statistical Package for the Social Sciences (SPSS)

  • In Jeopardy (Score:4, Funny)

    by theendlessnow ( 516149 ) * on Wednesday September 06, 2017 @11:54AM (#55148019)
    Most of the complaints are that you have to give Watson the answer first and then it gives you the question. Doctors were hoping for something the other way around.
  • all posts about AI are completely overblown.
  • Machine learning algorithms can be powerful when used on a narrowly focused problem or goal, but curing cancer is definitely not a narrowly focused problem. There are aspects of machine learning that might prove to be useful but it's not a catch all solution whereby we simply say, "Hey Watson, cure cancer" and expect it to churn out a cure. What is sounds like they are doing is inputting doctor notes and hoping for Watson to apply treatment plans based on historical success of past similar patients. Firstl
  • An AI system delivering well below what was promised. Must be a world first.

Slashdot Top Deals

On the eighth day, God created FORTRAN.

Close