IBM Pitched Its Watson Supercomputer as a Revolution in Cancer Care. It's Nowhere Close (statnews.com) 108
IBM began selling Watson to recommend the best cancer treatments to doctors around the world three years ago. But is it really doing its job? Not so much. An investigation by Stat found that the supercomputer isn't living up to the lofty expectations IBM created for it. It is still struggling with the basic step of learning about different forms of cancer. Only a few dozen hospitals have adopted the system, which is a long way from IBM's goal of establishing dominance in a multibillion-dollar market. And at foreign hospitals, physicians complained its advice is biased toward American patients and methods of care. From the report: The interviews suggest that IBM, in its rush to bolster flagging revenue, unleashed a product without fully assessing the challenges of deploying it in hospitals globally. While it has emphatically marketed Watson for cancer care, IBM hasn't published any scientific papers demonstrating how the technology affects physicians and patients. As a result, its flaws are getting exposed on the front lines of care by doctors and researchers who say that the system, while promising in some respects, remains undeveloped. [...] Perhaps the most stunning overreach is in the company's claim that Watson for Oncology, through artificial intelligence, can sift through reams of data to generate new insights and identify, as an IBM sales rep put it, "even new approaches" to cancer care. STAT found that the system doesn't create new knowledge and is artificially intelligent only in the most rudimentary sense of the term.
Nobody ever died buying IBM (Score:2)
Re: (Score:3)
Google "IBM and the Holocaust"
Re: Nobody ever died buying IBM (Score:1)
Fuck off creimer. Someone mod down.
Re: (Score:2)
Re: (Score:1)
Are the customers paying the bills we send them?
>> Yes.
Then...yes, it's doing it's job.
Greed rules all. (Score:2, Flamebait)
I'm curious, after a few dozen hospitals adopted Watson, did the "revolution" it created cause a negative impact to the massive amounts of revenue created by maintaining the status quo within the Cancer Industrial Complex?
If so, you have your answer as to why adoption would die off faster than someone mentioning a cure...
Re:Greed rules all. (Score:4, Insightful)
Why do you think it would do that? Being better at getting the right diagnosis quickly isn't going to make things any cheaper. If you think otherwise then you're probably just a deranged pot head or vegan kidding themselves.
A lot of cancers are rare and difficult to deal with. Your random guy at "General Hospital" is going to have no clue. He won't even know well enough to throw the $10K per month med at the patient.
PubMed on steroids could actually be quite useful for the average doctor who's not at a world leading treatment center like Mayo.
Re: (Score:1)
You really have no idea how much money is squandered annually on tests [latimes.com] that are performed by rote and have little chance of revealing the underlying issues.
Re: (Score:1)
> You really have no idea
Not only do I have an "idea", I also know what those tests really cost because I actually pay attention to my medical bills.
On the other hand, I quite literally know of patients KILLED for lack of robust testing. This strange deranged "too much testing" mentality is probably getting a lot of people killed.
This mentality especially makes no sense in socialized medicine where ALL of the associated costs have already been paid for and really there is no good reason to not have a dia
Re:Greed rules all. (Score:5, Insightful)
Except Mayo is not a 'world leading cancer treatment center'. It's one of literally thousands of places with excellent oncology teams.
Who look at the exact same data as Watson.
And come up with pretty much exactly the same result. Sans Watson.
The clueless doc at General Hospital doesn't even figure into this. If a patient has a complex / rare / difficult cancer they get referred to a regional cancer center. A lot of cancer treatments are pretty straightforward due to the large number of trials that have been done over the years. The databases have existed for decades and obviously are getting better and more complete over time. The real killer, so to speak, for Watson is that it could never really beat the industry standard 'tumor board' composed of various meat space biologic computers. Perhaps one day. When AI is actually a bit more than a marketing term.
Re: (Score:2, Troll)
> The clueless doc at General Hospital doesn't even figure into this. If a patient has a complex / rare / difficult cancer they get referred to a regional cancer center.
Perhaps in your ideal TV world it does. In the real world, this quite often does not happen.
The problem with a rare cancer is KNOWING to look for it. If you don't KNOW what you're looking for, then it's hard to find it. THAT is the problem that better knowledge tools can help solve.
Quite often doctors don't know to send you out to the "on
Re: (Score:2)
Why do you think it would do that? Being better at getting the right diagnosis quickly isn't going to make things any cheaper. If you think otherwise then you're probably just a deranged pot head or vegan kidding themselves.
You've got to be fucking kidding me if you think reducing the amount of tests and getting more efficient at diagnosing cancer does not impact revenue. Talk about delusional...
Re: (Score:2)
What tests do you think they can skimp on? I know someone who died from a very survivable cancer because they skimped on testing.
You simply don't understand how expensive cancer treatment is. I am not talking about the diagnostic tests leading up to the treatment, but the treatment itself. Even the most expensive and esoteric diagnostic tests are CHEAP compared to the cost of actual treatment.
You don't really understand anything about this subject. All you know are some deranged conspiracy theories you've h
I've seen better from high school students (Score:5, Interesting)
And at foreign hospitals, physicians complained its advice is biased toward American patients and methods of care.
Are you seriously telling me that they sold a multi-million dollar machine and didn't even include a goddamn machine learning step to adapt to local variations? Aren't the IBM guys supposed to be experts? Or at least guys that know how to pick up a fucking phone and dial an expert?
This is the kind of rookie mistake I see in my undergrads...
I sure hope I'm reading this wrong, because it sounds like people might die from maltreatment over this.
At its heart, Watson for Oncology uses the cloud-based supercomputer to digest massive amounts of data — from doctor’s notes to medical studies to clinical guidelines. But its treatment recommendations are not based on its own insights from these data. Instead, they are based exclusively on training by human overseers, who laboriously feed Watson information about how patients with specific characteristics should be treated.
Ahh I guess I was wrong. There is no machine learning at all yet.
In the case of Watson for Oncology, those human operators are a couple dozen physicians at a single, though highly respected, U.S. hospital: Memorial Sloan Kettering Cancer Center in New York. Doctors there are empowered to input their own recommendations into Watson, even when the evidence supporting those recommendations is thin.
But hey, looks like the dying part could be correct. I only hope those doctors know what the variations across the world requires, because they will be giving recommendations both for japanese highschool girls and African village elders without even knowing it, and I don't think those groups have the same contextual issues.
Re: (Score:3, Interesting)
Re:I've seen better from high school students (Score:4, Interesting)
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
Well, that's a bit worthless now isn't it.
Even as an American cancer patient, I don't want other methods from abroad to be ignored. HELL, even in the US there are differences of opinions regarding treatment. I even get different ideas from two oncologists that work together in the same office. A system like this should be able to sort through all that stuff and find most successful approach for an individual patient.
Re: (Score:2)
While that's true, I would bet most of the complaints are about recommendations involving expensive equipment that some smaller hospitals overseas don't have. If you have that available, it's likely the best option. If it's not, then it's a huge waste of time.
No surprise -- most chemotherapy seems problematic (Score:3)
Search on "oncologists would not have chemotherapy".
Boosting the bodies own defenses against cancer in various ways (including nutrition, intermittent fasting, immune-system tuning, etc.) is another approach at least generally without negative side effects -- wonder if Watson has been fed enough alternative data to recommend it (especially for prevention)?
Example: https://www.drfuhrman.com/lear... [drfuhrman.com]
"Cancer screening is promoted as preventive health, and while this may detect early forms of cancer so it can be
Re: (Score:2)
At its heart, Watson for Oncology uses the cloud-based supercomputer to digest massive amounts of data — from doctor’s notes to medical studies to clinical guidelines. But its treatment recommendations are not based on its own insights from these data. Instead, they are based exclusively on training by human overseers, who laboriously feed Watson information about how patients with specific characteristics should be treated.
To be fair, this is close to how automated driving is being handled as well. A much more complex problem than diagnosing cancer.
Re: (Score:3)
So supervised learning doesn't count as "machine learning"? It sound like a fairly generic classification setup.
Re: (Score:3)
Ahh I guess I was wrong. There is no machine learning at all yet.
In the case of Watson for Oncology, those human operators are a couple dozen physicians at a single, though highly respected, U.S. hospital: Memorial Sloan Kettering Cancer Center in New York. Doctors there are empowered to input their own recommendations into Watson, even when the evidence supporting those recommendations is thin.
That's all you need to know from the article. The value of machine learning is the ability to find subtle trends by processing in unbiased fission large and diverse data sets. Instead they fed it a limited data set that was strongly biased by the fact that it is based on 12 oncologist who coordinate their decision (Tumor boards, where all oncologist will gather and review each case is the standard practice in US hospitals). Then there is the "standard of care" approach which is a must for US hospitals - onc
Re: (Score:2)
Except the whole "standard of care" idea implodes when you treat cancer. Seemingly identical patients with the same condition and treatment quite often exhibit wildly differing results. You really can't take a "one size fits" all approach.
Cancer seems to be one area in particular that takes all of the apparent progress we have made and make it seem like we really don't know anything at all yet.
Re: (Score:3)
From the 2017 Amazon review it appears that all of the URLs in the text are now bad, after one year. Not sure how severely that impacts the value of the book but the reviewer only gave it two stars. So that "four star" average rating should be taken with a grain of salt. I certainly would not buy it new, you can get it for a little more than half the (allegedly) discounted Amazon new price.
Re: (Score:2)
And looking at the contents of the book with Amazon preview it looks like it is entirely about using Watson's services, instead of being about Watson's architecture, implementation or technology - on other words, understanding anything about Watson itself.
In that context, where web access is the whole point of the book, bad URLs would make it next to useless.
Re: (Score:1)
Criemer, could you please stop posting these. Just because you're posting anon doesn't mean we don't know it's you.
Computers don't learn (Score:2)
Re: (Score:3)
Well, come on, there's a big difference between a program where the human sifts through the data and codes in rules of some form, and a program that sifts through the data and comes up with the rules (explicitly or implicitly).
I'm a huge skeptic about the _effectiveness_ of much-vaunted AI revolution in general based on everything I've done in machine learning and everything I've read and seen; and the Watson revolution specifically because of and what my friend who helped build Watson has told me, but you'
Re: (Score:2)
Okay, yes, agreed there that creativity is not what these algorithms do. And when they try to, they are terrible.
Re: (Score:2)
That's not entirely true. There are algorithms to handle machine learning but most of them require a trial-and-error approach. Computers can burn through thousands of choices and outcomes and pick the most effective.
The downside to this type of learning in medicine is that you have to kill thousands of people before you get the "best" treatment.
Re: (Score:2)
They essentially "learn" the same way your brain does by connecting, inspecting, and re-configuring weighted pathways. If they don't learn I'm not sure what you are doing.
Re: (Score:2)
Winter Is Coming (Score:2)
Re:Watson is elementary (Score:4, Insightful)
No. There is no such thing as "AI" (yet). It's possible but probably not within our lifetimes. It's taken us well over 30 years since functional MRI came into the market and we're just beginning to understand what general areas of the brain are involved in doing "something", let alone individual neurons and synapses.
To claim the ability to "create intelligence" when we don't even understand the question yet is hubris (and salesmanship) on the side of IBM. Trust me, there will be several boom-bust cycles before AI becomes 'intelligent'.
Re: (Score:2)
Re: Watson is elementary (Score:1)
Re: (Score:2)
Raises the question if AI is sufficiently advanced, and actually is able to think for itself -- will it operate in a kind of a black-box fashion, and if so, how will we know it's arrived at 'AI' ?
Re: (Score:2)
Well, if it's a black box without input or output, then you can say whatever you want about the computer, it doesn't make any difference whether it has if it doesn't "do" anything.
To arrive at "AI" you need to be able to evaluate it's inputs and it's outputs and see whether they are sufficiently intelligent. As I said, we haven't defined intelligence ourselves yet but it seems to involve being able to make progressively 'better' decisions based on past experiences, and we know we can do that (somewhat). The
Miscommunication (Score:2)
The people at IBM thought it was about "caring about people who are cancer (astrological sign)".
Re: Hey slashdot did the same thing (Score:1)
Rrandomized trial of parachutes? (Score:2)
Pilar Ossorio, a professor of law and bioethics at University of Wisconsin Law School, said Watson should be subject to tighter regulation because of its role in treating patients. “As an ethical matter, and as a scientific matter, you should have to prove that there’s safety and efficacy before you can just go do this,” she said.
Norden dismissed the suggestion IBM should have been required to conduct a clinical trial before commercializing Wats
Re: (Score:2)
Wait a minute...I certainly hope they test parachutes in the appropriate manner! It may not be a randomized trial, but that's because parachutes are different from diagnosing and treating medicine.
Re: (Score:2)
“Has there ever been a randomized trial of parachutes for paratroopers?”
Without even looking, I'd say "yes, there was" -- just not with live human beings, probably human-shaped dummies that weighed the same as their human counterparts. Idiot doesn't know how rigorously the military expects it's contractors to test and prove things before accepting them.
Sorry to be cynical, but... (Score:5, Insightful)
When captains of industry are talking about cancer treatment in terms of "establishing dominance in a multibillion-dollar market", does any rational person believe we're going to have an actual cure for cancer any time soon?
Re: (Score:2)
What's wrong here is that healthcare is seen as a growth market, as if the sources of healthcare funding were infinite and there was room for continuous long-term revenue growth.
If anyone really believed we had a way to solve the health care access problems, they probably wouldn't be investing in health care as a business sector because it would likely be a market facing at best flat costs if not declining overall spending.
Re: (Score:2)
That's a genuinely interesting point of view.
Re: (Score:2)
Sure I do. I imagine the first company to cure cancer will get a huge bump in stock price in the short term, which the CEO will then cash out on as he switches to, I dunno, making cars. Along the way, he'll be lauded as a hero.
There aren't many times when I think the stock market's short term profit obsession is good, but this is one of them.
Re: (Score:2)
I hope you're right. I am afraid, though, that this is one of the times where corporations will look at long-term trillions over short-term billions.
Does Watson give explanations (Score:3)
For something this critical, I would want the AI to explain itself. If an "expert" tells me that I need to that I need to take a pill for my cholesterol (intentionally choosing something rather minor), I will first ask her why this medicine and how does it work. I will expect to get a cogent explanation before paying for the drug. If I'm being asked to pay thousands of dollars for a cancer treatment, I'd expect someone to explain how the medicine works, and show me how it has been successful in other cases.
There are AI solutions that will show how they arrived at a recommendation. Intel has some AI that uses the feature as a selling point. Why would anyone just say, "Hmm? The computer says to give 'em hypocholoroacetiminophin. Wonder why? Where's my needle.", without getting an explanation?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
But I'd be willing to bet that's not what Watson is using. IBM is more concerned with making headlines than functional products. So they'll of course go right for the Neural Nets (Works like the human brain!1!) . Which is of course a black box that requires a huge
Comment removed (Score:5, Interesting)
Re: (Score:1, Insightful)
Can you please spell out the whole words instead of the failed abbreviations.
Re: (Score:3)
NLP = Natural Language Processing or maybe Neural-Linguistic Programming (is there a difference? I don't know)
UIMA = Unstructured Information Management Architecture
SPSS is statistical analysis software https://en.wikipedia.org/wiki/... [wikipedia.org]
originally stood for Statistical Package for the Social Sciences (SPSS)
In Jeopardy (Score:4, Funny)
See, that's why... (Score:2)
The Data is flawed (Score:1)
Who would have thought (Score:2)