IBM Promised Its AI Platform Watson Would Be a Big Step Forward in Treating Cancer. But After Pouring Billions Into the Project, the Diagnosis is Gloomy. (wsj.com) 90
Can Watson cure cancer? That's what IBM asked soon after its AI system beat humans at the quiz show "Jeopardy!" in 2011. Watson could read documents quickly and find patterns in data. Could it match patient information with the latest in medical studies to deliver personalized treatment recommendations? "Watson represents a technology breakthrough that can help physicians improve patient outcomes," said Herbert Chase, a professor of biomedical informatics at Columbia University, in a 2012 IBM press release. Six years and billions of dollars later, the diagnosis for Watson is gloomy [Editor's note: the link may be paywalled; alternative source]. WSJ: More than a dozen IBM partners and clients have halted or shrunk Watson's oncology-related projects. Watson cancer applications have had limited impact on patients, according to dozens of interviews with medical centers, companies and doctors who have used it, as well as documents reviewed by The Wall Street Journal. In many cases, the tools didn't add much value. In some cases, Watson wasn't accurate. Watson can be tripped up by a lack of data in rare or recurring cancers, and treatments are evolving faster than Watson's human trainers can update the system. Dr. Chase of Columbia said he withdrew as an adviser after he grew disappointed in IBM's direction for marketing the technology. No published research shows Watson improving patient outcomes. IBM said Watson has important cancer-care benefits, like helping doctors keep up with medical knowledge.
Nobody ever got fired for buying IBM... (Score:4, Funny)
after they replaced the HR system, no one could figure out how.
Re: (Score:1)
Re: (Score:2, Interesting)
Was there 22 years. It used to be a place where solutions mattered. Last few years was a lesson in selling smoke and mirrors. Everything is built on a stack of lies. The Watson-jeopardy team had all questions to all answers ahead of time.
Re: (Score:2)
Insofar what an expert does follows some kind of logic, you could say the same for him.
Human minds have quite a diverse and useful bag of tricks, which is what makes human experts so versatile. I think we're at the point where we can reproduce individual tricks from that bag and perform them with inhumanly repetitive perfection. But that perfection is actually a liability, because it leaves not room for common sense, which throws spanners into logical works all the time.
Training (Score:3)
This is the advantage we have over AI algorithms: our brains can receive, interpret and learn from a massive variety of data and they do this every hour we are awake. This gives us a huge brea
Re: (Score:2)
I had something slightly different in mind. To a hammer AI trained on nails, everything looks like (possibly a very weird) nail. Yes, as a result of life experience, in a human's head the spanner will clamor to be used.
Re: (Score:2)
"Common sense" is usually wrong, though. It is a cliche that generally refers to "mount stupid," knowingly or not.
Re: (Score:3)
I wouldn't say it's "usually" wrong; that's way too strong. But it sure is unreliable.
Re: AI is bullshit (Score:1)
Troll
Re:AI is bullshit (Score:4, Interesting)
While I agree with you that the current artificial ignorance (A.I.) (*) that tries to pass for Artificial Intelligence is a joke -- nothing more then a glorified table lookup -- you're jumping the gun to say "no: intelligent computers will never happen". i.e. a.i. = Actual Intelligence.
THE fundamental problem is that Scientists don't know what the fuck consciousness is. Without a way to measure it you can't copy it (or create it.)
Let's pretend it is 100 years in the future, and we have ways to:
* Upload
* Download
consciousness. With the ability to "clone" consciousness we _actually_ would have a way to have an intelligent computer. But yeah, until we get to THAT point, we're (probably) barking up the wrong tree with "just throw hardware at it."
Second, you are ignoring bio-computing. If tomorrow's computers switched from using electricity to using chemistry, much the way a physical body does, then again, intelligent computers is within the realm of possibility.
The million dollar question is: How do we get there? I'm not aware of anyone knowing. If they do know, they sure as hell aren't saying -- and I can't blame them. Think of the implications: If we could clone human consciousness effectively death would be wiped out which I'm sure there are enough bad sci-fi writers out there who have discussed this before.
Your lament about the sorry state of software reminds me of that old Murphy's Computer Law [imgur.com] joke:
Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization.
(*) Yes, A.I = artificial ignorance was intentional.
Re: (Score:2)
What kind of tripe is this that you're spewing? While I agree that replicating consciousness seems like it would be an impressive and groundbreaking accomplishment, it's certainly not a pre-requisite for intelligence.
Re: (Score:2)
> consciousness ... is certainly not a pre-requisite for intelligence.
OK, prove it. Show me intelligence *without* it (*).
(*) Once we agree on _exactly_ what consciousness is.
Whose billions? (Score:2)
If it was IBM's gamble, then nothing of value was lost. If, however, the billions were invested by medical teams duped by impossible promises, then that's a different story.
(Story is paywalled and I'm too lazy to read TFA before commenting.)
Pouring billions into Watson? (Score:2)
I doubt IBM poured billions into oncology Watson. All it was doing was creating recommendations for doctors from reading patient reports and treatment research reports.
IBM sure poured billions into Watson and it's their biggest future product. The oncology seems like a small side project.
Re: (Score:2)
Re: (Score:1)
dichloroacetate [newscientist.com] is probably what OP is referring to
Re: (Score:2)
Didn't it show early promise, but not work?
Re: Cancer is cured already (Score:1)
Yup, kills the liver. Considering the huge grants and Nobel prizes and personal fulfillment that would come from curing cancer, the idea of a secret cabal is on level with rumors of Exxon hiding the technology for 500mpg cars.
IBM should get into eye health (Score:4, Funny)
Judging from the other comments, IBM's AI system has improved human vision to a perfect 20-20 hindsight.
Re: (Score:2)
Eye health is already too competitive [slashdot.org]
watson's finger (Score:4, Informative)
Watson beat people at Jeopardy because it always got to answer first, because its button-pressing finger was faster than a human's button-pressing response. A fairer assessment of Watson's Jeopardy-playing abilities vs humans would have Watson respond with the same button-mashing-delay profile as its competitors. Beyond that, the relevant question is not whether Watson can beat humans at Jeopardy, or mash a button faster than a human, but whether it can analyze data better than a human to detect cancer (or solve whatever medical problem). And for the most part, it doesn't matter whether the answer comes back in 10 ms, 300 ms, or a a minute, or an hour. Like with any other tool, the question is whether it can help get the job done better for a reasonable price.
Re: (Score:3)
Re: (Score:2)
But with an online course, he may be better than Watson.
Re: watson's finger (Score:2, Informative)
Watch the PBS NOVA on Watson. They accounted for the delay for humans to buzz in and intentionally made the computer wait to give a âoefairâ chance. The rest of your post is spot on about fast turnaround not being applicable in all use cases.
Re: (Score:1)
I saw that special, and they mostly just uncritically reported what IBM was saying, which was a tad one-sided. Ken Jennings talks in more detail about the buzzer advantage (or not) in this blog post [ken-jennings.com]. While he starts off by saying "[s]ome have called this an unfair advantage; I’m inclined to think of it as a fair one", he goes on to say this:
I think that's a key quote. He writes quite a bit to defend IBM,
Re: (Score:2)
AI is not a fast path (Score:2)
Today's AI is not really there yet, today it is just a big automated filter and matching machine there really is not any intelligence behind it ATM.
No one has figured out how to program a concept in to data. No one knows yet how to program thought, consciousness in to an algorithm. At least that I know of.
Just my 2 cents
It's TIME to STOP (Score:1)
I know the Washington Post does it but that doesn't mean that putting periods in headlines looks any less fucking retarded. And don't go starting with Huffington Post "This Is The Most Scandalous Detail Of What Happened, And Here's How You Should Feel About It" headlines either. If I wanted entertainment/news I would go to Maddox's site because even though he sucks, he's still better at it than any of you ever will be.
Not really suprised (Score:2)
Re: (Score:2)
It's not spitting out nonsense (generally), it's just not doing better than a person would.
The reason is that going off datamining isn't guaranteed to give you better results. YOu have to have a situation where all that data is actually meaningful.
The problem with oncology is that there are some cancers where a specific mechanism can be targeted, there's test for that situation, and a specific drug. A human can read a + on a piece of paper and prescribe the appropriate drug just as well as Watson can.
The ot
Re: (Score:2)
So again, Watson does about the same as a person would.
Except if you read up on it, it didn't. It did worse. In some cases giving out completely inappropriate treatment suggestions a newly minted oncology resident would know are bullshit. Much of this has been attributed to mistakes and biases made in its training.
Re: (Score:2)
If the overall performance is similar to a human, but some of the mistakes it makes are as easily identified by a human as you presume, then having the human reviewing the machine performance would give a better result than just the human or just the computer.
Alas, it was just bullshit, and the humans also make mistakes that look stupid to other humans.
Re: (Score:2)
they put the cart before the horse (Score:2)
For AI to shine with anything this complicated, you really need detailed and consistent testing. This is where tech needs to go first in medicine. In very many areas, it would help to have a massive increase in testing and fidelity of the tests.
After the success of the genome project, we should have moved to a massive effort to develop cost efficient full system scanning and testing instead of starting the brain project.
We should launch a project to figure out how to measure every aspect of health imaginabl
The current approach to AI is a DEAD END. (Score:3, Insightful)
Now, what they should be investing 'billions and billions of dollars' in, is research and development of newer, better instrumentation for observing a living brain in action (and I do NOT mean 'a better fMRI, I mean invent something that's a new and different approach). Only when we can see the total system in action will we even have a chance to understand how it works, the problem being that once it's dead, it's dead, and dissecting it isn't going to show you what you need to see.
Re: (Score:2)
I've said it before, I'll keep saying it: until we actually understand how a biological brain produces the phenomena we call 'thinking', we will not be able to create 'machine intelligences' that match or exceed human beings. Period. Now, what they should be investing 'billions and billions of dollars' in, is research and development of newer, better instrumentation for observing a living brain in action (and I do NOT mean 'a better fMRI, I mean invent something that's a new and different approach).
There's a HUGE market today for the kinds of things that the current machine-learning approach does plenty well enough at at looks like it has a lot more growth area - license-plate recognition, facial recognition, image recognition, medical imagining recognition in the case of Deep Mind. It looks like the billions that are being invested today are a good investment.
And you propose switching that investment to a speculative thing that might bear fruit in 50-100 years time, and if it did then the result woul
Re: (Score:2)
And you propose switching that investment to a speculative thing that might bear fruit in 50-100 years time, and if it did then the result would be a general-purpose intelligence that replaces a lowly-paid human being? Why should someone invest billions in that?
They're putting short-term profits ahead of something that isn't garbage. Face it: the so-called 'AI' they keep trotting out has had billions invested in it, thinking it's going to be Just Another Design Cycle, and it turns out that it f
Re: (Score:2)
That's the point it DOESN'T 'do the job plenty well enough' it always falls short of the mark because it has ZERO capacity to actually THINK, your dog or cat has better cognitive ability, and people will trust this half-assed excuse for AI too much and disasters will happen.
I think what we've discovered is that the "capacity to actually think" is by and large unimportant for most of the needs we have -- good-enough large scale image recognition, good-enough medical imaging assessments, and others that I listed.
Disasters? You'll have to spell out why you say the "ability to think" or general-purpose intelligence will lead to fewer disasters rather than more. Personally, I reckon it would lead to more disasters just through sheer complexity and unpredictability and un-debuggabil
Re: (Score:2)
I think what we've discovered is that the "capacity to actually think" is by and large unimportant for most of the needs we have
Yeah? Who the hell is this 'we' you're referring to? Not anyone I've ever talked to. I think you're making that up, and the 'we' is actually just 'you'.
If a "thinking" industrial robot kills a human, how the heck will we debug that or fix it?
At least you can then ask it why it did what it did, instead of even the programmers that wrote it telling you "We have no idea why it did that", which is the current 'state of the art' in AI; even the programmers have no idea what's going on 'under the hood' when it's running.
Re: (Score:2)
[I think what we've discovered is that the "capacity to actually think" is by and large unimportant for most of the needs we have] ... Yeah? Who the hell is this 'we' you're referring to? Not anyone I've ever talked to. I think you're making that up, and the 'we' is actually just 'you'.
As you wrote, "that's all they've been obsessively focusing on". The "they" in your sentence are the ones who presumably think that deep learning is good enough for their purposes, else they wouldn't pouring their billions into it. I'm just describing to you what the market has perceived, which you yourself observed too.
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
I never understood why people think image recognition is new "AI". It isn't. License plate readers (and image recognition) have been around for decades.
Image recognition was indeed around for decades. It was based on convolutions for edge-detection, haar cascades for face recognition. You'll have used these if you had a camera with face detection up to a few years ago. If you've coded with OpenCV you'll have used these APIs, e.g. https://docs.opencv.org/3.4.1/... [opencv.org]
It was an old technology that had run its course and really was a dead end. It wasn't making progress. It required too much custom human coding for things you wanted to recognize, and it was hit-or
Re: (Score:3)
> until we actually understand how a biological brain produces the phenomena we call 'thinking', we will not be able to create 'machine intelligences' that match or exceed human beings.
I don't know; depends on your definitions. To me, an intelligent machine is defined by its behavior, not by its internal design or building materials. We can probably build something that behaves very closely to a human, even though internally it's built of switching silicon, or ants running through tubes.
Historically, our
Re: (Score:2)
It's a dead end if you want to build Lt. Cmdr. Data, but honestly I just want a glorified Roomba. I mean we struggle to make a burger flipping robot, imagine having a chef in your kitchen 24x7x365 and for bonus points it'll set the table, be your waiter and clean the dishes. And that can't just vacuum the floors but scrub the toilet, dust the furniture, rinse the sink, clean the windows and so on. And that can take my dirty laundry, sort it, wash it, dry it, iron my shirts and hang them in my closet. I don'
Not yet there (Score:5, Informative)
Maybe because finding patterns without actually understanding anything is not really "intelligence". The AI hype is slowly dying and even non-IT/non-science-related people finally have finally come to a realization that ... not understand how these trained networks operate and that turns them into a black box you cannot really trust and which is bound to give absolutely wrong results.
1) AI is not a magical pill that can solve all the problems in the world
2) There isn't too much "intelligence" in AI
3) Coding real intelligence is a lot harder than using throwing reinforced convolutional neural networks at everything
4) We do
It's not like we understand how the human brain operates but we have certain reasons to believe it's mostly rational, intelligent and infallible (with exceptions, of course) since it has got us here - the age of technology and an improved quality and increased length of life which no other animal has been able to achieve.
I'm not against reinventing the biological intelligence that the human beings possess but it surely looks like we haven't come close to it.
No published research? Or no need? (Score:2)
Would a doctor want to publish research showing that their expensive need is diminishing?
Re: (Score:3)
Re: (Score:2)
Citation, please.... for all of your assertions.
Re: (Score:2)
Re: (Score:2)
Please provide a citation for your claim that conversation about technical subjects is restricted to published academic materials.
If IBM had bothered to check with the experts (Score:3)
Well professional bionformaticians had already been working on the problem of personalized medicine and medical diagnosis before IBM and Watson got involved. If you listen to them, there is a clear consensus of how that is is going to work in the future.
Part 1: Because of a dependence of both disease and the effectiveness of treatments upon personal genetics, every person will get sequenced at birth. That will do at least three things: reduce what otherwise appears as statistical noise in assessing treatment efficacy by resolving interdependencies between the treatment and personal genetics, improve estimates of the likelihood of any individual developing a disease or disorder, and help to identify the best treatments for specific individuals.
Part 2: Every patient treatment and its outcome become a trail logged into a massive database along with the patient's medical history and genetics. Currently, massive amounts of information about the effectiveness of treatments is discarded because the records of treatment after a drug is released are not accumulated. Now, before a drug is introduced to the market, there are clinical trials on a subpopulation, and that becomes an authoritative record of the drug effectiveness. That is a tiny fraction of the potential information out there and insufficient to assess interactions of drugs with other factors such as genetics.
One of the barriers to implementing that system is the price of sequencing, about $1000.00/person. Prices are projected to fall until sequencing becomes ubiquitous.
The other barrier is privacy legislation (HIPAA) and financial incentives acting on institutions against information sharing. Despite endless funded government initiatives to implement sharable electronic medical records, patient medical information remains siloed within provider and insurance networks. Rather than work to share information, those institutions are competing to build the largest silo. (This circumstance exemplifies a typical type of government ineptitude, which is to continuously and futilely throw enormous sums of money at a problem rather than simply and cheaply reforming the legislation and regulation giving rise to the perverse incentives which created the problem. Information sharing for research medical use to benefit personalized medicine was the main driver behind the U.K. slackening medical records privacy, demonstrating that in the U.K. not all government officials are complete idiots.)
Finally, the main point of this post: The bioinformaticians have wished for that future because they knew that the problem of personalized medicine was information-starved before IBM threw billions at the problem. Given adequate information, the computational solutions of personalized medicine are already known by those humans with domain-specific expertise.
If IBM had instead invested those billions in reducing the cost of sequencing further and in lobbying government to fix the stupid incentives and restrictions acting against medical information sharing, the problem could have been solved by now. Another case of someone with a hammer looking for nails by pounding on things to see if they move.
Had Watson been genuinely intelligent, it would explained all that that IBM.
Re: (Score:2)
We need stronger laws.
Wrong. We need smarter laws.
Re: (Score:2)
Smart laws. Smart. Just stick them on the blockchain. /s
I do totally agree though. But without a national ballot measure system, there isn't any sort of mechanism to implement smart laws. You could write a smarter law, give it to your congress critter, and even if they got it passed, it would be a totally different law that they actually passed.
Our whole system of government is built around making sure the government isn't too smart! Smart governments are too good at protecting their own power.
Oh noes (Score:2)
Theranos? (Score:2)
Par for the course for AI (Score:3)
Re: (Score:2)
Apologize - mouse slipped resulted in bad moderation. Please mod this up - it is a good comment.
Evolutionary Origins of Cancer (Score:1)
IBM Is Missing Out (Score:3)
They are missing out on a major opportunity here.
IBM has this AI thingie called Watson. Right now they are tasking it with cancer related programming.
Don't they have enough nerds there to convince their bosses to give it proctology programming?
Then after they feed the patient a laxative that they can call No Shit Sherlock, the AI's controller can put it to work with the command, "Dig Deeper Watson."
Watson IBM is like my old doctor (Score:2)
He's just not as good as a young experienced one. Old doctors assume they've seen it all and 70% of the customers are diagnosed as 'stomach flu' anyway, just on general principle.
And nobody is as old as IBM.
Treatments are evolving too fast (Score:3)
If "treatments are evolving faster than Watson's human trainers can update the system", wouldn't the same be true for oncologists trying to keep up with the latest and greatest?
Re: (Score:2)
All sophisticated modern AI systems (except possibly Deep Learning) suffer from the knowledge acquisition problem https://en.wikipedia.org/wiki/... [wikipedia.org] Deep learning has its own different problems (ex. how did it get that answer? nobody knows!), but if things change you can basically just retrain it with the newest data.
As your system/model gets bigger and bigger with more and more moving parts (dimensions, sub-models, rules, algorithms, data, etc.), it becomes more and more brittle, because all these different
This is exactly what I thought would happen (Score:3)
"...treatments are evolving faster than Watson's human trainers can update the system"
And if you do away with the mass of humans doing a particular area of expertise and turn it over to "AI" you will freeze that area at that level of AI expertise. The AI has no motivation array. It can't look for ways to "do it better," as humans constantly do. It cannot advance the field. AI is potentially a human disaster if too much trust is given it. Remember, Watson was built, programmed, turned on to play Jeopardy by humans. Then it was turned off when it had satisfied the motivations of its creators. It did not WANT to play Jeopardy, or anything else. It is a rock, a tool. Nothing else. Unmotivated intelligence is not intelligence. We must not rely on it.