Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
AI Medicine

Leaked Training Shows Doctors In New York's Biggest Hospital System Using AI (404media.co) 31

Slashdot reader samleecole shared this report from 404 Media: Northwell Health, New York State's largest healthcare provider, recently launched a large language model tool that it is encouraging doctors and clinicians to use for translation, sensitive patient data, and has suggested it can be used for diagnostic purposes, 404 Media has learned. Northwell Health has more than 85,000 employees.

An internal presentation and employee chats obtained by 404 Media shows how healthcare professionals are using LLMs and chatbots to edit writing, make hiring decisions, do administrative tasks, and handle patient data. In the presentation given in August, Rebecca Kaul, senior vice president and chief of digital innovation and transformation at Northwell, along with a senior engineer, discussed the launch of the tool, called AI Hub, and gave a demonstration of how clinicians and researchers—or anyone with a Northwell email address—can use it... AI Hub can be used for "clinical or clinical adjacent" tasks, as well as answering questions about hospital policies and billing, writing job descriptions and editing writing, and summarizing electronic medical record excerpts and inputting patients' personally identifying and protected health information.

The demonstration also showed potential capabilities that included "detect pancreas cancer," and "parse HL7," a health data standard used to share electronic health records.

The leaked presentation shows that hospitals are increasingly using AI and LLMs to streamlining administrative tasks, and shows that some are experimenting with or at least considering how LLMs would be used in clinical settings or in interactions with patients.

Leaked Training Shows Doctors In New York's Biggest Hospital System Using AI

Comments Filter:
  • Essentially enshittification in the medical system. Well, as far as that has not yet happened in the US.

    • Im fine with this. Doctors are so overworked that they have to literally minimize the number of seconds they interact with each patient. If the system shaves 60 minutes of reading charts off their day, and they then need to spend 10 minutes double checking the LLM for errors, that’s still an extra 50 minutes they have to do extra stuff

      Is it a potential source for error? Sure. But, it’s not like the current system has a stellar record when it comes to medical errors.
      • by Amouth ( 879122 )

        how you train it really maters.. a good example is a past attempt that resulted in the realization that it didn't learn how to identify skin cancer at all, but rather learned to identify rulers..

        https://www.sciencedirect.com/... [sciencedirect.com]

        • And who vets the training and how? Is it going to rely on ancient MySpace tombstones?

          As we enjoy this musing tale, many hundreds of thousands of poison pages are purposefully trying to make AI do things that are unintended, and it's working. And it's only just begun.

          Checking grammar is one thing, medicine uses a long vocabulary of strange names and interactions. Procedures, however, are something completely different.

          Depending on AI is like depending on your drunk brother to drive you home in a hurricane.

  • Certification? (Score:3, Interesting)

    by Teun ( 17872 ) on Sunday November 03, 2024 @06:26PM (#64917115)
    I assume several of the jobs this AI is taking over require certification, who is certifying this AI?
  • Hope their hospital lawyers have their depends strapped on.

  • by peterww ( 6558522 ) on Sunday November 03, 2024 @07:44PM (#64917205)

    There are TONS of health care companies, products, etc that are very publicly and explicitly using AI in healthcare settings. Just Google it for god's sake. It's not a secret. Did you just think nobody would buy it? That nobody would use it? Are you nuts?! Of course they will.

    And it's not all as garbage as you might imagine. They're not just strapping ChatGPT to a prescription pad. There's a lot of well-thought out products that already had tons of useful functionality baked in. A lot of the time they're just adding some AI bullshit around an existing product so they can sell it.

    You literally can't sell any product today unless the word "AI" is in it. The first thing the customer's going to do is ask you "But does it have AI? Your competitor has AI. Does that mean your product isn't as good?"

    Please don't let the news media freak you out with their clickbait articles. Yeah, AI is bullshit. But it's not always as bad as it sounds.

  • Considering that HL7 is in itself a form of metastasized cancer this shouldn't come as a surprise...

  • Not really new (Score:5, Informative)

    by Retired Chemist ( 5039029 ) on Sunday November 03, 2024 @08:15PM (#64917231)
    A cardiologist friend of mine has complained for years about the automated system his hospital (a famous teaching institute) used to read EKGs. It could read ordinary ones quite well but was unreliable compared to trained human on unusual cases. I suspect that these AI systems will have the same problem. It is difficult to train such a system to recognize rare cases because they will be poorly represented in the data set. The basic problem is that people tend to have an exaggerated faith in automatic systems and are generally too overworked or lazy to double check them.
    • This is a great example. If there’s one thing that a computer should be able to do well, it would be to interpret the simple two dimensional data of an EKG. They can’t even do that well. The one use I see for having AI is to respond to the tons of emails that patients write and that doctors are now forced to respond to ( or get a dreaded one star review. Because that’s how doctors are now judged.)
    • A cardiologist friend of mine has complained for years about the automated system his hospital (a famous teaching institute) used to read EKGs. It could read ordinary ones quite well but was unreliable compared to trained human on unusual cases. I suspect that these AI systems will have the same problem. It is difficult to train such a system to recognize rare cases because they will be poorly represented in the data set. The basic problem is that people tend to have an exaggerated faith in automatic systems and are generally too overworked or lazy to double check them.

      hmmm. You raise a good point about limitations in specialty cases, but I wonder if we’re approaching a tipping point for AIs in routine settings. An ophthalmologist and an oral surgeon friend of mine, both of whom run their own practices, asked me decades ago when they’d be able to hand off the routine tasks of diagnosing and prescribing—corrective lenses for one, dental fixtures for the other—to an expert system. At the time, back in the late ’90s and early 2000s, my answer as

  • Do schools ban cell phones because teachers are stupider and less entertaining and more sarcastic than just chatting with an AI about whatever tweaks your curiosity?

  • by ZipNada ( 10152669 ) on Sunday November 03, 2024 @09:58PM (#64917309)

    "answering questions about hospital policies and billing, writing job descriptions and editing writing, and summarizing electronic medical record excerpts and inputting patients' personally identifying and protected health information"

    Relatively simple jobs that the machine is likely to do reasonably well. Probably it can help with some diagnoses. Seems innocuous to me.

  • AIs that are specialized in detecting cancer should be used to detect cancer. They are very good at it.

    AIs that aren't specialized in a particular task tend to cause problems. That's where we need to be very careful.

    Just don't confuse the two types together.

"It's like deja vu all over again." -- Yogi Berra

Working...