Slashdot stories can be listened to in audio form via an RSS feed, as read by our own robotic overlord.

 



Forgot your password?
typodupeerror
Medicine The Internet

Web-Crawling Program Spots Disease Outbreaks 52

Posted by timothy
from the ebola-ebola-ebola-ebola-ebola-dropsy dept.
no1home writes "There is a story at Discovery Channel's site about a new utility for mapping disease. The premise is to have bots crawl the web looking for stories about disease outbreaks and log them onto a map. '"We were originally thinking about how we could expand disease surveillance and pick up outbreaks earlier than traditional methods," said John Brownstein of Harvard Medical School and Children's Hospital Boston, who created HealthMap in September of 2006 with Clark Friefeld, a software developer at Harvard Medical School.' But then it was noticed by Google.org and has since grown into its own website, HealthMap Global disease alert map, and claims to be able to identify 95% of all disease outbreaks, some of them before WHO or CDC."
This discussion has been archived. No new comments can be posted.

Web-Crawling Program Spots Disease Outbreaks

Comments Filter:
  • Great! (Score:3, Funny)

    by Apple Acolyte (517892) on Sunday July 20, 2008 @12:43AM (#24259731)
    Now all the hypochondriac hyper-nerds have another reason to sit home on their computers, cloistered from the outside world. :-)
    • Well, it's not really a "map of disease"
      breakouts. In fact the map part is rather just
      a shiny pony?

      A list could have done just the same amount
      of good. Since for the most part each area has
      one pushpin that just sums up the area.
      [FWIW, I only looked at US pins.]

      I was expecting a cluster map, like you see on...
      Wunderground Wundermaps [wunderground.com]

      or on...
      http://www.housingmaps.com/ [housingmaps.com]

      At least if it was a cluster map I could
      look at an area and think, "I sure as heck
      ain't traveling there for

      • At least if it was a cluster map I could look at an area and think, "I sure as heck ain't traveling there for work this week.

        On a global scale, check out the RSOE EDIS [hisz.rsoe.hu] (Emergency and Disaster Information Service).

        It aggregates all sorts of disasters, from short-time events such as automobile accidents, and current tropical storms, to longer term ones, such as epidemics and forest fires.

    • Is this the opposite of the "Manly Joe" who never lets on that he is sick and takes computer lessons from the Amish?

  • CLOSE THE PORTS [quartertothree.com]
    • Re: (Score:2, Interesting)

      by Anonymous Coward

      How about a bot that spots new memes by checking Wikipedia articles for repeated vandalism and sudden article protection.

  • Catch the video (Score:5, Informative)

    by Joe Decker (3806) on Sunday July 20, 2008 @12:52AM (#24259783) Homepage

    Fascinating TED Talk [ted.com] on a similar (the same?) project? As I recall, some of video was a bit unpleasant to watch, but (IMHO) very worthwhile.

  • I still think (Score:2, Interesting)

    by drDugan (219551)

    a facebook or other social app for people to self report symptoms is a great idea that no one has uilt yet. one could even "out" symptoms of their friends or speculate which friends made them sick. lots of issues with it, but a different data source for inf disease folks, even if the data was not completely accurate, would be helpful in predictions.

    too busy to do it myself now...

    • > one could even "out" symptoms of their friends or speculate which friends made them sick. lots of issues with it, but a different data source for inf disease folks, even if the data was not completely accurate, would be helpful in predictions.

      Yeah, right, just what we need, an inaccurate resource for the insurance companies to data-mine. Your premium has now increased by a factor of 5, just because someone with your name (Mike Smith) allegedly made someone else sick. Great.

      No thanks.

    • Re: (Score:3, Funny)

      by dynchaw (1188279) *
      Today
      Jenny Smith gave you the clap! Give her measles? 3:56pm
    • Re: (Score:2, Funny)

      by Anonymous Coward

      Better still, look at what symptoms people are searching for. Sure, you'll get a little noise every time a repeat of "House" is aired. However, when google gets systemic swaths of "butt bleeding" , "grey vomit" and "ocular hemorrhoids", bad things might be coming.

  • by AndGodSed (968378) on Sunday July 20, 2008 @01:29AM (#24259893) Homepage Journal

    as far as I can make out. It relies heavily on human reporting. And sometimes it takes a while for news on disease outbreaks to make the news.

    Unless there is some way to report directly TO this crawler, I seriously doubt the claim that a web crawler can know of outbreaks before the WHO does.

    hmm... I just referenced The Who - a band...

    • Band *!synonym;
      *!synonym=Band.TheWho.BandTitle;



      //wait a second..
  • Neat (Score:3, Interesting)

    by symes (835608) on Sunday July 20, 2008 @01:35AM (#24259923) Journal
    This is neat - although how useful it is I don't know... It'll be pretty obvious that communitites not tied to the www 24/7 will be sorely under-represented. Also, the disease categories seem a bit narrow - it would be cool to have stuff like murder, violence and alcohol related-disease in there. Ok, not transmissible diseases in their own right but they still have some pretty profound health-related consequences.
  • Usefulness? (Score:4, Insightful)

    by Ender_Wiggin (180793) on Sunday July 20, 2008 @01:40AM (#24259943)

    The CDC, and local and state health departments all have a list of "reportable" diseases. (Things from TB to gonnorhea to ebola to SARS) If a doctor encounters them, they are supposed to notify the health authorities. That is for biostatistics and epidemiology purposes.

    If they have to look these cases up in the news instead of getting notified by hospitals and clinics, then the system is in a really bad shape.

    • Re: (Score:3, Insightful)

      by symes (835608)

      If they have to look these cases up in the news instead of getting notified by hospitals and clinics, then the system is in a really bad shape.

      Very true - but might there be value in understanding the public's awareness of disease? One thing that this map might measure is a communitites awareness of transmissible disease and awareness *should* lead to protective behaviour. So if there's a mismatch between regular epidemiological stats and this map then perhaps public health bods should going in there telling people to wear condoms, wash their hands, etc.

    • Re: (Score:3, Insightful)

      by jamesh (87723)

      The CDC, and local and state health departments all have a list of "reportable" diseases. (Things from TB to gonnorhea to ebola to SARS) If a doctor encounters them, they are supposed to notify the health authorities. That is for biostatistics and epidemiology purposes.

      What about non-reportable diseases? German Measles, Chicken Pox, and many others are not reportable, and most people wouldn't even bother going to the doctor if their kids came down with them (or is that not the case anymore? seems like every

  • by bm_luethke (253362) <luethkeb.comcast@net> on Sunday July 20, 2008 @01:44AM (#24259955)

    What the designers expect from this will highly color my opinion on it. The article linked isn't exactly clear on this.

    Do you want to track and try and predict disease breakouts in first world areas then probably decent, track world wide stuff then terrible. Outside of the obvious (self reporting) there is the whole issue of how much of the world is on the internet? While much of the first and even quite a bit of the second world countries are on the vast majority of the population doesn't have computers, let alone internet access.

    I can easily see many many great uses for this and I expect all of them to be explored at some point - I can also clearly see many not so great uses and I fully expect them to be used too. As the old saying goes, there are lies, damn lies, and statistics.

    In fact we can already see the maps being posted and used by people who have little to no understanding (if we are generous, I'm sure some understand and use them to further their own aims) to say things the data *can not say* and it isn't even mainstream yet. *sigh* It's like many things we have today - the greater amount of good it can do the greater amount of abuse one can use it for too.

    • Just because a doctor doesn't live in the first world, doesn't mean he's somehow incapable of reporting it.

      More to the point, internet based tracking has already proven it's worth in the SARS outbreak. The first clusters of what came to be known as SARS cases were located by GPHIN, and reported to the WHO, who didn't themselves issue a report on SARS for weeks.

      What's needed now is development of this infrastructure, with doctor's everywhere in the world reporting infectious diseases, web crawlers sifting t

  • Apparently news reporting is still good for something. I never would have guessed.

  • I was looking forward to viewing the "erectile dysfunction" map based on viagra posts.
  • In Iraq (Score:1, Offtopic)

    by Venik (915777)
    Seems that the only outbreak in Iraq is rabies. Figures. Must be Al Qaeda.
  • wasted effort (Score:3, Interesting)

    by timmarhy (659436) on Sunday July 20, 2008 @02:14AM (#24260069)
    pathology labs already have a list of reportable diseases that the CDC monitor (you know, their job).

    why would anyone rely on reports from the media on what outbreaks are going around when you have trained professionals with lab equipment diagnosing these illnesses to begin with?

    • If your purpose is to scientifically track and deal with real diseases, reporting from the lab is useful.

      If your purpose is to dig up scare stories as headlines to sell newspapers/websites/etc, the which ever way digs up more scare stories (regardless of being true or not) is useful.

  • I've been playing Pandemic 2 [crazymonkeygames.com] all night and this is really freaking me out.

    I think I'm moving to Madagascar...

  • Another method... (Score:3, Informative)

    by longacre (1090157) * on Sunday July 20, 2008 @02:58AM (#24260219) Homepage
    The New York City Dept of Health monitors sales records of certain medications gathered from drugstore chains to detect disease outbreaks and biological attacks.
    • The New York City Dept of Health monitors sales records [...] to detect [...] biological attacks.And just how many of those have they detected? Seriously, if you have to look at sales records to identify a biological attack, how do you disguingish it from a regular disease outbreak? Secondly, if that's the only way to identify it, it's not really that effective - roughly as effective as the terrorist plot to kill all Americans by having them die in car crashes. So far they're getting about 130 people a day

  • by Bazman (4849)

    Another report of the same HealthMap thing was on /. not too long ago:

    http://science.slashdot.org/article.pl?sid=08/07/09/1424247 [slashdot.org]

    and what I said then still stands - the plural of "anecdote" is not "data". I defy anyone to come up with useful statistical models and tests on actual disease incidence based on web-crawling for disease names.

  • I believe we just saw this. A search on this site for the single word "disease" shows this link 3rd:

    http://science.slashdot.org/article.pl?sid=08/07/09/1424247 [slashdot.org]

    No google foo required to find this dupe.

    jeff

  • I think the day has come where hospitals need to maintain reliable Blogs considering they have the resources to do so. If it proves efficient, outsourcing to small IT service providers to maintain these blogs/websites will also be good for businesses.
  • by denzacar (181829) on Sunday July 20, 2008 @08:22AM (#24261581) Journal

    It is not that I don't like the idea but it is essentially flawed.

    A) It still requires human input. No one reporting the disease does not mean that it is not there.
    Looking at former Yugoslavia and seeing only 1 case of meningitis while here in Bosnia everyone knows about (and it is on TV, radio and in the papers) the brucellosis [wikipedia.org] epidemic that has been going on for months or even years maybe.

    B) That input must be made over the internet.
    Look at Africa. It is practically squeaky clean. There is one case diarrhea in the entire Botswana. And everyone is completely healthy up in the North.
    Could it possibly be due to the lack of internet-based inputs instead of due to the lack of diseases?
    Check out UK or the East Coast of USA. They are crawling with diseases.

    C) It should preferably be in English. Can the crawler read any of these articles:

    http://www.zzjzfbih.ba/content/view/66/13/ [zzjzfbih.ba]
    http://www.dw-world.de/dw/article/0,,3259389,00.html?maca=bos-rss-bos-all-1475-rdf [dw-world.de]
    http://www.slobodnadalmacija.hr/BiH/tabid/68/articleType/ArticleView/articleId/14733/Default.aspx [slobodnadalmacija.hr]
    http://www.dnevniavaz.ba/dogadjaji/panorame/bruceloza-prepolovila-prodaju-livanjskog-sira- [dnevniavaz.ba]
    http://www.blic.co.yu/repsrpska.php?id=44508 [blic.co.yu]

    Basically, what they come to is that there is a SHITLOAD of cases of brucellosis among the various cattle in Bosnia.
    And that it is going to stay that way for a long time, cause nobody is really doing anything about it.

    It is a fine idea, but unless you have every square kilometer of Earth covered with internet access and people who will report it in a language that the crawler understands - it is beyond useless.
    Even dangerous.
    Zoom out over Asia and turn on the Google in Chinese under Feeds. China's disease count jumps from around 40 to around 140.

    No. You can't fix all the problems by "putting it on the internet".

    • Flawed doesn't mean it's a bad idea or shouldn't be done; much the opposite in fact.

      A) It still requires human input. No one reporting the disease does not mean that it is not there.

      If no one is reporting the disease, it doesn't matter whether you don't report it online or by ACME 15-second express to their doorstop - it's not being reported.

      B) That input must be made over the internet.

      Duh. Some places will clearly benefit more than others... so what? If I have the ability to increase the literacy rate in five nations, I'll do it even if I can't do it in all the rest yet.

      C) It should preferably be in English.

      Unavoidable. Except it's not too hard to write in an ability to pick up on

      • by denzacar (181829)

        This isn't meant to solve all the problems, but I'm sorry - if implementing this concept means only one life is saved over the entire planet, it is entirely worthwhile. And if it increases reaction time to any single outbreak, it'll be irreplaceable.

        No - it isn't.
        Did you bother to read what I said at all?
        Or look at the map they are presenting?

        Example I made of China should point you out just how many cases are unseen if you miss a single language.
        And Google still speaks only a handful of them - badly.

        While the lack of internet infrastructure has just made Africa free of disease. WO-FUKIN-HOO!
        US is choking with disease and epidemics but Africa is clean.

        Disease control costs money and resources. Both human and material.
        A tool like this is nothing more th

        • While the lack of internet infrastructure has just made Africa free of disease.

          Not in the least! I don't think anybody is suggesting closing up shop and making this the be all and end all of epidemiological tools. All it is is just another way of discovering or verifying an outbreak. You don't have to choose between tits and cholera; you're lucky enough to be able to get your tits as an extra bonus to halting the spread of cholera.

          • by denzacar (181829)

            Right now it is far less useful than for example googling for news about diseases.

            For fuck's sake... look at the map. Half the diseases it lists are what could otherwise be described as "tummy aches". And most of them across US.
            It does not discover or verify anything. It is only generating noise at one end and ignoring everything at the other.
            It is a blind man holding an elephant's dick.

  • From the FA: HealthMap isn't just for doctors, specialists and public health officials, however. If travelers are heading to Paraguay they can see if there is an instance of Yellow Fever, for instance, and get vaccinated before they leave.

    Uh ... they better distinguish between the human-to-human diseases and the ones that are spread to humans from elsewhere. Yellow Fever, for instance, is always present in the jungle wildlife and only occasionally spreads to humans. I'd hate to have some unsuspecting tour

  • Did anybody else read the title of this story and wonder if Spider-Man was dabbling in epidemiology?
  • Here's one to mull over...

    Let's say for a moment that you come home after a long day and start to notice you have a bit of a cough, a stomach ache or even joint/muscle pain. You then hop onto your computer and post an innocent message to either a web forum you visit regularly or to a personal blog that you aren't feeling well with a brief description of what's bothering you.

    Now, let's say this is a common habit you have where you make such posts every few days, particularly with days you don't feel well.

    Wha

  • Time to put Bob on the case
    http://en.wikipedia.org/wiki/ReGenesis [wikipedia.org]

  • While it is nice to tout a system that "picks up" disease outbreaks "some of them before WHO or CDC", it is important to remember that:

    - this system does not DETECT outbreaks (nor does it claim to): it presents a map of already reported outbreaks

    - no disease outbreak EVER ever been initially detected by an automated system before an alert doctor or other healthcare provider: a human has ALWAYS been the "sensor" that detects disease outbreaks on the ground.

    - the WHO and the CDC are not SUPPOSED to iden

6 Curses = 1 Hexahex

Working...