Forgot your password?
typodupeerror
Math Medicine Science

Dead Salmon's "Brain Activity" Cautions fMRI Researchers 287

Posted by kdawson
from the as-we-may-think dept.
AthanasiusKircher sends in a Wired writeup on what should surely be a contender in the next Improbable Research competition: wiring a dead salmon into an fMRI machine and showing it pictures of humans designed to evoke various emotions. "When they got around to analyzing the voxel... data, the voxels representing the area where the salmon's tiny brain sat showed evidence of activity. In the fMRI scan, it looked like the dead salmon was actually thinking about the pictures it had been shown. ... The result is completely nuts — but that's actually exactly the point. [Neuroscientist Craig] Bennett... and his adviser, George Wolford, wrote up the work as a warning about the dangers of false positives in fMRI data. They wanted to call attention to ways the field could improve its statistical methods. ... Bennett notes: 'We could set our threshold [of significance] so high that we have no false positives, but we have no legitimate results.... We could also set it so low that we end up getting voxels in the fish's brain. It's the fine line that we walk.'" The research has been turned down by several publications, according to Wired, but a poster is available (PDF).
This discussion has been archived. No new comments can be posted.

Dead Salmon's "Brain Activity" Cautions fMRI Researchers

Comments Filter:
  • Re:spoooooky (Score:4, Informative)

    by Anpheus (908711) on Sunday September 20, 2009 @04:26PM (#29484763)

    It proves that an fMRI, like most machines, needs to be carefully operated and the mechanisms understood, as there are risks of false positives for results.

    The paper is about intentionally observing a dead creature, and coming across a few false positives and why that happened.

  • Re:Terri was alive (Score:5, Informative)

    by eldavojohn (898314) * <.moc.liamg. .ta. .nhojovadle.> on Sunday September 20, 2009 @04:29PM (#29484783) Journal

    God bless you, Terri Schiavo.

    Not sure if this was a troll but I'll bite.

    This is not about brain activity post-mortem! This is about the stupidity of some fMRI data. This is about the voodoo correlations that come out of fMRI data [slashdot.org] in popular research that is peer reviewed. They did this to prove a point, not claim dead fish think. Even if we did, I could use your logic to claim that every time we bury a dead person we are burying them with cognitive abilities -- obviously not true! I thought the summary covered that very well as the paper being titled "Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: An argument for multiple comparisons correction."

  • by syousef (465911) on Sunday September 20, 2009 @04:48PM (#29484915) Journal

    Looks to me like the dark matter syndrome: "Our theories wrong? Our calculations off by an insane amount? Unpossible! That can never be. Nature must be lying!"

    I find it amazing that people who haven't even bothered to study the data or the reason for hypotheses like dark matter feel the need to make ass backwards comments about people who've literally dedicated their lives to it. What do you actually know about dark matter and the current state of the evidence? Do you even understand it at a layman's level let alone understand the insanely complex math? Have you heard of the bullet cluster? Do you know about the rotation curve of galaxies? Do you understand anything about the cosmic microwave background and its fluctuations? Do you understand the background theories you're ridiculing? Do you know why General Relativity fits the data we have collected so well? Have you even bothered to find out why scientists believe in these things? Dark matter and dark energy aren't just theories that a bunch of arrogant pricks pulled out of their asses. These are our best attempts to fit multiple kinds of data into a single theory of nature. Your attempt to imply it's just scientists refusing to believe the data is at best childish. At worst you're no better than a flat earther.

  • by ceoyoyo (59147) on Sunday September 20, 2009 @04:54PM (#29484967)

    "That's precisely what this study is showing: even a dead fish has changing brain-blood oxygenation levels."

    No, they're showing that the noise inherent in the scan can be taken for signal if you aren't careful with your stats. The dead fish is NOT exhibiting varying blood oxygenation levels.

    Even the worst fMRI experiments that get published use a repetitive design, or equivalent. The simplest setup is to administer a stimulus or have the subject do something then stop, then do it again then stop, then do it again, etc. When you're done, you look for signals that vary in tandem with the stimulus.

    A dead fish's brain does NOT have blood oxygenation levels that vary in that way. For the purposes of the experiment they're basically constant. However, if you look at enough different measurements, the noise superimposed on that static signal will correlate with the stimulus.

    The fish is just for laughs. They could have easily done the same thing with a jar of agar.

  • by Kickasso (210195) on Sunday September 20, 2009 @05:11PM (#29485069)

    Atlantic salmon is called Salmo salar in biology-speak. It is the model species of the entire order Salmoniformes. Salmon doesn't get any truer than that. Pacific species belong to the genus Oncorhynchus. They are true salmons too. "Trouts" belong to both Oncorhynchus and Salmo (and another 5 genera). Some of these trouts have anadromous forms (that is, go to the seas and return to the rivers to spawn), for instance, the rainbow trout (called steelhead in its anadromous form) is Oncorhynchus mykiss and the brown trout (sea trout) is Salmo trutta.

  • by Brian Gordon (987471) on Sunday September 20, 2009 @05:53PM (#29485337)

    wrong [wikipedia.org]

  • Re:Discussion (Score:3, Informative)

    by yali (209015) on Sunday September 20, 2009 @05:56PM (#29485367)

    And why wasn't this published? The very conclusion is that we should be more careful when trusting fMRI results and conduct more testing before jumping to conclusion.

    Perhaps because what he's saying isn't new? As far as I can tell he's merely restating a substantive point that was recently made by someone else [wiley.com], which attracted substantial publicity [newsweek.com] as well as sober rebuttals [wiley.com] (along the lines of: nobody actually uses the flawed statistical methods that you're critiquing). All this guy is doing is illustrating the point in an absurd and attention-grabbing way.

  • by joepa (199570) on Sunday September 20, 2009 @06:19PM (#29485509)
    The poster highlights a very well-known problem in statistics that folks doing brain research are well aware of and almost always correct for. The issue is that, when you're doing a large number of statistical tests, like you are with brain imaging data, you're likely to get a lot of false positives. You can correct for this by using a very conservative significance threshold (i.e., "p-value"), directly controlling for the proportion of false positives using a statistic called the "false discovery rate," controlling for false positives via monte carlo simulation, etc. etc.

    Most neuroscientists who do brain imaging are very familiar with these correction methods, and apply them with great success. If anything, neuroscientists tend to be too concerned with false positives, such that they end up actually missing real activations because they're over-correcting.

    So it's actually really unfortunate that this study is getting so much popular media attention, because it's giving people the impression that researchers aren't aware of this problem and/or that that they aren't doing anything about it. That couldn't be further from the truth.
  • Any questions? (Score:5, Informative)

    by benntop (449447) <craigo AT gmail DOT com> on Sunday September 20, 2009 @07:46PM (#29486055) Homepage Journal

    Hey guys - I am the first author of the Salmon poster. If you have any questions that you would like us to answer then post it as a reply below and I will do my best to respond as soon as I can.

    You can find some more information on the poster at the following link:
    http://prefrontal.org/blog/2009/06/atlantic-salmon-index/ [prefrontal.org]

    Best ~ Craig Bennett

  • Re:Discussion (Score:2, Informative)

    by powrogers (558764) on Sunday September 20, 2009 @08:00PM (#29486121)
    I was there, I saw the poster, it was a humorous joke meant to remind fMRI newbies to control their type I error. It was in no way publishable research and was not intended to be. Most people who do fMRI research already make the effort to do the stats correctly. Multiple comparisons correction for fMRI is old news - the authors' most recent fMRI stats citation was from 15 years ago. And no there wasn't any activity or signal change or anything else in the brain of a dead salmon. It has nothing to do with the "double-dipping voodoo" of Vul and company's recent temper tantrum, which is a completely different statistical error.
  • by benntop (449447) <craigo AT gmail DOT com> on Sunday September 20, 2009 @08:06PM (#29486159) Homepage Journal

    Ma8thew is correct - the fish had been dead for some time. I purchased it on a Saturday morning, so it was likely from Friday's shipment of seafood.

  • joepa - You have a lot of very good points. Most neuroscientists are aware of the multiple comparisons problem and, at minimum, try to control for it using increased statistical thresholds (high p-values) and minimum clustering values (have to have several contiguous voxels). The trouble with this approach is that it is a soft control of the multiple comparisons problem. You still have no idea of what the false positive rate will be across the whole brain, only on a quasi voxel-to-voxel basis. Using techniques like false discovery rate (FDR) or Gaussian random field familywise error correction (FWER) you are able to have a much stronger case regarding what degree of your results are true or false.

    You are also correct that a majority of neuroscience results are corrected using FDR, FWE, or another correction method like permutation. The trouble is that a sizable fraction of articles still report values that are uncorrected. The Salmon paper is our argument that most, if not all, fMRI research needs strong multiple comparisons correction.

  • Re:Any questions? (Score:3, Informative)

    by benntop (449447) <craigo AT gmail DOT com> on Sunday September 20, 2009 @08:37PM (#29486329) Homepage Journal

    owlstead - I hear you - I have been a fellow /. reader for years and have observed firsthand the waxing and waning of articles. The above post was mostly a courtesy if anyone was genuinely curious about some aspect of the poster. That and I felt somewhat compelled to post a comment - as a longtime reader it is quite an honor to see some aspect of your own work on the Slashdot main page, even if it was for a dead fish.

    Thanks.

  • by IICV (652597) on Sunday September 20, 2009 @09:01PM (#29486423)

    So yes - why *does* it fit the data?

    The bullet cluster: http://en.wikipedia.org/wiki/Bullet_Cluster [wikipedia.org]

    A short, probably wrong explanation: two clusters of galaxies collided with each other. By analyzing the emissions of the resultant impact, we can see where all the baryonic (normal) matter went - baryonic matter smacking into other baryonic matter produces energetic particles like x-rays, which we can see. However, by examining the gravitational lensing caused by these two galaxies, we can determine where most of the mass went - and it's really far off from the center of the baryonic matter. Indeed, it looks like most of the mass of each galaxy did not interact electromagnetically with the mass of the other.

    The theory of dark matter explains this really well. Baryonic matter interacts electromagnetically with other baryonic matter, and so when the bullet cluster hit, its baryons slowed down (like a bullet flying through water). However, dark matter does not interact electromagnetically with baryonic matter, or very much at all with other dark matter, so the dark matter components of each galaxy just kinda ignored the impact and kept on going.

  • by DMUTPeregrine (612791) on Sunday September 20, 2009 @11:45PM (#29487447) Journal
    Actually, there is direct evidence of some dark matter. Quite a lot of gas and dust has been found that wasn't observable before the "dark matter" hypothesis came out. Some estimates I've seen put it as high as 20% of the "missing" mass. Dark matter is matter we haven't observed yet, and it MAY (probably) have properties that make it difficult to observe.
  • Re:spoooooky (Score:4, Informative)

    by DrLudicrous (607375) on Monday September 21, 2009 @07:46AM (#29489389) Homepage
    It detects the oxygenation of blood. The mechanism behind this is a different magnetic moment of oxygenated hemoglobin, oxygenated hemoglobin is diamagnetic vs paramagnetic while deoxygenated. This is called the BOLD effect (Blood Oxygen Level Dependent). The difference in the two conditions magnetic property affects the MRI signal lifetime in the near vicinity. This results in contrast developing between tissues with oxygenated blood vs tissue with deoxygenated blood. The idea behind fMRI is that when you use a certain part of the brain, it requires oxygenated blood, which will lead to contrast. Unfortunately, due to low overall signal strength/contrast-to-noise ratio, the image must be signal averaged. Hence if you were tapping your finger to see which part of your brain "lights up", you would have to repeat this action, and have your MRI scan be synced to your action so that the same part of the brian is being imaged over the same interval each time. It's tricky, but my understanding is that it's quite feasible. There are many other mechanisms for causing localized signal lifetime changes, without having RTFA, I can't be sure what they took under consideration.
  • ardeaem - At face value you are absolutely right. The majority of cognitive neuroscientists do use multiple comparisons correction in their research. Our commentary is targeted at the remainder of researchers who continue to use uncorrected statistics. The percentage is larger than you might believe, and my co-authors and I are of the opinion that we need to get our statistical house in order for the field to mature.

  • Re:Straw man (Score:3, Informative)

    by benntop (449447) <craigo AT gmail DOT com> on Monday September 21, 2009 @05:07PM (#29496585) Homepage Journal

    venicebeach - Again, good points. The trouble is that multiple comparisons correction is not the de dacto standard in any neuroimaging journal. Some journals, like NeuroImage and HBM, have become quite good about requiring correction in the results. Still, even they are not 100%. Other journals with a lower impact factor are quite a bit worse, with uncorrected statistics used in almost 50% of the studies. So, either people know about the problem and are willingly choosing to ignore it when they publish or they are unaware of the seriousness of the problem and need a salient reason to begin correcting. We believe it is the latter, which is why we published the Salmon.

    As for the argument about it being counter-productive, I fully agree. We presented the poster at the Organization for Human Brain Mapping meeting last June, which was our target audience. I then uploaded the poster to my website so those researchers could grab a copy. The poster got picked up by a few weblogs and eventually spiraled into what you see on Slashdot. We were quite content to publish the paper in a sleepy corner of neuroimaging and wanted it to remain as a discussion piece among scientists.

If A = B and B = C, then A = C, except where void or prohibited by law. -- Roy Santoro

Working...