Gut Bacteria-Autism Link May Just Be Misinterpreted Data From a Confusing GUI (medium.com) 83
Remember that mouse study which concluded gut bacteria may contribute to autism symptoms?
Jon Brock, a cognitive scientist with 18 years research experience on neurodevelopmental conditions, including autism, has posted a Medium post summarizing new critiques of the research emerging online. (For example, from Professor Thomas Lumley, a statistical researcher who has concluded that the study's analysis "is wrong," and "arguably due in part to a poor GUI design.") Soon after publication, scientists began expressing concerns about the paper on social media. These were echoed in a blogpost by drug discovery chemist Derek Lowe and then in a series of comments on the PubPeer website. Looking more closely at the data, the results are a whole lot less compelling than the media coverage, the press releases, and even the paper itself suggest...
The differences between mice with autistic and non-autistic donors are subtle if they exist at all. And there are reasons to be skeptical about even these small effects. Mice are not tiny humans with tails. Autism is defined in terms of human behaviour. And so the claim that mice showed "autism-like" behaviour relies on an assumption that the mouse behaviours under investigation are in some sense equivalent to the behaviours that define autism in humans...
But even if we accept the premise that mouse behaviours are directly analogous to behaviours exhibited by autistic humans, the evidence is both weak and inconsistent. It's fair to say, I think, that the authors have presented the data in its most flattering light... Since posting this critique last week, further developments have cast more doubt on the conclusions of this study. The authors responded to criticisms on PubPeer. In doing so, they released the code for their analyses, which appear to show important discrepancies between how the analyses were described in the paper and how they were actually conducted.... Lumley suspects that the culprit is the confusing interface of the SPSS software the authors used for their analyses. There's no reason to see this as anything other than an honest mistake. But, as Lumley notes in his post, the episode shows the importance of researchers sharing their analysis code as well as their data.
Jon Brock, a cognitive scientist with 18 years research experience on neurodevelopmental conditions, including autism, has posted a Medium post summarizing new critiques of the research emerging online. (For example, from Professor Thomas Lumley, a statistical researcher who has concluded that the study's analysis "is wrong," and "arguably due in part to a poor GUI design.") Soon after publication, scientists began expressing concerns about the paper on social media. These were echoed in a blogpost by drug discovery chemist Derek Lowe and then in a series of comments on the PubPeer website. Looking more closely at the data, the results are a whole lot less compelling than the media coverage, the press releases, and even the paper itself suggest...
The differences between mice with autistic and non-autistic donors are subtle if they exist at all. And there are reasons to be skeptical about even these small effects. Mice are not tiny humans with tails. Autism is defined in terms of human behaviour. And so the claim that mice showed "autism-like" behaviour relies on an assumption that the mouse behaviours under investigation are in some sense equivalent to the behaviours that define autism in humans...
But even if we accept the premise that mouse behaviours are directly analogous to behaviours exhibited by autistic humans, the evidence is both weak and inconsistent. It's fair to say, I think, that the authors have presented the data in its most flattering light... Since posting this critique last week, further developments have cast more doubt on the conclusions of this study. The authors responded to criticisms on PubPeer. In doing so, they released the code for their analyses, which appear to show important discrepancies between how the analyses were described in the paper and how they were actually conducted.... Lumley suspects that the culprit is the confusing interface of the SPSS software the authors used for their analyses. There's no reason to see this as anything other than an honest mistake. But, as Lumley notes in his post, the episode shows the importance of researchers sharing their analysis code as well as their data.
Re: (Score:1)
What you describe sounds like psychopaths and narcissists, not at all like autistics.
Re: (Score:3)
Re: (Score:2)
So fecal transplants won't cure autism?
Jenny McCarthy? Is that you?
Re: (Score:2)
Given it's at least partially hereditary the early diagnosis can prevent autistic offspring long before abortion becomes an option.
I wonder if it isn't too late already (Score:2, Funny)
and helicopter parents are not as I type this, shopping Craigslist postings for premium poop they plan to jam up the butts of their precious offspring.
If you're looking for a good pyramid scheme, it may also be too late to invest... /s
Re: (Score:2)
the most brilliant article on slashdot in the last decade had to be the guy that started the twitter channel justsaysinmice ... all those 'correlation' that damn near everyone takes for gospel is based on a single unverified study on _mice_
Re: I wonder if it isn't too late already (Score:2)
Something as complex and seemingly human-specific as autism would not appear to be one of them. And here, we saw the peer-review process kick in almost immediately, within days. Exactly the way science should work
Re: (Score:2)
I’m far from anti-science. I’m just getting sick of these headlines that contradict themselves. I’m not talking specifically about animal testing of medication. I’m talking about the fact that they use statements like “people who eat worms might help you live longer”. Then they go cite a study where out of two uncontrolled groups, the group that ate worms, on average, lived 43 seconds longer than the group that did not. I made up the worm eating part for illustration.
What is with user interface designers? (Score:5, Insightful)
It's incredible that, after people worked out how to make user interfaces decades ago, there are still terrible user interfaces. And I don't mean just new styles, I mean user interfaces that are inefficient, inconsistent, needlessly complex, and error-prone.
Re: (Score:3)
What we need are more user interface designers in the same way we need more people coding to become programmers. After all, look at how flawless all the code out there is.
Re: (Score:2)
They have significantly decreased in usability in the last 5 years.
Why label an icon or use colour or shading, when you can make EVERYTHING BRIGHT WHITE with unlabelled black and white icons!!!
Sigh
Re: (Score:2)
It's incredible that, after people worked out how to make user interfaces decades ago, there are still terrible user interfaces. And I don't mean just new styles, I mean user interfaces that are inefficient, inconsistent, needlessly complex, and error-prone.
That's what happens after decades of user interfaces being dragged through decades of trying to maintain some level of backward compatibility.
Without that backward compatibility the market for what's new is limited. If the backward compatibility is non-linear, and starts to look like a tree or web, then UI constructs can be duplicated or conflicting.
Re: (Score:2)
It's the opposite of backwards compatibility. It's trying to be creative and original with respect to something that was already more or less optimal.
Re: What is with user interface designers? (Score:2)
Re: What is with user interface designers? (Score:3)
You should see the software that academics cobble together. Itâ(TM)s often so confusing that only the original research team can even tell what itâ(TM)s supposed to do. Thatâ(TM)s what happens when your code is written by a biology grad student who happened to have computer science as half of his double major in undergrad.
Re: (Score:2)
Re: (Score:2)
Actually GUIs are not much of a problem if you have people doing research with them that are actually smart and understand what they are doping. These bad GUIs just make it more obvious that many "scientists" have no business doing science because they have no clue what they are doing. You can conclude the same from doing paper reviews (which I regularly do) and this is not a new thing. Probably because people that are boring and not inventive and hence will not "rock the boat" have a much easier time findi
Re: (Score:2)
I think they all start out wanting to build good interfaces; but most of them give up after they learn they can’t actually track the killers by making a GUI with Visual Basic.
Re: (Score:2)
"It's incredible that, after people worked out how to write novels centuries ago, there are still terrible novels."
I don't know, maybe it's because it's nowhere nearly a mechanical process, and it requires a lot of skill and talent to find out what the final users will find good and enjoy?
Re: (Score:2)
"It's incredible that, after people worked out how to write novels centuries ago, there are still terrible novels."
I suppose there is some software where the objective is for the user to "enjoy" the experience, but most is meant to be practical and efficient. A database front-end that is effectively data-entry plus validation rules does not need, and is compromised by, a user interface that expresses its designer's creativity.
Most software is a tool for getting something done. Things like hammers and cutting tools haven't had their user interfaces change since the Stone Age.
Re: (Score:2)
"A database front-end that is effectively data-entry plus validation rules" is a terrible user interface for any application other than an input form for data dumps, which was my point. It is well known that attractive products are easier to use [nngroup.com] than functionally identical ugly ones, even if that blows your developer's rational mind, BTW.
Is your assertion that "after people worked out how to make user interfaces decades" meant to imply that we know how to create CRUD, and therefore all such interfaces can b
Re: (Score:2)
I don't see how any of that relates to user interfaces that are "inefficient, inconsistent, needlessly complex, and error-prone".
I am referring to things like:
Words such as 'accept', 'continue', and 'save' being used inconsistently so the user has to memorize the behaviour for every different window and dialogue box.
Dialogues asking yes/no questions and not offering 'yes' and 'no' as options.
Error messages describing two courses of action and then offering 'confirm' and 'cancel' as options without clearly s
Re: (Score:2)
Ah, you're familiar with Windows 10 I see.
Need medicine to go beyond correlation (Score:5, Insightful)
Medicine seems to concentrate entirely on looking for any correlation and then immediately publishes that correlation even if there is no hypothesis as to why that correlation exists, let alone an experimentally tested hypothesis. This is such a low bar that a high rate of false results is very likely even if the data are collected and analysed properly.
I get that the systems medicine studies are vastly more complex and harder to understand in detail than those we deal with in physics (Higgs bosons don't decay differently based on their individual lifestyle choices!) but given the number of garbage studies like this which keep getting published and then debunked isn't it time the medical field started to raise the standards required for publication before its credibility gets even more damaged?
We already have dangerous movements like the anti-vaxxers who deny well established medical facts. The publicity around bad results like these just encourages more people to doubt what their doctor is telling them.
Re: Need medicine to go beyond correlation (Score:4, Insightful)
Re: (Score:2)
If true, it's sad to see such short-sighted thinking. Conclusively demonstrating a given hypothesis is false also increases human knowledge. It's not as exciting as a positive outcome, and certainly won't generate headlines, but when searching for new knowledge, I'd presume one typically has to definitively rule out a number of hypotheses before you hit upon the correct one. If not, the researchers were either extremely lucky, or the hypotheses was rather obvious, and as such, of limited value. Publishi
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
There are actually very few people that are against vaccines
There are enough that there are now regular outbreaks of diseases like measles that we did not see a decade or so ago. That's the problem with vaccines: it does not take many idiots to compromise the herd immunity.
Re: (Score:2)
There are enough that there are now regular outbreaks of diseases like measles that we did not see a decade or so ago. That's the problem with vaccines: it does not take many idiots to compromise the herd immunity.
Add this to poorly controlled borders, where checkpoints are bypassed regularly, and that's just asking for all kinds of trouble. People, pets, plants, or food products can carry all kinds of diseases.
Border patrol should stop checking phones and laptops for dirty pictures and instead direct that energy to do something real quick and simple about diseases. A $50 no contact thermometer from WalMart for every border patrol agent will save a lot of lives. That's just a theory I have of course. How hard cou
Re: (Score:2)
Add this to poorly controlled borders,
What year is it? Shouldn't it be affordable to immunize the world against measles by now?
Re: (Score:2)
Last I checked, UNICEF was able to buy the MMR vaccine for about 9 per shot. They can vaccinate 11 people for $1.
Yaz
Re: (Score:2)
So someone with a common cold will be refused entry?
Seriously?
Re: (Score:2)
Why won't you simply close all international airports and be done with it?
Re: (Score:2)
Yes, seriously. No foreigner is obligated entry to the USA.
True but banning us because we have a cold is going to inflict a serious amount of damage on your economy and international reputation. The ridiculous clamp downs after the 11th September attacks caused all major scientific conferences, at least in my field, to move from the US and this trend has only recently largely reversed.
Of course, you are entirely within your rights to ban foreigners with colds from entering the US but, if you do you can forget the US as a centre for international scientific rese
Re: (Score:2)
The vaccines themselves help cause the problem. When everybody got Measles, (and just remember, very, very few peopled died or had severe effects, that is a story told to scare you) mothers would pass on immunity to their children while breastfeeding. So children getting Measles was actually a very rare event. Now that everyone is vaccinated with somethings that does not last a lifetime, like the real thing does, and is not as effective, it gets passed on the children much more easily since the passed-along
Re: (Score:2)
Medicine in general seems reluctant to admit that they're really not good at being scientists. Very, very smart researchers with lots of data, with lots of good science behind them, but when they try to innovate they don't seem to get the scientific method as well as they think they do.
Re: (Score:2)
As my stats professor explained, the medical people require roughly a 5% degree of confidence in any given statistical result. The PhD students require a positive result to graduate. Thus, if you want to graduate, run a study with at least 20 different questions/tests, and you will get your positive result.
The details are a little more complex than that. In practice, you probably want more than 20 questions, and the statistical methods are usually good enough to detect completely random nonsense. Howev
Re: (Score:2)
Physics traditionally requires 5 sigma (which is a silly metric because it only works if you assume a Gaussian distribution) to accept a particle *discovery*. Most of those discoveries have thousands, or tens of thousands of papers associated with them, each of which has a much lower bar.
If you want to draw that comparison, what medicine lacks is a meta-discovery standard. Something like a rigorous, quantitative meta-analysis with reasonable N from multiple labs and p 10^-6. That's a good idea.
Re: (Score:2)
It's important to publish observations promptly, so other people can look at it. Science should be aiming to encourage relatively rapid publishing and understanding among readers that just because something is published doesn't make it correct. Journalists especially should have enough professional ethics to resist the urge to take a study in mice based on an N of six, and splash it all over the web without at least some very strong caveats.
Physics is the go-to example of hard science, and they tend to do i
Re: (Score:2)
Autism probably does not have one cause (Score:1)
Especially with the broadening it has undergone over the years to mild symptoms that in some way suggest milder forms of the full fledged symptoms. It's a symptom just like other symptoms such as coughing or insomnia. A treatment that helps in some scenarios but not all is not valueless, but of course it would be helpful to find a method for determining if a technique would work for a patient without trying it first.
If my bacteria is reading this right ... (Score:2)
Gut Bacteria-Autism Link May Just Be Misinterpreted Data From a Confusing GUI
And that's why people study GUI design, rather than just going with their gut - bacteria.
The politicians are right (Score:3)
Science is a process and taking the steps along the way as the answer is premature. Newtonian science worked well enough to put a man on the moon but it was a working hypothesis. Similarly medicine is working well enough to vaccinate populations against harmful pathogens but the tiny number of adverse effects show that human biology is not completely understood. The anti vaxxers think they have the answer. They do not have the answer any more than scientific medicine does, but they are currently completely wrong in drawing the conclusion that vaccination causes autism because there is no evidence for that.
Research like this seeks to address concerns like this and needs to be done. It also needs to reach completion before you can draw any conclusions from it. As this latest interpretation shows the answer is still out of reach.
The point is that some fields of science have got a lot further along the road towards the answer - like climate change science. Human medicine has only recently discovered the micobiome and the G verses E dynamic. (Genomics verses Environment). One small study on mice does not equal the paradigm shift of shifting the earth from the center of the universe to the sun becoming the center. This also worked well until we understood that there is no identifiable center of the universe.
You need to work with what is currently known until you know better for sure.
But man made global warming link is 100% solid (Score:2)
But man made global warming link is still 100% solid, amirite? The "science" is settled?
Re: (Score:2, Informative)
Yes - because there are lot of independent scientists and lots of independent papers supporting it.
The bacteria-autism thing has a single paper supporting it. Nobody should really trust a single paper, it's when other scientists reproduce it and also publish their own papers that science as a whole trusts it.
Re: (Score:2)
But man made global warming link is still 100% solid, amirite?
97% solid. Get your numbers right, man.
GUIs are bad for data analysis (Score:1)
This kind of thing happens all the time in statistical analysis, and I've done this myself before I made it a policy to do all my data analysis with a script-based approach. Anything that is point and click is not reproducible and very prone to mistakes. Since most papers don't include the data and the analysis steps, it's impossible to know how people got their results. This is what I consider the dirty secret of science: it's based on a ton of trust that the people who did the analysis knew what they were