Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
Science

The Band of Debunkers Busting Bad Scientists (wsj.com) 122

Stanford's president and a high-profile physicist are among those taken down by a growing wave of volunteers who expose faulty or fraudulent research papers. WSJ: An award-winning Harvard Business School professor and researcher spent years exploring the reasons people lie and cheat. A trio of behavioral scientists examining a handful of her academic papers concluded her own findings were drawn from falsified data. It was a routine takedown for the three scientists -- Joe Simmons, Leif Nelson and Uri Simonsohn -- who have gained academic renown for debunking published studies built on faulty or fraudulent data. They use tips, number crunching and gut instincts to uncover deception. Over the past decade, they have come to their own finding: Numbers don't lie but people do.

"Once you see the pattern across many different papers, it becomes like a one in quadrillion chance that there's some benign explanation," said Simmons, a professor at the Wharton School of the University of Pennsylvania and a member of the trio who report their work on a blog called Data Colada. Simmons and his two colleagues are among a growing number of scientists in various fields around the world who moonlight as data detectives, sifting through studies published in scholarly journals for evidence of fraud. At least 5,500 faulty papers were retracted in 2022, compared with 119 in 2002, according to Retraction Watch, a website that keeps a tally. The jump largely reflects the investigative work of the Data Colada scientists and many other academic volunteers, said Dr. Ivan Oransky, the site's co-founder. Their discoveries have led to embarrassing retractions, upended careers and retaliatory lawsuits.

Neuroscientist Marc Tessier-Lavigne stepped down last month as president of Stanford University, following years of criticism about data in his published studies. Posts on PubPeer, a website where scientists dissect published studies, triggered scrutiny by the Stanford Daily. A university investigation followed, and three studies he co-wrote were retracted. Stanford concluded that although Tessier-Lavigne didn't personally engage in research misconduct or know about misconduct by others, he "failed to decisively and forthrightly correct mistakes in the scientific record."

This discussion has been archived. No new comments can be posted.

The Band of Debunkers Busting Bad Scientists

Comments Filter:
  • Seems now that fame can find its blame on debunking famous claims. (sorry)
    • by Darinbob ( 1142669 ) on Wednesday September 27, 2023 @12:05PM (#63881077)

      Replicating results is cool again, because of the scene in Oppenheimer where they immediately reproduced results in a mere 15 seconds of film.

    • by sfcat ( 872532 )
      The problem is that there is a formula used by universities when hiring researchers and professors. That formula includes things like numbers of papers and number of citations but it does not include number of experiments reproduced. That's the problem, not some amount of "fame".
  • by Too Late for Cool ID ( 1794870 ) on Wednesday September 27, 2023 @11:42AM (#63880983)
    Why choose publish or perish when now you can do both!
  • by crmarvin42 ( 652893 ) on Wednesday September 27, 2023 @11:43AM (#63880987)
    I review for a journal in my field, and spotted the same figure appearing 2x, with two different Y-axis labels, in the same paper recently. Cannot be sure if it is a mistake or malicious, but this campaign to pay closer attention to this kind of fraud is probably why I noticed it.

    section editor was notified, and will be following up to sort it out. Hope I am just being paranoid, but with all the press around how often this happens, better to be safe than sorry.
    • Cool of you to notice and to start questioning. Big fan of Data Collada. They are way over my head, but it's brilliant that they do this work gratis.

  • If you have the access to read the article and write the summary can you at least link to the supporting data/info. I give zero shits as to what some WSJ has to say about it, but I would like to see the underlying reports and info.
  • https://www.latimes.com/opinion/la-xpm-2012-jul-13-la-ol-blowback-pscyhology-science-20120713-story.html
    • by wiggles ( 30088 )

      Soft sciences are notoriously coopted by people to push political agendas. The Nazis, for example published a lot of eugenics garbage.

  • This is hard, intensive, difficult work.

    We live in a post truth era.
    We can't trust the documents we find online, as they are likely ginned up by someone with an agenda somewhere. If they're not outright lies, they're lies of misinformation and framing.
    AI generates music, conversations (the arguments between Trump, Biden, and Obama playing Age of Empires 2 are hilarious), and whole news articles.
    We can't trust even the images we see, as AI image generators are able now to convincingly portray just about any

    • by sfcat ( 872532 )

      We live in a post truth era.

      No we don't that's the slogan of those that don't wish their propaganda to be bound by truth (even though effective propaganda often is) nor do they want an educated population who can see through their BS. We can detect AI and to date there really hasn't been any big AI generated scandal. I have news for you, reality is like gravity. Nobody escapes it forever, only for a time. And most of the time reality comes back and wins. Just ask these professors that have just lost their jobs. If you do, you wi

  • In many forums, including scientific journals, poorly supported claims in one direction are ignored. Poorly supported claims in the opposite direction are called out with rigorous evidence demanded. This was harder to do when immediate physical evidence was presentable, like "if you add a material made this way to a material made this way, the following reaction occurs'.

    These days the bias surrounds politicized topics and there are definitely agendas being pushed. There needs to be unbiased review of scie
    • All social "science" research is pushing an agenda, and that includes cognitive psychology, where it has already been shown that most results cannot be reproduced. There is nothing new about researchers in these fields pushing agendas; what is new is that these agendas are motivated by politics.

  • Would hate to see them find the reports of goat paste "curing" covid were false because that would set up an infinite loop for the deniers.

    "These guys are great because they show all the fake science out there. This is why we can't trust scientists."

    "These guys are frauds because they debunked every single study showing goat paste "cures" covid. This is why we can't trust scientists.:

  • Some long jail sentences would probably slow down the volume of fraud.

  • A peer reviewed publication doesn't mean it's absolute fact. It means it's passed two hurdles - the author's best-effort attempt to validate the data, and the referee's cursory vetting of their publication. It's not until there is a significant body of literature supporting the claimed facts that the science is considered 'solid'. If it takes a decade to determine the science is bunk then that's part of the process. Sometimes it takes a few weeks (like LK-99), and sometimes it takes a lot longer (Just found
  • https://datacolada.org/ [datacolada.org] Their work is intriguing. However, one of their tools - looking at the distribution of the last digit in a set of data - is obviously subject to fraudsters getting better at creating fake data. I am sure that it applies to other approaches as well. That said, though, the techniques the fraudsters use seem remarkably crude, for example simply duplicating favorable data and salting it into the data set.
    • by sfcat ( 872532 )
      I hope you are confused about that, the way you use the statistic technique you reference is to compare the distribution of the first digit of the data, not the last. That distribution should be log-normal. I'm not going to explain any more because its how the IRS catches tax cheats so it isn't discussed outside of specific fields.
  • These people are literally doing the Lord's Work, fighting against falsehood and lies. I really think that -- and I'm an atheist!

    People that say "Believe the Science" are trying to replace science as a religion, you only trust it when you have no other choice. (But dressing up in funny costumes, waving your hands, and declaring "Deus Vult" is placeboism -- and not even trying.)
  • by davide marney ( 231845 ) on Wednesday September 27, 2023 @03:19PM (#63881563) Journal

    We have TWO problems on our hands.

    Problem #1 is using "FOLLOW THE SCIENCE!!" as a cudgel to shout down disagreement. Some very highly placed politicians (I'm thinking specifically of Dr. Fauci) have done exactly this, to their shame. It's easy to ignore obvious partisans who swing this club, but not 50-year veteran bureaucrats in charge of multi-billion dollar budgets.

    Problem #2 is arguably much worse: the exploding number of experiments that cannot be reproduced. Thousands of papers retracted. Reputations ruined. Entire dependency chains of studies now suddenly shown to be based on unsubstantiated claims. This is the problem being addressed by the trio of researches being discussed here.

       

  • by Miles_O'Toole ( 5152533 ) on Wednesday September 27, 2023 @05:33PM (#63881905)

    A University of Massachusetts student caught a couple of Harvard "trickle down" economists putting their thumb on the scales to favour their corporate-friendly conclusions. They were allowed to pass off what appears to be bad faith cherry picking as an oversight, though it takes a real stretch to believe they weren't doing it on purpose.

    https://www.bbc.com/news/magazine-22223190 [bbc.com]

  • Plenty of fertile ground here! However doing so AS WILL KNOW, will result in them be de-Platformed and de-Financed and otherwise completely disenfranchised. Maybe this debunking will happen in my lifetime, but not holding my breath.

In any formula, constants (especially those obtained from handbooks) are to be treated as variables.

Working...