The Neuroscience of Screwing Up

resistant writes "As the evocative title from Wired magazine implies, Kevin Dunbar of the University of Toronto has taken an in-depth and fascinating look at scientific error, the scientists who cope with it, and sometimes transcend it to find new lines of inquiry. From the article: 'Dunbar came away from his in vivo studies with an unsettling insight: Science is a deeply frustrating pursuit. Although the researchers were mostly using established techniques, more than 50 percent of their data was unexpected. (In some labs, the figure exceeded 75 percent.) "The scientists had these elaborate theories about what was supposed to happen," Dunbar says. "But the results kept contradicting their theories. It wasn't uncommon for someone to spend a month on a project and then just discard all their data because the data didn't make sense."'"

Sometimes screwing up leads to success ...

[xmas2003]

The WIRED piece threads what is written in the summary around the story of how Arno Penzias and Robert Wilson at Bell Labs discovered Cosmic Radiation [wikipedia.org] after being puzzled for a year about background noise on their radio telescopes ... even scraping pigeon poop off their gear as a possible source until they realized the signal was real - Homer Simpson would have said D'OH! ;-) [komar.org]

Re:Ridiculous

[Shadow of Eternity]

Not always, sometimes your data doesn't make sense because you made a mistake somewhere that wound up turning your results into garbage.

Re:Or you can edit your data....

[dwguenther]

Yeah, it's just you. AP News found no evidence of massaged or ignored data (http://news.yahoo.com/s/ap/20091212/ap_on_sc/climate_e_mails). So climate science is a poor example of this thesis.

Re:Ridiculous

[RobertF]

Anyone who has spent time working in a lab knows that not all data is equal. You can get useless results if something isn't quite as clean as necessary, or perhaps you were in a bit of a rush and didn't connect everything perfectly. Any interesting experiment usually has numerous points at which humans can mess things up. Errant data is usually a sign that you have improperly set up the experiment, so you'll spend most of your times reviewing and fixing procedures until you get what you expected.

Re:Or you can edit your data....

[J Story]

And there's no real evidence of the proper scientists massaging or ignoring anything. Just because a detailed, written account of everything doesn't exist in stolen, incomplete private documents doesn't mean it doesn't exist at all.

The behaviour surrounding the data is certainly indicative of a lack of confidence in the findings. Refusing FOI requests and claiming that "the dog ate it" do not show a group filled with the belief that their research is unassailable.

Re:Why most scientists and engineers screw up

[NeutronCowboy]

But still, I do believe that genetic differences affect what we are and that genetic differences can be attributed to where our genes came from.

The theory that race has nothing to do with intelligence has nothing to do with political correctness, and all with science: specifically, the scientific discovery that the taxonomy of human races is not definitive, not specific and has no basis in genetics. Which in turn means that the ggp's assertion that race was a statistically significant factor in their research means that their research was utter crap to begin with.

So let me ask you this then: what makes you think that race is the same as genetics, or that you can even reliably a race? I mean, outside of some outdated and non-scientific notions of physiognomy and phrenology?

Re:Ridiculous

[Arancaytar]

It is possible to end up with crap data because the premise of your experiment is wrong. You can ignore a variable that should have been controlled or kept equal, or you can measure the wrong variable.

You can also end up with data that neither confirms nor denies your hypothesis, because it allows no statistically significant conclusion.

Re:Why most scientists and engineers screw up

[Toonol]

The idea that race is a fiction is a bad, well, fiction, and a clear example of the distortion of thought due to political correctness.

There are a number of human traits (and the genes which cause them) that statistically cluster into groups that correspond to what we consider race. You can test a person's DNA and determine their racial heritage, to a fairly accurate degree. Obviously race is real, if you can nearly automate measuring it. The fact that statistical clusters don't have firm boundaries doesn't mean those clusters don't exist.

Is race relevant? Not for most purposes, but it is for some. I understand that Asians are more likely to have difficulty digesting milk, for example; blacks have a higher tendency to have sickle-cell anemia. Declaring that any test that shows a tendency for races to vary based on genetics is CERTAIN to be flawed because you don't believe race exists is ludicrous.

Two Relevant Quotes

[bill_mcgonigle]

"It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong."
  - Richard Feynman

"The most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! but rather, "hmm.... that's funny...."
  - Isaac Asimov

Re:Ridiculous

[dcollins]

This is well known and called the "file drawer effect" (or publication bias).

http://en.wikipedia.org/wiki/Publication_bias [wikipedia.org]

"In September 2004, editors of several prominent medical journals (including the New England Journal of Medicine, The Lancet, Annals of Internal Medicine, and JAMA) announced that they would no longer publish results of drug research sponsored by pharmaceutical companies unless that research was registered in a public database from the start.[11] In this way, negative results should no longer be able to disappear."

Re:Why most scientists and engineers screw up

[VoidEngineer]

Here's the problem. If you can't order every single human into one race or another, your model is flawed. If you're forced to resort to mixes of races, well, then you don't have any distinct race left.

Race concepts fall apart once actual taxonomic principles are applied to them.

Sort of. In a traditional hierarchical phylogenetic taxonomy, yes, race concepts fall apart. But if they don't necessarily fall apart with a cladistic genetic taxonomy.

Defining race is a classic problem of, well, classification. Put another way, it's like organizing books. Where do you place 'War and Peace'? In the fiction section? In the history section? In the classics section? In the russian literature section? It could legitimately be placed in any of those sections. The problem is that the book has a single physical instance. The book only exists in one place at one time. So, it can only be placed in one category at a time. And this is the problem with any phylogenetic based hierarchical taxonomy. It's not unique to race; it also applies to species, books, weblinks, and any other number of objects. It's why, before search engines, we had all these portal sites, like Yahoo!, who were focused on creating giant taxonomies of weblinks. And it was always a pain, because we had this intuition that a weblink should only exist in a single category at a time. This was a hold-over from library systems, where any particular book can only be placed on a single shelf at a time.

But then we discovered tagging. With tagging, a new type of taxonomy is possible, where a single entity can be placed in multiple categories at a time. And it turns out that tagging is equivalent to a genetic taxonomy. Each tag is equivalent to a gene (or meme, to be more precise). And we now give webpages lists of keywords, which function like a genome of sorts.

So, you're correct that race concepts fall apart at a hierarchical, phylogenetic based taxonomy. But with a genetic based taxonomy, race is 'tagged' by combination of genes... melanin count, lactose sensitivity, sickle-cell anemia, etc.

And what's more, this tagging and clustering, is a precursor to speciation. Consider the following simplified hypothetical example: a) mutant gene (A) interacts with the gene for lactose sensitivity such that, together, they cause a change in sperm mobility due to a lack of calcium, and b) another mutant gene (B) interacts with the gene for sickle-cell anemia such that, together, they cause a change in permeability to an egg due to lack of iron. If these two things were to hypothetically occur, it would make for a situation where sperm and egg couldn't unite, and a lactose intolerant father and sickle-cell anemic mother couldn't have children. Now then, one more consideration: say that these two mutant genes were actually very advantageous. Mutant gene A protects against flu and pnemonia, and mutant gene B codes for sexy pheremones. If these mutant genes are advantageous, then they'll spread throughout the population. But as the mutant genes spread through the population, the carriers of those genes, who also carry the genese for lactose intolerance and/or sickle cell anemia, would lose the ability to breed together. And this would be defined as a speciation event. Not only would those people be of different races, they would be unable to breed together, and would be different species.

Anyhow, it's worse than people fear. Not only does race actually exist, it's a precursor to speciation. Race just doesn't fit neatly into hierarchical phylogenetic taxonomies. Genetic taxonomies allow for overlapping, fuzzy boundaries. And that's exactly what Race is. Race doesn't fit into neat little hierarchical tree structures; rather, it's a fuzzy network of genes.

Re:Why most scientists and engineers screw up

[Just Some Guy]

Blacks do not have a higher tendency for sickle-cell anemia, a certain group of people in Africa do. Blacks in the US do not have that trait.

ORLY? The US Government [ornl.gov] says:

In the Unites States, it affects around 72,000 people, most of whose ancestors come from Africa. The disease occurs in about 1 in every 500 African-American births and 1 in every 1000 to 1400 Hispanic-American births. About 2 million Americans, or 1 in 12 African Americans, carry the sickle cell trait.

...making your closing amusingly ironic:

How much does it suck to be so wrong? Your cognitive dissonance must be at a record high.

Re:Ridiculous

[Simetrical]

Our educational system is totally broken when the educated just want things to fit. Even in mathematics, we're promoting a crop of "just tell me what to do!"

As a grad student in pure mathematics, I'm curious: do you mean in low-level math education, or mathematical research? Basic math education is often just about giving you the tools you need to do your job, so there's nothing wrong with just telling people what to do. Higher-level courses (meaning the kinds only pure-math majors typically take) do require you to actually understand the material and be able to prove things from first principles, probably far more so than any other field.