How Scientists Know An Idea Is a Good One 140
Physicist Chris Lee explains one of the toughest judgment calls scientists have to make: figuring out if their crazy ideas are worth pursuing. He says:
"Research takes resources. I don't mean money—all right, I do mean money—but it also requires time and people and lab space and support. There is a human and physical infrastructure that I have to make use of. I may be part of a research organization, but I have no automatic right of access to any of this infrastructure. ... This also has implications for scale. A PhD student has the right to expect a project that generates a decent body of work within those four years. A project that is going to take eight years of construction work before it produces any scientific results cannot and should not be built by a PhD student. On the other hand, a project that dries up in two years is equally bad. ... the core idea also needs to be structured so, should certain experiments not work, they still build something that can lead to experiments which do work. Or, if the cool new instrument we want to build can't measure exactly what I intended, there are other things it can measure. One of those other things must be fairly certain of success. To put it bluntly: all paths must lead to results of some form."
For certain values of "good" (Score:5, Insightful)
That's not a description of a good idea. That's a description of an idea that fits into an arbitrary 4-year timescale that fits with a PhD program's average length.
Failures are very necessary part of science (Score:4, Insightful)
To quote Thomas A. Edison, "If I find 10,000 ways something won't work, I haven't failed. I am not discouraged, because every wrong attempt discarded is another step forward".
Read the literature... or not (Score:4, Insightful)
A big part of the problem is that there are few negative results in scientific literature. Ever found a paper with a clear negative outcome? I didn't. This "positive bias" in scientific publications is probably causing a major blow to the efficiency of scientific research.
Re:but ... (Score:4, Insightful)
The good ones need ink as well.
Re:Read the literature... or not (Score:4, Insightful)
There is a reason why you are wrong. There aren't enough forests to support publishing all possible negative results or enough time to read them. More aptly, there are plenty of "negative" results in the scientific literature. If you count the number of scientific papers that are in disagreement on a particular point, there are a great many of them. Science works best, when there is actually evidence gathered to accept or reject a particular scientific hypothesis. A purely negative result can be obtained without taking any data at all and hence, is of little value in advancing science.
Luck... (Score:4, Insightful)
...and the ability to think on your feet.
It is not possible to plan 4 years ahead to ensure success. What you get instead is a PhD project plan that's wrapped in a set of general concepts (AKA escape routes) in case you hit a dead end. I'm currently doing a life science PhD and have changed tack at the half way point. A number of my colleagues have also, often quite drastically, whether for reasons of practical feasibility or time constraints.
If we know accurately what we were going to work on that far in advance, it has probably already been done.
Re:Read the literature... or not (Score:4, Insightful)
Re:4 years.. (Score:5, Insightful)
Four years? Ha! That's a good one!
The easiest way to enforce that is for the awarding institution to say that if it isn't done in 4 years, it will be taken as a complete failure.
No, that rule would result in a lot of thesis committees approving completely crap theses. Believe it or not, thesis committee members are human and have a lot of difficulty telling kids that their last four (or five, or eight) years of work are worth no recognition and please leave. Thesis advisors become emotionally attached to their students and want to see the succeed/graduate, even if those students are incompetent. Sometimes you can compensate for the incompetence with time. Only rarely will a thesis committee 'over-rule' the advisor, with their input generally taking the form of 'this would become acceptable if the student adds [foo] over the next year or so.' Mandated time to completion is a recipe for diminishing the quality of theses and migrating a PhD from someone prepared for reasonably independent work to a glorified MS. Probably already moving in that direction, as many 'PhD's aren't really ready to work independently until they've finished two or more post-doctoral internships.
Hindsight (Score:5, Insightful)
Re:Failures are very necessary part of science (Score:5, Insightful)