Slashdot Log In
Current Scientific Publishing Methods Problematic
Posted by
ScuttleMonkey
on Mon Oct 13, 2008 12:12 PM
from the accurate-problem-bad-analysis dept.
from the accurate-problem-bad-analysis dept.
A recent examination of current scientific publishing methods shows that they are problematic at best, treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers. "The authors then go on to discuss a variety of economic terms that they think apply to publishing, but the quality of the analogies varies quite a bit. It's easy to accept that the limited number of high-profile publishers act as an oligarchy and that they add value through branding. Some of the other links are significantly more tenuous. The authors argue that scientific research suffers from an uncertain valuation, but this would require that the consumers — the scientists — can't accurately judge what's significant. "
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Huh? (Score:5, Funny)
What is wrong with the free market? When has it ever failed us?
Re:Huh? (Score:5, Informative)
When have we ever had free markets?
It is tough to have a free market when there is a monopoly on the issuance of money controlled by a private cartel.
If the America people ever allow private banks to control the issuance of their currencies, first by inflation and then by deflation, the banks and corporations that will grow up around them will deprive the people of all their prosperity until their children will wake up homeless on the continent their fathers conquered.
Thomas Jefferson
Parent
History check (Score:5, Informative)
I love Thomas Jefferson as much as the next American, but there are certain things you listen to him on and some things you don't. Civil liberties, the scope of government, certainly. The economy.... not so much.
Jefferson wanted us to farm our way to victory. Here's some primary source stuff [historytools.org] on the subject for your edification/amusement.
Just because he's a founding father doesn't make him a visionary on everything. See also: slavery.
Parent
Re: (Score:3, Interesting)
What is "farming"? More to the point, what was farming in the 1770's? In the 1700's the industrial revolution had not yet begun. In those days, wealth was still centered to a large degree around land. Jefferson has no other context in which to suggest goals people should strive for.
But what was "farming"? Did he mean people should be wheat farmers, grain farmers, turnip farmers? Di
Re: (Score:3, Interesting)
You raise an interesting and valid point. Certainly Jefferson would be aghast at what "farming" has become now to say the last. And while the Industrial Revolution proper hadn't happened, certainly Britain and the rest of the world had experienced factories. Jefferson was appalled at a supposed "moral decline" that accompanied such things.
And certainly he would be more in favor of self employment, but there are many forms of self-employment that he would undoubtedly be dubious of to say the least. And
Re: (Score:3, Insightful)
Re:Huh? (Score:4, Insightful)
Not only that, it's even worse (though I agree completely with what you said). Even if you assume perfect information, Steve Keen has proven that the equilibrium for finite number of companies is always a cartel.
The classical argument about competition on free market that economists make is wrong, because it at one point relies the assumption that they affect each other so little that it can be neglected, but then they are trying to add up those zeros together. Another possible refutation is that if this argument was true, then the iterated prisoner dilemma would never have a cooperative stable solution.
Parent
Re: Clear example of a failure of "market" (Score:5, Informative)
A softball question. One simple example of the failure of the market is the apparent inability of science publishers, particularly in the pharma area, to publish so-called negative results or to spin negative results as if they are postive. In epidemiology and in pharmacology, negative results are at least as important as postive ones ("first, do no harm"). Yet, the greater economic forces of pharmaceutical sales (and nutricutical sales, and outright woo sales) incent the supression, or simple failure to publish, of such findings in pernicious ways. Check out Ben Goldacre's site (and buy his book while you are there) [badscience.net]. Tucked away among various rants against, among other things, media coverage of medicine, you will find several discussions about this very phenomonon, and why it is so incredibly bad.
Parent
Vickrey Auctions (Score:3, Interesting)
A recent examination of current scientific publishing methods show that it is problematic at best. Treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers.
Agreed. I think we need to switch this whole process to Vickrey Auctions [wikipedia.org]. Then you can explain to the authors of the papers that they will receive $75 for their paper instead of $100 because whoever bid $100 was gaming the system. Why is it suddenly so popular to turn everything possible into an auction system with 75 different flavors of said auction system?
Re: (Score:3, Interesting)
Despite the summary, authors aren't literally like sellers at an auction. They don't receive any money for their papers. For journals with page charges they have to pay to publish!
Publishers are only "bidders" in the sense that they are competing to have the best papers published in their journals. But they don't compete monetarily — as I said, they don't pay authors. They compete through prestige: authors (often) want their papers to be published in the most famous, widely read, influential jour
Publishers as Middlemen? (Score:4, Insightful)
publishers as bidders at an auction, authors as sellers, and the community at large as consumers
Hmmm, there's some term I'm thinking of that deals with people in the middle of the source and the destination that take money for acting as men in the middle when they're not doing anything or providing any service except being in the middle of the transaction. Also, it's beneficial to the sellers & the consumers to eliminate these people. I think they're called 'rich greedy bastards.'
Seriously, hosting a document for me to view doesn't cost $100/mo. so why are you trying to charge me that? I know it's primarily physics but if any other field wanted to pull their heads out of their asses, they would leave the journals to the professors and start up something like arxiv [arxiv.org] for the rest of humanity that can't afford an outrageous premium!
Re:Publishers as Middlemen? (Score:5, Insightful)
Seriously, hosting a document for me to view doesn't cost $100/mo. so why are you trying to charge me that?
So they don't devalue the print versions, which is where they make all their cash.
Parent
Re:Publishers as Middlemen? (Score:4, Interesting)
Parent
Re: (Score:3, Informative)
Thankfully, the NIH has seen the light and now requires that all papers funded by NIH grants are deposited in the open access PubMed Central [nih.gov]. So now biologists can get in on the open access fun.
Re:Publishers as Middlemen? (Score:5, Informative)
Parent
Re:Publishers as Middlemen? (Score:5, Insightful)
Scientific journal publishers are surviving on one thing alone: inertia. And while it makes me sad to see the RIAA try to pull culture with it to its grave, it makes me *furious* to see these groups trying to pull science down with them.
Scientist do the writing, the editing, the peer-review, the *typesetting*... and then turn over the rights to their work for the privledge of paying up to $3,000 per seat to access it. When disseminating information was expensive, this made sense, but now... not so much. But like produces of shiny plastic discs, they'll pervert the laws for years to come to try to buy a few more years of life.
Parent
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)
2. Most publications actually require the author to suggest who should peer review it, so the publisher usually doesn't even have to work to figure out who should review what.
Re: (Score:2, Informative)
They should know better than this (Score:5, Insightful)
I have not read the article. If the summary is accurate reflection of the authors' point about this, then it is at once misguided, and foolhardy. The purpose of business in a modern capitalist economy is to produce goods at low prices that the consumers can afford, generate enough profit to please the shareholders and to set aside enough money to do research to develop the goods and services to increase these profits and consumer good down the road. Sure, businesses cannot be left alone to do what they wish and government regulations limit unchecked profit-mongering, but the primary purpose of businesses is to establish a market share and earn profit for the shareholders.
Contrast this with the purpose of scientific research. The purpose varies from gaining a more accurate understanding of physical, chemical and biological phenomena to leveraging these phenomena into processes and contraptions that improve the quality of human life (where you lie on this spectrum depends on how pure/fundamental or applied your area of research is). The only shareholders in this process are the authors of scientific work, and their reward varies from just scientific renown to funding for future research or even commercialization of the fruits of their research. However, to achieve the most progress, scientific research tends to be 'open source', in the sense that anyone capable of understanding, and with financial resources to buy access to the journals (if the work is not presented in the growing number of free journals online) can read not only what was done, but also how it was done (something commercial concerns never reveal).
Of course, scientific journals are often run like a business (at least successful and well-renowned ones), but to extend these ideas to the actual business of carrying out research is utterly misguided. The goals of business (from a businessman's pov) and science (from a scientist's pov) are very different. The authors might as well apply these ideas to conduct of a military for all the relevance it has.
There are a host of other objections to such treatment as well, but I will pause here as people know what they are.
Re: (Score:3, Informative)
Do any of you idiots who say this actually know the root cause of the current financial implosion?
It's a damned housing bill from the early 90s (Clinton years) which forced lenders to offer mortgages to lower income minorities whether they could afford those mortgages or not. That compulsion spawned huge growth in the housing market and an entire industry built around buying and selling those mortgages and the associated real-estate as well as cashing in on the inflated housing prices caused by the increase
Sounds like they need a cms and acls (Score:4, Insightful)
Why don't they just start their own wiki? I would find a great deal of value in a wiki moderated by a team of reputable scientists that published their findings to the great peer review workflow.
Blame Google? (Score:2, Insightful)
They could do more to help. At $15-25 a read most scientific papers are beyond the means of ordinary folks. Even if you are at a university or institute that subscribes to some journals the chances of getting what you want if you're casually browsing across disciplines (from which many great creative insights come) is slim.
I think the search engines contribute to this problem because the search algorithms do not take account of availability. Search for any specialist subject and you will see the first 10 pa
I can see it. (Score:3, Interesting)
wrong model ? (Score:3, Interesting)
The authors then go on to discuss a variety of economic terms that they think apply to publishing
Which is probably the most problematic point.
While even in the current crisis "market models" are still hip, they don't give the answers to all questions. A scientific conclusion that starts with "if X were a market" must question, among all the other validations, the "if" part as well.
And while economy provides interesting theories that are helpful in many cases - just like evolution, it does not fit everywhere. So the very first thing that would've to be established is that the model fits.
In this case, I've not seen enough of that, so any conclusions drawn are meaningless until then.
Did anybody else read that as ... (Score:2)
pot, meet kettle (Score:4, Interesting)
Is this serious, or just push back from economists who are upset that a number of papers and editorials have recently appeared in high profile scientific journals questioning the description of economics as science? Allegories, for example, are not scientific.
uncertain valuation (Score:3, Insightful)
Instead of claiming that the whole system is broken, just fix the breakdowns in the current system.
The authors argue that scientific research suffers from an uncertain valuation, but this would require that the consumers â" the scientists â" can't accurately judge what's significant.
Any given paper does suffer from 'uncertain valuation', but uncertain doesn't mean the consumers have no clue.
Consider the impact factor of the journal to which the paper was submitted, the reputation of the author, the actual evidence that has been presented in the paper, the fact that the paper has undergone peer review, and what impact the paper would have its claims were to have merit.
In combination, these factors allow better papers to tend to float to the surface. This is fairly typical of a market.
Most fields have decent market regulation built-in, in the form of peer review and independent verification of results.
Politically charged fields (global warming) tend toward unregulated market behavior. Papers are no longer selected based on scientific merit, but instead on hype/scare factor. This makes the value of a paper much more uncertain, and leads to a nasty failure mode (see the current world economy).
There are models that do not have such nasty failure modes (or at least have very different failure modes). Usually, these also fail to produce such good results in the common case.
I dislike the current scientific publishing system because the publishers tend to be paid by both the author and the consumer, and can generally force the author to relinquish copyright. However, the quality of the system seems to me to be far better than this article supposes.
change through consensus (Score:5, Interesting)
I'm a grad student in the natural sciences. Some other friends of mine and I started Labmeeting.com [labmeeting.com] because we are so eager to help change the way science gets published.
The current system of peer review is inefficient, arbitrary, and hidden from public view. We definitely need something new, but, as we said in our talk [blogspot.com] at BioBarCamp a while back, change needs to be gradual enough to preserve consensus.
That's why we're starting by just trying to make research tools that are useful to scientists in their everyday professional lives.
Putt's Law (Score:3, Insightful)
Due to the publish or perish mindset at universities, scientific authors must be prolific. Putt's Law (good book) lampoons this quite well, roughly akin to an Amway style ponzi scheme. Sharing new knowledge with the larger community is no longer among any of the major motives for cranking out papers. Frankly the system punishes those who would compile and distill the huge number of obtuse and often stupid articles into a useable form for us rank and file engineers. Such useful efforts are not "new and novel", despite what would be a great service for those of us doomed to wade through the stacks and stacks of crap papers written in acadamia-ese.
In my job, a hardware design engineer, I find that most of the modern papers are indecipherable and irrelevant at best. Only occasional gems make it through and actually apply to my day job (of designing state of the art T&M hardware). By contrast, the old journals from the 70's and 80's easily have a 10:1 better signal to noise ratio, despite their dated nature.
Dreadful article--not worth publishing! (Score:4, Interesting)
That this article got published in PLOS Medicine is a data point saying that the publication model for PLOS Medicine is flawed. That's about it.
The authors don't bother to back up any of their assertions. If there is a winner effect, for example, the most prestigious journals should have the highest rate of publication of junk results, whereas lower-ranked journals should be more accurate. So, is this true? Did the authors bother to look, or even to think about and discuss it?
Also, does "overpayment" correspond to "poor quality science" or to "only slightly more cool than the rejected paper, on second thought"?
Now, it is more true in the medical sciences that positive results are published that claim to show p0.05, but are one of a dozen similar studies 11 of which have not shown an effect (i.e. overall there was no significant finding). But this recognition has nothing to do with bidding per se; it's not that the journals are picking the high tail of a distribution of value so much as that they're seeking statistical significance without controlling for the number of times that the study was done.
And as the summary says (which is actually better than the research article itself, IMO), there are a number of other problems.
There are certainly ways that one might seek to improve scientific publishing. But this seems almost entirely off target and/or ill-supported to me.
We also only publish positive results (Score:3, Insightful)
One of the major problems with scientific journals and the peer review process is that we have a positive bias for publication, in that you are much more likely to be published if your study has positive results than if it has - equally valid so as not to have everybody else keep doing the same thing and failing - negative results.
Half of getting into Science and Nature is politics, not science.
And just TRY to get something published about improved methodology in statistics for genetics studies ... hah! You have to publish in obscure journals or start your own self-publishing annual or biannual workshop and then attach it to a positive study to get it out there.
It's a flawed analysis... (Score:3, Insightful)
"... treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers."
Not really, not at least in biomed papers.
With those the scientist actually *pays* the publisher to print the article. A paper, especially if it contains colour images (microscopic slides, colourful graphs etc) can set you back by several thousand dollars. That's why there's a little disclaimer under each article that states that since the author paid for the publication, the article legally is paid advertisement.
So no, the authors are not sellers. Only the publishers are the sellers, selling article space to the scientists, advertisement space to corporations and the end result back to scientists. The scientists who publish don't even get the complimentary free copy, although as an author, you can ask them to send you the PDF that you sent them in the first place; now this service is free. On the other hand, if you want the whole magazine on your bookshelf, fork out $200 for a single issue.
It is a wonderful business model:
- Get the article from scientist (free)
- Send it to other scientists for peer review (free)
- Accept it and charge the author (income)
- Sell the advertisement space, at least 50% of the mag (income)
- Print the mag (expense)
- Sell the mag *way* above production/distribution cost (income)
- Keep the copyright to every article so that the author can't republish it without paying you (possible income)
- Profit!!!
The Underpants Gnomes had no clue about business...
Plus there are further tricks - if you manage to bribe the execs of some research association so that membership in the assocation also means a compulsory subscription to your periodical, then the customer base is guaranteed. Of course it is usually sold as "we had to increase our membership fee, but now membership also includes a complimentary subscription to magazine X". This of course increases your readership, making your ranking higher among the sci mag list.
Ah, yes. Chances are that the money they take from the scientist is public money from some research grant. Yet the copyright to the article belongs to the publisher, a private organisation that actually has very little to do with scientific research and everything to do with making profit. That's clever! Even better, that when research grants are awarded, they check your publication record. If you published your articles on the Web, that is worth exactly nothing. Publishing a handful of articles in a couple of those high-ranking expensive publications significantly increases your chance to get a grant, so that you can publish some more...
Imagine if the **AA could find a way to implement this! Every musician and actor paying them to be included on a CD or in a film, plus selling ads even smack in the middle of a CD (music stops, someone screams about the advantages of washing powder X, music continues) and making it compulsory that members of any civil organisation buy a certain number of CDs or DVDs every month. Those Hollywood dudes know diddly squat about making profit.
Re:Doesn't seem to help scientists... (Score:5, Funny)
Parent
Re: (Score:2)
After you're done with the gas stations you can go for the financial institutions and congressmen. Be sure to kill a few for good measure.
Re:Doesn't seem to help scientists... (Score:5, Funny)
Robin Hood? Is that you?
No, he's just robbin' the 'hood. Easy mistake to make.
Parent
Re: (Score:2)
Re:Doesn't seem to help scientists... (Score:4, Informative)
Thanks, I thought it up after smoking two joints.
Parent
Re: (Score:2, Informative)
Re:Doesn't seem to help scientists... (Score:5, Informative)
No, authors don't get paid to appear in journals like Science and Nature. In fact, in most cases the author pays a fee for the space in the journal. It's a total racket.
The benefit to the author is that he can put the paper in his CV. The more big name journals you publish in, the more likely it is that you'll get grant funding and that all important tenure. It's publish or perish out there.
Parent
Re:Doesn't seem to help scientists... (Score:5, Funny)
Parent
Re:Doesn't seem to help scientists... (Score:4, Interesting)
It's also more likely that someone will actually read your paper if it's in a big wide-circulation journal (e.g., Nature) instead of a hard-to-find low-circulation journal. This is particularly true for papers outside your own specialization where you won't necessarily have heard of them at a recent conference. The publication volume is just overwhelming — if you're going to stay current, you need someone else to filter most of the junk for you, and that's the service which selective journals like "Nature" (and review articles) ultimately provide.
-JS
Parent
Re:Doesn't seem to help scientists... (Score:4, Insightful)
Parent
Re:Doesn't seem to help scientists... (Score:5, Informative)
I looked at scientific journals, and I honestly can't see much of an incentive to appear there. I mean sure, you might get published and that's got some merit...
Some merit? In many academic institutions, number of papers published in respectable journals is the preferred metric of performance, and will affect your promotion and the status/funding of your institution.
YMMV depending on the level of enlightenment and subject area of your institution - there are, of course, other aspects which can and should count - but number of papers is the "gold standard" and the safe bet.
If this were a scientific paper, I'd back that up with some references (but my institution definitely doesn't recognise /. karma and mod points, so I can't be arsed).
Parent
Well, in theory... (Score:4, Interesting)
If its a reputable journal, the evaluators assume that the articles have been thoroughly peer reviewed , and the quality can be taken as read.
The big assessment exercises (such as the 5-yearly RAE in the UK which determines the research ranking of universities) have to "assess" a metric shedload of papers - so they're not going to spend too much time on each one!
Of course, the reliability of this assumption is legendary [theregister.co.uk].
Parent
Publishing does help scientists... (Score:5, Insightful)
Having a high publication helps a graduate student land a good post-doctoral opportunity. As a post-doc, you'll need a good publication record (Nature, Science, Cell) if you want to land a good faculty position at a top university (tenure track). A scientist that can semi-regularly publish in the top journals will have an easier time earning grants (without such, they wouldn't be able to run a lab). Without a good publication record, a junior faculty won't get tenure (the review is typically 5-7 years for the biological sciences post hire). Publications - no, make that publications in good journals - is everything.
From the scientist's perspective, if they have pure research, then, they can put it on a web site, such as the university web site or even their own, and just skip the b.s.
Any yahoo can post on a website. The reasoning behind scientific journals is that the science is peer reviewed before being accepted. While not everything published on Nature, Science or Cell is top quality work (politics does play a role), the signal to noise ratio is much higher than say, International Immunology. The science presented in the top journals usually has a much higher impact factor than the 'lower' journals; i.e A paper published in Nature Immunology or Nature Medicine typically has a much broader impact on the field than, say, a paper published in Journal of Immunology. That's not to say that the JI paper is worse than the Nat. Imm. or Nat. Med. paper - it's not. Just that the JI paper will likely be much more narrow in scope.
Parent
Re: (Score:3, Insightful)
You make is seem so objective. As a scientist I can honestly say that publishing has become a racket. It used to be you sent a little postcard and received a copy of the article from the scientist who had published it. Now they want you to buy the damn thing on line or subscribe to that journal for hundreds if not thousands a year.
Peer review is often no more than an attempt to stifle other peoples work. At one time science was brought to the people..may be that's why we are such an scientifically illite
Re: (Score:3, Interesting)
You make is seem so objective. As a scientist I can honestly say that publishing has become a racket. It used to be you sent a little postcard and received a copy of the article from the scientist who had published it. Now they want you to buy the damn thing on line or subscribe to that journal for hundreds if not thousands a year.
Peer review is often no more than an attempt to stifle other peoples work. At one time science was brought to the people..may be that's why we are such an scientifically illiterate nation! We still put all the articles in Ivory Tower Journals that few people in the mainstream read.
Finally the politics of publishing is worse than you think furthermore, many journals it's not about how good the work is but whether you can adapt it to their "publishing format."
Your postcard method will still work 100% of the time. Every scientist will be delighted to email you a pdf of his requested paper. However, the new "racket" system, though incredibly expensive, does give new options for much more efficient distribution. Yes, the price is a major problem, but it didn't supplant the old system.
Re: (Score:3, Insightful)
Not really, most people out there would struggle to understand even the abstract of a proper[1] scientific paper.
Basics first. There are people out there who don't understand evolution. There are people out there who think catalysts are a loophole [slashdot.org] in the laws of thermodynamics.
Obligatory car analogy: making all the roads good enough for F1 cars[2] won't help if all the other cars are Ladas.
[1]
Re:Doesn't seem to help scientists... (Score:5, Insightful)
Information doesn't want anything.
YOU want information to be free.
Parent