The Unsung Heroes of Scientific Software (nature.com) 62
An anonymous reader sends this news from Nature: For researchers who code, academic norms for tracking the value of their work seem grossly unfair. They can spend hours contributing to software that underpins research, but if that work does not result in the authorship of a research paper and accompanying citations, there is little way to measure its impact. ... Enter Depsy, a free website launched in November 2015 that aims to "measure the value of software that powers science."
[Postdoc researcher Klaus] Schliep's profile on that site shows that he has contributed in part to seven software packages, and that he shares 34% of the credit for phangorn. Those packages have together received more than 2,600 downloads, have been cited in 89 open-access research papers and have been heavily recycled for use in other software — putting Schliep in the 99th percentile of all coders on the site by impact.
Check out free open source science software on SourceForge.
[Postdoc researcher Klaus] Schliep's profile on that site shows that he has contributed in part to seven software packages, and that he shares 34% of the credit for phangorn. Those packages have together received more than 2,600 downloads, have been cited in 89 open-access research papers and have been heavily recycled for use in other software — putting Schliep in the 99th percentile of all coders on the site by impact.
Check out free open source science software on SourceForge.
Comment removed (Score:5, Insightful)
No it isn't (Score:1)
Unless its freeware the guy gets paid for writing the stuff so he can't complain and if it is freeware then that was his choice to do it. He can't co-opt someone elses plaudits just because he created a tool they used.
If you really believe that then perhaps you think the company that built the power sockets or the desk or made the PC motherboard should also get a mention? Where does it end? We all use things invented and built by others to do our jobs.
Comment removed (Score:5, Insightful)
Re: (Score:1)
"Rather, the point they are making is that being the (co-)author of such a software should count as much as being the (co-)author a paper, with regard to getting tenure, or for general performance reviews within academia."
Why? The software is a tool. The software authors had nothing to do with the original research undertaken by the scientists. You might as well make the programmers of MS Word that the paper was written on co-authors too. Its absurd.
Re:No it isn't (Score:4, Insightful)
A lot of scientific software is extremely specific and designed to enable very specific types of research. There are many fields where there are 1 or 2 pieces of software exist and they where written inside the community.
They most certainly should get credit for writing that software and enabling very specific research since in many cases without the software the research would not even exist.
Comment removed (Score:4, Insightful)
Re: No it isn't (Score:1)
Right also word isn't free and it's as silly to credit MS as to credit Toyota for making the car that you drove to work . Crediting the guy that gave you a lift everyday and saved you an hour commute is reasonable and a better analogy
Re: (Score:2)
Re: (Score:1)
"They most certainly should get credit for writing that software and enabling very specific research since in many cases without the software the research would not even exist."
Without many other things the research wouldn't exist either. Unless the software or its authors came up with the idea of the original research or had some hand in how it was conducted then they have no business being on the paper and just because some piece of software is a one off doesn't change this in the slightest.
Perhaps the gu
Re: (Score:3)
This is a horrible comparison. Shakespeare could have done his plays in any theater.
I am talking about custom written software that solves an EXTREMELY narrow niche and without it a lot of the research in the field would not even happen.
I
Re: (Score:2)
But who did the research? As I mentioned in another reply, when I was a research assistant, if some professor came up with some algorithm or criteria for a simulation and asked me to implement it, why should I get any research credit? I did get mentioned a couple of times in credits for having implemented the software, but anyone at my level could have done it... I got zero research credit in those cases, and didn't deserve any.
Re: (Score:3)
Well the software I am talking about is the result of a lot of very difficult research that has taken many years to do. The people writing the software are doing the research.
There is no simple algorithm that a professor came up. It is a complex physical model and it is highly non-trivial to figure out which parts need to be modeled and why to get the correct physical behavior.
Others use the software to try and solve a specific problem that would have been impossible without a lot of very hard work in getti
Re: (Score:2)
Re: (Score:2)
It has had lots of research papers. However, when people use it the software/papers should also be cited.
Re: (Score:2)
In the past I remember some software would get a small academic paper, enough for a conference. Then it would get cited many times by others who used it. But software is getting more complex now and may need more than a team of 2 or 3 grad students to develop and maintain it.
Re: (Score:2)
"This is a horrible comparison. Shakespeare could have done his plays in any theater."
And the people doing the research could have probably done it in Matlab given enough time. Your point is?
Re: (Score:2)
Re: (Score:2)
But writing software isn't research or necessarily insightful at all - if one person comes up with a theory or model or formula, and another person codes it, the person who coded it didn't contribute anything to the actual research, and any decent programmer would be able to implement someone else's algorithms. If the programmer came up with a way to test or do something in particular, then they certainly should get credit, but merely implementing someone else's research is NOT research in, and of, itself.
Re: (Score:2)
Re: (Score:2)
The code is integral to the research. Your prof's theories are useless without the code that shows that they are of some use and that they work. Just as you are "replaceable", your professor is, too. So don't think any less of yourself just for being replaceable: most of us are. Furthermore, personally, I believe the implementer is due same credit as the researcher. One can't exist without the other, it's a very close symbiosis.
Re: (Score:2)
The code is integral to the research. Your prof's theories are useless without the code that shows that they are of some use and that they work. Just as you are "replaceable", your professor is, too. So don't think any less of yourself just for being replaceable: most of us are.
I didn't exactly mean it that way. I'm not trying to understate the importance of implementing something complex.
Furthermore, personally, I believe the implementer is due same credit as the researcher.
I just disagree with that... if the researcher comes up with a theory, and the programmer comes up with a way to test it, then sure - but if the researcher comes up with a theory and the programmer implements a way to test it that the researcher gave the programmer ("this is how you test it, here's the algorithm"), or it was some known way to test the theory, then the programmer hasn't actually
Re: (Score:2)
I don't think we're really talking about who gets credit on a published paper. We're talking about someone in the employ of a research institution or higher learning who works on software that enables others' research and allows them to publish research that they otherwise would not be able to. We're talking about someone receiving credit for the work that they've done with regards to their own professional accomplishments. Traditionally the only work that's been of any significance to their performance
Re: (Score:2)
If one person comes up with a theory or model or formula that is not implemented at the moment, it's not even generally possible to tell if their contribution is of any use. To get results, you need an implementation, and the implementation is absolutely crucial. Even in terms of algorithmic improvements in "pure" computer science, if the real life implementations are constrained such that the algorithmic improvement doesn't matter, the value of the theoretical improvement is decreased or it simply becomes
Spice 2g6 (Score:3)
The Spice electrical circuit simulation software was developed in FORTRAN on several platforms (including VAX VMS) in the 1970s. I managed to compile it for Linux and Windows years ago, and I host the source and binaries [rhadmin.org] on a laptop in my basement.
This specific version is in many circuits textbooks - newer versions are not compatible with the syntax of this release. I see a fair amount of traffic for it. I should probably spend some time on a nicer HTML5 download page.
Re: (Score:2)
These program are often being written by academics which can cut into time doing research. But these programs are vital for the research. If this were physics and the academic spent a significant portion of the year helping to build and manage a new particle collider, do they get credit for it or get accused of wasting time instead of writing papers?
This is not a binary freeware versus commercial software debate.
Re: (Score:3, Informative)
Yes, he chose to do it, but it's a choice that for many people means the end of their academic career. There's a huge amount of great scientific software coming out of the universities, but it tends to count very little toward a new faculty's tenure review. If you don't get tenure; you're fired. The unique issue for software is that it takes a very long time to write well and drains resources for its maintenance. For tenure, you're typically evaluated on your ability to get grants, research papers, depa
Re: (Score:2)
Re: (Score:2)
If someone is publishing 5-10 papers a year, this means someone else is doing the actual work and that person is only authoring the papers. That brings up the topic of authorship. Whose names should go onto a paper, and in what order? The first author is sometimes a person who puts together the manuscript template, gets comments from everyone else, and searches EndNote's pubmed connection for manuscript titles that match the context without actually reading the cited article. (You know what I'm talking abou
Re: (Score:2)
Unless its freeware the guy gets paid for writing the stuff so he can't complain and if it is freeware then that was his choice to do it. He can't co-opt someone elses plaudits just because he created a tool they used.
I think you have missed the point. It isn't a matter of "co-opting someone else's plaudits", more of getting an appropriate share of them. Perhaps that share should be a small one; or perhaps it should be dominant. The fact that it's freeware isn't really relevant. Do you think that the inventors of the alphabet, the number system, the periodic table of the elements, or the Web should be denied credit for their work just because they didn't "monetize" it? Believe it or not, there are still other values than
Re: (Score:1)
Isn't this a broader problem in academia? Where I work there is huge pressure for teaching commitments, and zero recognition for it. The same goes for admin: one of the senior "makes the departmental world go round" jobs was unfilled for months because nobody wanted it, everyone knew it was a huge time-sink which took away from research and grant proposals but also came with zero recognition. Eventually the Head of Schoo
Publish a description (Score:2, Interesting)
...of the software and then request citations as parts of the license agreement. It's not a guarantee, and yes some will fail to cite properly, but at a present a short (possibly even conference) paper with a citation request really is you're best bet to get some credit.
The difficult part is finding a journal that will accept a description-of-software type paper *and* has a decent ranking. I'm somewhat lucky to have such a journal in my field, but I know that other fields are not so fortunate.
Re: Publish a description (Score:2)
You don't have to mess with your license at all. Scientists are good at citing things if you give them something to cite.
We try to publish a few papers yearly about new aspects of our software... and the scientists that use those pieces of the software naturally cite this papers without issue. We post our citations on our website and many people also email the mailing list to ask for the appropriate thing to cite when they're publishing findings based on our software.
What about Scientific Linux? (Score:2)
Scientific Linux is a distribution along the lines of CentOS that is sponsored by Fermilab.
From their about page:
Re: (Score:3)
It's crap. It combines the worst of CentOS / Red Hat Linux with the pitfalls of academically managed IT.
Fedora is much better, where bugs do, gasp, get fixed in a timely manner. If you can manage the shortish release cycle.
You know, I see this a lot opinion from youngsters who have never administered mission critical systems. For your basic web server & SQL database, Fedora is fine - but how many copies of Fedora would ever get installed on the ISS?
Re: (Score:2)
Well, he is not on ISS. Why does he need to care?
The work is exploratory, not mission critical. The needs of academia are almost the opposite of ISS or even an average business/web setup. An occasional crash is not a big deal in academia. Getting a new algorithm that someone recently published to work is. Many systems in academia are sloppily managed and that's fine. People aren't doing IT here. These are researchers doing science, not sys admins. IT is an after-thought. Its just-enough IT. People look to U
Re: (Score:2)
The mention of the ISS was not accidental: Scientific Linux has, in fact, been installed there [scientificlinux.org]. As a matter of fact, in experimental physics the OS often is mission critical. You need something to run the detectors, after all. While at a low level that usually means firmware or ASICs, at the level of the experimenter that typically means using a computer, with a rather conventional OS installed. Stability there is vital, as a crash could mean (worst case) damage to the equipment, or best case some usually r
Re: (Score:2)
They also produce a version for CPUs without PAE, because CERN have loads of old boards controlling $deityKnowsWhat.
Very few mainline distros will work on machines like my old T-40. I think there's an LTS mint that does too.
Re: What about Scientific Linux? (Score:1)
How crap is it compared to cent is. I thought it was a recompiled red hat from source
Maybe slower releasing updates
Re: (Score:1)
Unfair question! I'm sure hats aren't allowed on the ISS.
Re: (Score:2)
Re: (Score:2)
Depsy currently only scans Python and R projects.
It's probably going to take a bit more processing power to include all popular programming languages.
Re: (Score:2)
SL was so much outdated that we ditched it out of our cluster 2 years ago and replaced it with ubuntu server. Having to manually install a compiler that handles C++11 in 2014 is pretty bad. Hell, we even ditched it out of the whole CS department. I grant you it's robust (even if we had some dirty problems), but it's too outdated to be usefull.
No need for heroes (Score:5, Insightful)
Please stop the bullshit of heroes with superpower in science, as science never was about glorifying people and personality cults. Leave that to the entertainment industry.
The whole science star system is doing much more harm than good to the actual scientific outcomes.
People tend to optimize the metric that is used. If that metric is popularity, they'll do as much as they can to become popular, with no correlation whatsoever to the importance of their original field. Publishing crappy results on a new dataset so that everyone can beat you gets you more citations than providing insights to why some methods work and some don't. Publishing a shitty software that allows a million master student to make up wrong results for the master's thesis get you more download than writing a correct implementation of uncommon algorithms.
I went into science because I didn't give a shit about the smoke and mirrors that are so important in other fields. Most of my best technical students now are just disgusted by the "appearance prevails" mentality that is at the core of other disciplines. Please leave science as it ought to be: efficient but careless about the image.
How do you get listed? (Score:3)
How do you get listed? My software project isn't listed. We have a few hundred users and we're nearing about 1000 citations. It's an open source project on GitHub.
How do we tell Depsy about it?
Re: How do you get listed? (Score:2)
Oh - I see now. It's only for Python and R. We're in C++ so no go. Bummer.