Experiment On Public Pre-reviewing and Discussion of Workshop Paper Submissions

An anonymous reader writes: The ADAPT workshop (6th international workshop on adaptive, self-tuning computing systems) is trying a new publication model: all papers have been submitted via Arxiv, are now publicly discussed via Reddit, and will then be selected by a Program Committee for a presentation at the workshop. The idea is to speed up dissemination of novel ideas while making reviews more fair and letting the authors actively engage in discussions, defend their techniques, fix mistakes and eventually improve their open articles.

Re:

[Beck_Neard]

Figures that this would be the first response on slashdot.

Your two-bit brain is not capable of understanding the real world. Ask your parents for a genetic refund.

Re:

[Anomalyst]

the integrated face system is a system that reviews faces and villafies them for submission.

FTFY

Re:

[guises]

The Reddit thing is the notable part, and bizarre. A very odd choice. It seems likely they're doing it for publicity - no different from Facebook integration. It's just a workshop after all, and doing something weird like this can draw people in.

Re:

[serviscope_minor]

The Reddit thing is the notable part, and bizarre.

I interpret it differently.

Reddit is basically a discussion forum and allows voting on stuff. I was assuming they they were going to reddit because that meant they don't have to set up their own forum, secondly, don't have to persuade everyone to sign up to their own forum (a probably nontrivial fraction of people will already have reddit logins).

Re:

[enriquevagu]

No, they are not doing this for publicity. Their goal is to improve the review process by attracting more people interested in the topic to the review process and make it more open.

I happened to attend the workshop last year and there was a very interesting discussion at the end about how to modify a review process to make it more open. While I didn't take part in the discussion, there were many aspects considered about the open peer-review process, both positive and negative. For example, some authors migh

Re:

[the_povinator]

Agreed. The ICLR conference, which is a machine learning conference organized by Yann LeCun (who now heads Facebook's machine learning group), is a bit like this. They use their own site for discussions, not Reddit, though.

Re:

[fph il quozientatore]

The same reason why you can't steal an already-published idea: your paper is publicly available and timestamped on arxiv, so everyone provably knows that it was your idea first. Moreover, a google search will likely reveal your previous contribution.

Re:

[techno-vampire]

One other thing I like about this form of reviewing articles is the fact that it can also prevent a small group of reviewers with an axe to grind rejecting everything that doesn't fit their world-view. I won't say that it's being done right now, but I do know that it's been claimed at least once.

Ah, arXiv

[mbone]

You do know that some people are blocked from arXiv, and at least in some cases there is no obvious reason why (and no real appeal)? (Opponents of string theory, for example, seem to get this, or at least complain about this, fairly often.) I have seen this in action, it is real and it is capricious.

I do not think that arXiv is suitable for a filter for a public meeting as long as its internal filtering is opaque in this fashion.

Re:Ah, arXiv

[fph il quozientatore]

I have never heard of this, and I am interested. Can you name an example of a respectable scientist (not a "fringe" controversial person, I mean) who has been banned?

Re:

[Anonymous Coward]

I have never heard of this, and I am interested. Can you name an example of a respectable scientist (not a "fringe" controversial person, I mean) who has been banned?

Nobel Laureate Brian Josephson had this happen to him. He posted a record of his interactions with the arXiv moderators at

http://www.tcm.phy.cam.ac.uk/~bdj10/articles/arxiv_correspondence.html [cam.ac.uk]

Often the people who get their papers rejected by arXiv are just ordinary researchers who are trying to post ordinary papers. One such example is the following.

http://blog.tanyakhovanova.com/2013/12/arxivs-police [tanyakhovanova.com]

Re:

[Anonymous Coward]

Has Josephson been banned, or is he just upset that his paper, which seems to be more on the philosophy of biology and quantum mechanics, was deemed not appropriate for being listed as quantum physics? The paper is still on arxiv. His communications with the moderators and higher ups could maybe have been a little more professional. Not that it should matter. But the actual complaint is very questionable, and it is not a clear case of the moderators blatantly disregarding what they should be doing, as i

Re:

[Anonymous Coward]

Has Josephson been banned, or is he just upset that his paper, which seems to be more on the philosophy of biology and quantum mechanics, was deemed not appropriate for being listed as quantum physics?

Josephson claims that he was temporarily "blacklisted" from posting to arXiv. See

http://www.tcm.phy.cam.ac.uk/~bdj10/archivefreedom/main.html [cam.ac.uk]

Re:

[eric_harris_76]

Not banned, and not from arXiv, but a recent Nobel laureate was co-author of a paper that recently didn't survive peer-review at a couple of supposedly prestigious medical journals (JAMA and NEJM, as I recall). (Link below.)

They took their paper elsewhere, where it was better received.

How recent? The latest round of awards. How recently? The rejections happened before the awarding of the prize.

Feather in the cap of the publishing journal, and a black eye for the two that rejected it.

https://duckduck

Re:

[mbone]

I have never heard of this, and I am interested. Can you name an example of a respectable scientist (not a "fringe" controversial person, I mean) who has been banned?

Marni Sheppeard [physicsworld.com].
Peter Woit [columbia.edu].

Note that they are not (as far as I can tell) banned, just blocked. Nothing is made public, it's just that certain things seem to happen consistently. And, in my experience, moderated papers are not available to the public.

Note that the real problem here is not that papers are moderated. I understand the desire for moderation. It's the way it's being done that is problematic.

Re:

[Sibko]

Can you explain to me why a "fringe" scientist (err... controversial person...?) shouldn't be allowed to speak on Arxiv? I just find it really curious that you immediately imply it's okay to censor a certain kind of speech you don't personally like. I mean, you're pretty much using the No True Scotsman fallacy right here; if the OP comes up with a name you can just declare him to not be a "real" scientist and you'll never be proven wrong.

You either have an open forum and the idiots that come with that, or y

Re:

[serviscope_minor]

Can you explain to me why a "fringe" scientist (err... controversial person...?) shouldn't be allowed to speak on Arxiv?

Because the actual scientists using arXiv don't want to have to manually filter out a crapflood of spam from whackadoodles. Make no mistake, if the normal channels get too much "fringe" science, then the real scientists will leave and it will be worthless for anything but timecube theories.

I just find it really curious that you immediately imply it's okay to censor a certain kind of speec

Re:

[mbone]

If you are unable to contact the organizer of a workshop when something like that comes up and work out an alternative when needed, then you will likely have much bigger problems.

I was actually thinking more about the organizers than the submitters (I have organized scientific meetings). One reason why organizers like this is that the arxiv paper submission is part of their automated submission process (i.e., they don't have to set up a paper hosting service, arXiv does it for them). This means that there is a real risk that arXiv is involved in their Editorial process, and that they might not even know it, and I think most conference organizers would find that unacceptable.

Just Havta GNow

[Anomalyst]

is the program to be in Java and interpreted, C and compiled or specific to a CPU family and assembled, possibly in binary and toggled in via the front panel (don't reviews require a panel of ancients)?