zlacker

>>sixtyf+(OP)
The main issue is that currently there's no way around publishing at the established journals and conferences, because of the reputation they've built up. Funding and career advancement hinge on publications in these venues. If we accept that publishers don't offer much at all in return for the publication fees/library subscription costs (barely any editing, reviewers work pro bono, hosting PDFs is very cheap nowadays), the main issue is one of "reputation transfer".

One way to cut out the middle man would be to convince journal editors to run sibling journals alongside the existing journals, so for each "Transactions on XYZ" there would be an "Open Transactions on XYZ" (as close in title as is legal). Importantly, each sibling journal would be run by the exact same academics (who are doing the real work on tax money anyway), and according to the same process as the original journal, just without involvement from a traditional publisher. PDFs would be hosted on a site like arXiv. The goal would be that submitting to the open "sibling" would be the obvious rational choice (same people, same decision-making, no fees, open access), which in time even the funding agencies and tenure committees would have to acknowledge.

>>frankl+vk
I volunteer for a project [1] where the idea of "cutting out the middle man" is taken even further: removing the journal from the reputation transfer.

So rather than a reviewer lending their reputation to a journal, and that journal then conferring a stamp of trustworthiness onto an academic work, reviewers lend their reputation directly to the works they review. The works themselves can still be shared far and wide, e.g. via ArXiv.

Of course, inertia is still a massive force to work against.

[1] https://plaudit.pub

>>Vinnl+Nu
This is looks like a horrible idea. The system violates anonymous peer reviewing standards. These are sometimes even mandated by law, e.g. our funding institution would not allow us to count such a publication as "peer reviewed." There are many good reasons why peer review needs to be anonymous on both ends (the reviewer is anonymous and the manuscript is anonymized). Many reputable journals have even switched to triple blind peer review, i.e., the editor in chief does not know which two peer reviewers are selected.

The biggest problem I see with the proposed system is that it's unfortunately often easy to recognize who wrote a paper (which is bad for ordinary peer review already) and in a personal endorsement system this would lead to collusion among researchers with low integrity. You wink through my papers, I wink through your papers. Also, imagine you don't endorse the paper of the senior researcher in charge of your postdoc funding...

>>jonath+ux
In practice (as you partly allude to), the problems you speak to already often occur for traditional peer review, except they're less traceable there. Sure, collusion rings and nepotism is (still!) possible, but the data is out there for everyone to see and to call you out on. Now winking through a paper risks your reputation (the very thing that makes your endorsement relevant!), rather than only being potentially noticed by an observant editor.

(Something similar goes for researchers in charge of your postdoc funding: how many co-authorships are earned, and how many are ways to game the current system?)

I'm certainly not saying that a public endorsement system is the end-all-be-all and won't have its own problems. However, I do get frustrated every now and then by the institutional inertia that arises from holding new initiatives to higher standards than existing ones (see also: using the Impact Factor to evaluate academics).

>>Vinnl+7D
I don't think it's a good idea to explain away valid criticisms as inertia. I also don't buy the implicit claim that giving up the anonymity of the peer reviewers could improve peer reviewing. It seems obvious that giving up anonymous peer review would make things worse and that putting the reviewer's reputation on the line would not improve things. It creates all kinds of conflicts of interests and biases. You also need to understand that the people who game the system have a low reputation anyway. They find a niche, build up their collusion ring with people they meet at conferences, and get a tenure-track position or full tenure because of their high, though low quality publication track record. Moreover, if the peer review is properly anonymous, then reviewers are way more critical than if it's not anonymous. Hence, acceptance rates in top journals in my area are below 5%. If the reviewer is not anonymous, they would be way more permissive, since they don't want to completely shatter their relations with their colleagues. Higher acceptance rates means lower value publications, not higher standards.

There are many things that needs to be addressed to improve science, not just peer review. Hiring policies need to change and indicator counts need to be used more adequately, empirical studies need to be pre-registered and the p-value needs to be 1% or lower. Peer review is just one factor and mostly a monetary issue. If all research institutions would spend enough money to make all commercial journals available for everyone, there would be no particular problem with peer review. The problem of many researchers in poorer countries is that they don't. The problem is not that peer review does not work.

>>jonath+JR
I don't mean so say that criticisms aren't valid; just that perfect is frustratingly often the enemy of better.

For example, the status quo can be justified by "it seems obvious that" rather than being evidence-based. In practice, single or double blind is very often not actually blind [1], in which case it's hard to argue that it would be any different than transparency. Likewise, a new solution "creates all kinds of conflicts of interests and biases", without even considering whether the existing CoIs and biases are any better. Likewise, if people who game the system today have a low reputation, why would the same not hold true in a new system? Likewise, does anonymous peer review actually improve the quality of the reviews (e.g. [2])?

It's good not to blindly embrace something new, but I think it's important to withold judgement too, and see how it plays out in practice, and to make an effort to compensate for the prejudice people typically have towards the status quo.

And yes, absolutely agreed that much more needs to be addressed to improve science. But I also very much take issue with the idea that peer review is fine as-is.

[1] https://absolutelymaybe.plos.org/2017/10/31/the-fractured-lo...

[2] Heading "quality of feedback": https://plos.org/resource/open-peer-review/

>>Vinnl+ne1
The point is that there is nothing inherently wrong with blind peer review, you're proposing the classical solution to a nonexistent problem. The problem is that too many reputable journals were lured over to big publishing houses and these want to use them as a perpetual money-printing machine. To fix the problem, you create free, voluntary-based reputable open access journals with triple blind peer review that will (slowly) replace the expensive ones. This is happening.

Maybe you should consider that you're the stubborn one who links to blog posts from open access journals with a strong bias to reform the peer review system as evidence. Plos journals have this agenda because they don't have good enough pools of reviewers, which is the problem of all new journals. However, it's not as if scientists all over the world and funding authorities haven't thought about the topic. People should not withold judgement about these issues, that's an odd request and would weaken the position of scientific staff towards political decision makers who certainly won't withold judgement.

To give you an example of what is being done by funding authorities, at our university we are required by law to publish every article in openly accessible format. Otherwise they do not count at all towards our salaries, they will be ignored completely. There are public open repositories for that. The only thing that bothers everyone is that big publishers like Elsevier only allow draft versions in those repositories and there is sometimes a 1 year mandatory delay until they can be put online. That needs to change and EU authorities are working on it.