zlacker

> many articles would require 2 hours+ to confidently review as a professional

I think ML (and really all other fields) are the same. Skimming a paper never really leaves you certain of how rigorous it is.

I agree that a naive "just add voting" "review" mechanism would not suffice to replace journals. However there's no requirement that the review algorithm be so naive. Looked at differently, what is a journal except for a complicated algorithm for performing reviews?

> I am afraid the need for these publishers will still be there and they will just exist regardless, and it will still be preferred by academics.

Agreed. I doubt publishers are going away any time soon (if ever) regardless of how technically excellent any proposed replacement might be. I still think it's worthwhile to pursue alternatives though.

>>fc417f+(OP)
I agree with your points overall. Regarding "what is a journal except for a complicated algorithm for performing reviews"? I think one point is that there is a hard-to-quantify social contract between the journal editors and specialized reviewers, which are partially hand-selected over many years (and opaque). Editors overall do rely on verifiable experts in the field with an established reputation, both publically and privately. Reviewers can have some sort of direct interaction with the editors, coming with opaque trust-verification. Editors also tend to go to scientific meetings as well and do have undocumented or unofficial interactions with scientists (and their favorite reviewers). Now, reviewers might ask their students to review a paper and they sign off on that after a quick skim, but not all of them do and especially if the paper has more caliber/weight to it, they do tend to take it seriously personally.

Another issue when going to a decentralized tool is that I think it should apply some sort of gate-keeping to only allow academics or verified scientists to contribute reviews, but then you also need a way to prevent bias/friend/self-citation network interactions between the academic reviewers, which means you would need to keep good track of them? Not sure how to handle that.