Review Bounties

Fixing incentives in peer review

A funding model to replace article processing charges (APCs), and subscriptions which pays reviewers and incentivises diligence with a ‘bug bounty’.
literate science

Richard J. Acton


June 24, 2023

There is legitimate added value in some of the conventional functions of the academic publishing industry and the people adding that value should be remunerated fairly for their labour.

The ‘journal’ as currently constituted provides:

The bulk of the most vaulable labour performed in this process is performed by the reviewers who are unpaid. The rest of the labour and costs here simply detract from the bottom line of the publishers who because of the vertically integrated nature of the publsihing pipeline have little incentive to provide a high quality service in areas such proofreading. The bulk of the value acruces to the owners of the publshing companies who can extract rents on the copyrighted articles, and restrict access to artifically limited high presige publication space which they can charge to access via APCs. They pay their professional editors relatvely poorly, whilst expecting them to judge the quality of an unreasonablly large number of articles per unit time and according to dubious criteria. This leaves the door open to considerable corruption and gaming of this system.

As an alternative to this way of working I propose ‘review bounties’

In it’s simplest form it would look like this:

Instead of paying an article processing charge authors offer a review bounty. the ‘Editor’ of a journal agrees to mediate the review process for some fraction of the review bounty. A portion of the remaining bounty is divided among the reviewers of this paper, if the editor deems the review of succicient quality, according to clearly indicated expectations of what constitutes a good qualtiy review. The remainder of the bounty is offered as a ‘bug bounty’ such that anyone identifying an error which materially alters a conclusion of the work can claim it.

Bug bounty claims would be adjudicated by a pre-specified procedure or rules for deciding if the claim is valid, for example:

  • If the Authors agree
  • or if the editor and a majority of the reviewers agree, (editor breaks ties)

Under this model the journal performs its conventional functions of aranging review and hosting / distributing the published paper but the reviewers get paid for their labour and an incentive is created to find errors in the published literature. The papers are published under an open license such as a CC-BY or (preferably in my view) CC-BY-SA so that they can become a part of the knowledge commons. This incentive structure encourages authors and reviewers to avoid errors in the first place as they are staking some cash on the assertion that they have not made any errors whilst also incentivising 3rd parties to try and spot errors in the published literature

Extensions to the review bounty model

In it’s simplest form ‘journal’ combines a number of potential functions

  • Additional parties
    • Proofreaders & type-setters Instead of having a journal do this in house they could directly be in on the bounty cut
    • Hosting, host of sites for servering publications, and services for administering their publication could also be directly included, especially in a context where journals are no longer as relevant and publications might be shared independently of them.
  • Extened bounties from interested parties e.g. a pharma company is contempating starting a new program based on this work they can put a large bug bounty on it to try and attract additional 3rd party srcutiny and save them money on the long run from on investing in a dead end if an error is discovered early.
  • Grant award bodies could require minimum bug-bounty amounts / proportions to ensure that work theat they fund gets adaquate srcutiny

Integration into a larger picture of publication workflow reform

This proposal forms a part of a larger set of reforms to the conventional publishing model that I will be writing about here under the heading literate science