Skip to main content

How to start your own preprint review community on PubPub

With PubPub, anyone can publish and distribute meaningful, impactful reviews with appropriate metadata that can be picked up by aggregators in about an hour — at no cost and with no technical expertise required.
Published onFeb 01, 2021
How to start your own preprint review community on PubPub

Since we launched the Connections feature last year, we’ve been thrilled to see communities on PubPub using it for everything from supplementary material to editorial commentary and beyond. One of the most exciting uses of the feature has been publishing reviews of preprints, most prominently demonstrated by the MIT Press’s groundbreaking Rapid Reviews: COVID-19 (RR:C19) journal, published in collaboration with UC Berkeley.1

We’re particularly excited about this use-case because we think the “Publish, Review, Curate”2 models being pioneered by Rapid Reviews and other innovative groups like PREReview, Peer Community In…, Review Commons, and eLife’s Sciety could fundamentally change scientific publishing — making it more open, more transparent, more efficient, and, crucially, more equitable by recognizing evaluation as an essential part of scientific careers.

The community is still working on the processes, workflows, standards, and values that will support this emergent form of publishing. But that shouldn’t stop anyone who wants to explore these models from starting now.

With PubPub, anyone can publish and distribute meaningful, impactful reviews with appropriate metadata that can be picked up by aggregators in about an hour — at no cost and with no technical expertise required.

How it Works

If you’re already familiar with PubPub, the core principle is simple, and can be adopted by any existing community. All you need to do is create a Pub with a Review Connection to a preprint. PubPub will automatically add a card to the user interface indicating that the Pub is a review. When you publish your review, you’ll have the option to deposit it to Crossref as a Review with the appropriate relationship type and metadata about the review. Once deposited, the review will be available via the Crossref API, which will allow preprint services, review aggregation services, and browser extensions to find and display your reviews on their sites if they find your community meets their requirements.

When you publish your review, you’ll have the option to deposit it to Crossref as a Review with the appropriate relationship type and metadata about the review.

In addition to reviews, PubPub supports Commentary, Discussion, and other Crossref event types (and we’d love to work with you to expand beyond those pre-defined types).3 We’ve also developed relationships with major preprint servers and aggregators to make it easy to pull in reviews from PubPub communities.

Step 1: Create Your Community

The first step is to create your community on PubPub. This requires creating a PubPub account, which is a simple one-step process. Your community will need a name and a URL ( to start. A description, logo, and colors are great to have, but you can always add them later.

How to Create a Community on PubPub

Step 2: Add Information About Your Community

Once you’ve created it, you’ll want to add information about your review community, so readers know why they should trust your reviews. This document walks through the first steps of designing and structuring your PubPub community. We recommend including a mission statement, editorial board, and review policies if you have these elements, but it can be as light or thorough as suits your purposes. The Harvard Data Science Review and RR:C19 serve as good examples for providing readers with this kind of contextual information

You can add this information directly to the homepage, if you’d like to get started quickly, or create new Pages for each one. Either way, you can visit the Pages tab in the Community Dashboard to get started. You can design your pages with blocks of rich text, banners, buttons, lists of Pubs, and arbitrary HTML. Once you’re ready to make your pages public, simply set it to public and click save.

The Pages tab in the community dashboard.

By default, new pages will automatically be added to your site’s navigation bar. To customize your navbar, visit the Community settings tab and scroll to navigation.

Navigation settings in the Community Dashboard settings

Step 3: Create Your First Review

When you’re ready to create your first review, click the Create Pub button on the top right of any page.4 This will start a new Pub in our collaborative editor. The Pub will be blank at first. If you’re starting with a review that was drafted elsewhere, you can import it from multiple file types.

How to Navigate the Pub Editor and History

If you’re starting the review from scratch, you can just start typing. PubPub’s editor saves your private changes automatically, and stores a full keystroke-by-keystroke changelog that you can use to see changes over time.

If you’re working with others on the review, you can add them as Members to the Pub so they can collaborate on it with you in real-time. To do this, click the Dashboard button in the top right and go to the Pub dashboard. Then, visit the Members tab and add your collaborator. They’ll need to have a PubPub account to be added. You can add collaborators with different permissions depending on how what they need to do — from just viewing and commenting on the view to creating a public Release of it. Full permissions are described in the permission dropdown on the Members tab.

The Pub Dashboard Members tab

Alternately, you can invite someone to view or edit the Pub without an account by clicking on the “sharing button” and copying the view or edit share URL with your collaborator.

Step 4: Add A Review Connection To The Preprint or Content You’re Reviewing

At any point during the drafting process, you can add a Connection to the preprint you’re reviewing. To do this, navigate to Pub Settings on the right of your Pub header, then visit the Connections tab of the Pub Dashboard. From here, paste in a link or DOI to the preprint. For most preprint servers, PubPub will automatically pull in the title, authors, publication date, and abstract of the preprint, though you will be given the option to edit them. Once you’re ready, choose the type of connection you’d like to make and add the connection.

Once you do, the preprint preview will automatically display at the top of your Pub. Here is one example from RR:C19.

Step 5: Finalize and Publish Your Review

A Tour of the Pub Header

If you’d like, you can also change the look and feel of your Pub and add header and preview images by clicking the edit theme button.

When you’re ready to publish, simply click the publish button in the Pub header. You’ll be prompted to add an optional text describing the release. If you’ve added contributors who don’t have the permissions to publish directly, they’ll be able to click request publication. Pubs that are pending your review and publication will show up in the Reviews tab in the Community dashboard.

Step 6: Deposit Your Review

How to assign a DOI on PubPub

Once the review is published, it’s time to deposit it so that others can cite and find your review. To deposit it, visit the Pub Settings tab, scroll down to the DOI section, and click Preview Deposit. This will show you a preview of how PubPub will deposit your Review, and give you the ability to select the type of review and, if applicable, recommendation. You’ll also be able to see a full preview of the XML we will deposit with Crossref. When you’re ready to submit, click submit.

The deposit preview settings

Note that it may take a while for Crossref to process the deposit on submission, and errors can occur that we are unable to detect. In most cases, deposits are processed within an hour. If your DOI is not functional after 24 hours, please contact us at [email protected].

Advanced: Connecting Review Summaries and Review Reports

PubPub supports lots of different preprint review processes. You can choose to publish only a review summary, only review reports, or any combination. If you want to publish a bundle with a review summary and individual review reports, we recommend creating the following Connections, and depositing both the summary and the reports:

Review Summary

  • Review of connection to the preprint

(Here is one example from RR:C19)

Review Reports

  • Review of connection to the preprint

  • Supplement to connection to the review summary

(Here is one example from RR:C19)

Note that you must deposit the review summary before connecting and depositing the review reports. Otherwise, Crossref will complain that the review reports link to a summary that doesn’t exist yet. If you make a mistake, don’t worry. You can always re-submit the deposit.

In the PubPub UI, these connections will create cards at the bottom of the review summary that link to the individual review reports, and cards at the top of the individual review reports that link to both the preprint and the review summary. You can change the order of appearance of these cards by dragging them in the Pub’s connection settings.

In Crossref, this will create review relations between the review summary and review reports and the preprint. It will create supplement relations between the review summary and the review reports. These relations will allow others to discover your reviews and choose whether to use the summary or the individual reports in their work.

Advanced: Categories

Some communities may want to divide their reviews into topic areas, types of reviews, or other categories. PubPub supports this via a feature called Collections, which allows you to group Pubs, customize the display of those Pubs, and invite collaborators with specific permissions for those Pubs.

To create a Collection, click the Create Collection button in the Community Dashboard. Most communities will likely want to create tag collections. Once you’ve created your collection, you can add Pubs to it from the main Collection dashboard, and invite Members from the Members tab in the same way you would a Pub. In this way, you could give a topic editor, for example, the ability to publish Reviews in their topic without giving them full access to your Community. You can also edit how Pubs will show up to visits to the collection via the Layout tab, which offers similar customization to Pages.

By default, the Collection will be private and restricted — meaning it will only be visible to Community or Collection Members, and only Community or Collection Members will be able to tag their Pubs with that collection. If you want to make a Collection public, allowing anyone to see it, you can make it Public from the Collection dashboard.

Advanced: Submissions

We’re working hard on developing a more fleshed-out submissions system. But via a combination of Collections and Layouts, you can create a fairly robust submissions process if you intend to accept submissions of any kind — whether it’s for suggesting preprints to review, or submitting reviews themselves.

The first step is to create a new private, restricted Submissions collection. Doing this will allow you to more easily group submissions that come in. If you have particular collaborators reviewing submissions, you should add them as Members to that collection.

Next, you’ll want to create at least one submission button on one of your Pages or Collection Layouts. To do this, create a Banner block on the Page or Layout with the Submission Button Banner template. In the Default Pub Collections field, add the Submissions collection. This will make it so that any Pub created via that button will automatically be added to the Submissions collection you created. If your community has multiple categories, you can create submission buttons for each one — either on a dedicated submissions Page, or on individual Collection Layouts. Wherever you create it, we recommend providing submission instructions in a text block above the button.

When users create submissions and submit them, they’ll appear in the Reviews tab in the Submissions collection.

Next Steps

Once your Community is up and running, you’ll want to promote it in spaces relevant to you. Often times, asking the authors of reviews to promote them to their departments and post it to their social media profiles is the best way to get started. Additionally, creating a social media profiles (particularly Twitter) for your community and posting all of your reviews, along with the link to the reviewed preprints, will help readers discover your community. If you tag @pubpub in your tweet, we’ll often retweet and help amplify your work.

The ecosystem around preprint reviews is still developing, but PubPub has established relationships with some preprint servers and aggregators that can automatically ingest and display reviews from PubPub on preprint pages. Inclusion in these services is completely dependent on whether your community meets their criteria for inclusion, and that makes sense. It can take time for your community to develop a reputation for trustworthiness and quality. If you feel like your work meets the criteria to be included on preprint servers and aggregators, send us a note at [email protected], and we’ll be glad to help make relevant connections.

Ideas, Questions, Need Help? Let Us Know!

Finally, PubPub itself is still developing tooling and relationships to better support preprint review and make publishing reviews even more streamlined. Right now, we’re particularly focused on improving our invitation, notification, submission, and review management systems. We’d love to co-develop those features closely with preprint review communities.

So, if you need help getting started, have ideas for how we can improve the platform, or are interested in fundraising for your review community, please send a note to [email protected].

David Reinstein:

the key step?