Reviews as OER, a lesson plan

Jennifer Guillano and Roopika Risam:

Reviews in Digital Humanities has been an ongoing experiment in building the capacity of digital humanists to offer peer reviews that blend humanistic and technical inquiry. This past fall, we experimented with scaling the work of capacity building by partnering with a class of graduate students and their instructor, Tanya Clement, in the Department of English at University of Texas.

To help Clement, we provided access to our submissions spreadsheet that tracks projects that have been nominated for review. With her students, she selected projects from that list and augmented them with other projects that students were interested in reviewing. We then solicited project overviews from project directors so by the time classes started for fall semester, students had access to raw materials. At the outset of their sequence of assignments related to Reviews, we met virtually with the students. We explained the motivation behind our experiment, reviewed our workflow, and answered their questions about what reviewing does in the discipline. Clement, as their instructor, then took over by teaching them how to review.

Generally, the process for using Reviews in a course is as follows:

• Three months prior to the start of the course: contact Reviews to propose your student issue. Please provide us with a description of the course, including the estimated number of students. If you wish to do a thematic issue, please share the theme you wish your students to pursue. 

• We will then schedule a call with you to identify potential projects from the open submissions we’ve received as well as generate a list of projects to invite, if necessary. Usually, we’ll ask for at least 4-6 projects more than the number of total reviews you intend to publish. This allows us to account for projects that are not ready to review or cannot participate. 

• 60 days prior to the start of course: We will send invitations to project directors to solicit overviews. Most overviews are returned in 45 days.

• 30 days prior to the start of the course: Schedule class meetings where Reviews editors will participate.

• Semester work: Students work with their instructor based on the timeline agreed upon in the course syllabus. Reviews editors can participate in revising project reviews, as needed. Once reviews are finalized, the instructor writes an editorial note and shares all final reviews and the guest editor note to Reviews for publication.

Tanya Clement:

For 10 years, I've been teaching a graduate-level Introduction to Digital Humanities course at the University of Texas, first in the School of Information and more recently in the Department of English. The class has historically attracted students from the School of Information and departments across the College of Liberal Arts, including African and African Diaspora Studies, American Studies, Classics, English, Geography, History, and Spanish and Portuguese. My goal in teaching the course was to empower students to engage in digital humanities research and scholarship based on their own interests and academic contexts. To that end, I wanted students to leave the course with a better understanding of digital humanities research goals, methodologies, and scholarship venues, including conferences, book series, journals, and projects across disciplines. I wanted them to know enough to either move ahead with a rigorous digital humanities agenda in their own research or to walk away with a better sense of digital humanities knowledge production as complementary to their own fields of study. The goal was for my students to develop proficiency in evaluating digital projects. I wanted them to understand that digital humanities is not just about using digital tools in the humanities. Rather, as Reviews shows, thinking critically about the role that digital methodologies play in knowledge production is at the heart of digital humanities research and scholarship.

Writing a review for the journal was a natural project for the course as it aligned with what I wanted to teach students about thinking critically about and within digital humanities. Finally, in Reviews, the students have a public platform on which to publish their work in a safe and guided environment facilitated by me but more so by editors Guiliano and Risam. Managing the students’ review-writing process was straightforward. I worked with Guiliano and Risam on identifying ten projects we thought students would be interested in reviewing.

The prompt the students received in the syllabus included a learning outcome and description as follows:

Goal: Students will demonstrate their understanding of Digital Humanities research questions, methods, and projects (including but not limited to digital archives, multimedia or multimodal scholarship, digital exhibits, visualizations, digital games, digital tools, and digital projects) as a blend of humanistic and technological inquiry.

Description:

This assignment is a collaboratively-written review of a DH project. Working with the instructor and the editors, students will write a public-facing review that is designed to be published in the journal Reviews in Digital Humanities. Two students will work together to write two reviews, which will be turned into the editors for evaluation and returned for final edits before the end of the semester. Reviews are intended to be published on the journal website in the beginning of the spring semester. 

1. Sign up: Each pair of students will write two project reviews. You can split them up or do them together, but sign up to review two projects. (Note: each project should only have two pairs or reviewers; some projects will only have one) 

2. Submission: Each student will upload one distinct review (even if you write them together, upload one review per person; please put both names on each review you have written together). The submission will follow the journal guidelines at https://reviewsindh.pubpub.org/review-process.

3. Peer review: Each student pair will be assigned two DH reviews to review the morning after you submit your own DH review. Your assigned DH review to review will be accessible as well. These peer reviews are due a week later in class. As a team, you can decide to split these up, and each person can review one, or review them together but each student should read both and be able to discuss both in our class workshop. Your review should include 200 words of critique (please just don't say, "this is great," etc. but help your classmate write a better review!) per review. You will post your peer review HERE where you got your assigned reviews. Criteria for review should be based on the parameters described on the journal guidelines at https://reviewsindh.pubpub.org/review-process.

Using this prompt, in teams of two, my students read previously published reviews from the journal, and we discussed them and the review criteria in class. Students chose projects they wanted to review and, over the course of six weeks, we wrote drafts together, shared drafts with each other, and wrote and shared peer reviews in small groups. I gave them feedback (and took some for my own review) and offered them the opportunity to make changes needed for publication. Each student in the course chose to work toward publication. 

Below is an example rubric that can be used to evaluate the students’ work:

As a result of this assignment, the students completed the following reviews:

Part 1 (February 2021):

• Louisiana Slave Conspiracies, reviewed by Keerti Arora and Anna Lawrence;

• On the Books: Jim Crow and Algorithms of Resistance, reviewed by Ann Marie Blackmon and Caroline Collins; 

• Slave Revolt in Jamaica, 1760-1761, reviewed by Ethan Warren;

• Relaciones Geográficas, reviewed by Anna Lawrence and Keerti Arora; and

• The Anti-Eviction Mapping Project, reviewed by Tanya Clement.

Part 2 (March 2021):

• Media History Digital Library, reviewed by Nina Gary and Becky Yatsuknenko;

• SongData, reviewed by Carl Teegerstrom and Kayleigh Voss;

• Furious Flower Digital Archive, reviewed by Gabrielle Roth; and

• Tribesourcing Southwest Film Project, reviewed by Carl Teegerstrom.

Part 3 (April 2021):

• The Collective Biographies of Women, reviewed by Ali Gunnells; 

• Mapping the Gay Guides, reviewed by Katherin Tairo-Quispe;

• The Lesbian and Gay Liberation in Canada Prosopography Project, reviewed by Becky Yatsuknenko and Nina Gary; 

• Readux, reviewed by James Lacitignola; and

• Distant Viewing Toolkit, reviewed by Patrick Sui.

For those interested in developing a Reviews in the Classroom, our general process is:

• Three months prior to the start of the course: contact Reviews to propose your student issue. Please provide us with a description of the course, including the estimated number of students. If you wish to do a thematic issue, please share the theme you wish your students to pursue. 

• We will then schedule a call with you to identify potential projects from the open submissions we’ve received as well as generate a list of projects to invite, if necessary. Usually, we’ll ask for at least 4-6 projects more than the number of total reviews you intend to publish. This allows us to account for projects that are not ready to review or cannot participate. 

• 60 days prior to the start of course: We will send invitations to project directors to solicit overviews. Most overviews are returned in 45 days.

• 30 days prior to the start of the course: Schedule class meetings where Reviews editors will participate.

• Semester work: Students work with their instructor based on the timeline agreed upon in the course syllabus. Reviews editors can participate in revising project reviews, as needed. Once reviews are finalized, the instructor writes an editorial note and shares all final reviews and the guest editor note to Reviews for publication.