How to write a good revision for CHI (and rebut some reviewer requests)

1. Read the paper reviews, then sleep on them

After receiving your reviews from the CHI conference, be sure to read the paper reviews first. This will help you identify any issues that need to be addressed in your revision. Then, please spend some time sleeping on them so that you can come up with a better version of your work.

Don’t vent about your reviewers on social media. Not only is it unprofessional, but it also reflects poorly on you and your work. You’re not alone in feeling frustrated or misunderstood after reading reviews. This is part of academic life, where we find constant rejection and criticism. Reviews are never completely objective or fair. That’s an illusion. Sometimes reviewers are lazy, mean, or wrong. The reviews might not be false, but you might need time and distance to appreciate the feedback. Don’t post if you’re an author, because it will likely backfire.

Don’t complain to the technical program chair (TPC). It’s rare for complaining to the Program Chair to change the outcome of a decision regarding your paper. It is normal to feel this way about reviewers, but try to remain objective and keep in mind the workload of the program chair. If you think a reviewer has not followed the proper process (like not enough reviews) or has been excessively unprofessional, you can contact the TPC. If you have an emergency that will impact your ability to write a rebuttal promptly, you can also contact the Program Chair.

2. Determine whether to revise the paper

3. Save the reviews

Copy and paste the original reviews into a document to save a record of them. Paste all the review text into your document, summarizing the originality, significance, rigour, and recommendation ratings at the top (possibly in a table). Make it clear which parts of the text are from each reviewer by dividing up the sections accordingly (e.g., [1AC], [2AC], [R3]). Highlight any critical parts of the reviews and add your own comments as needed (without modifying the text of the reviews). Essential elements to highlight are anything reviewers ask for clarification and positive and negative comments.

4. Extract questions, criticism, and suggestions into a separate document

Reviewer requests for changes, questions, and criticisms can be helpful when revising a paper for CHI. Compile all of these requests into a separate document so you can more easily track which issues have been addressed and which need attention. This will help you track what reviewers thought about your work and how their feedback has helped improve the final product.

Use colours to highlight meaningful sentences in the review. Keep the colours subtle and light, so they don’t distract from the text. This will help you quickly identify what the reviewer is saying and what they are asking for. It will help you to focus on essential points when you are revising your paper and make it easier to spot where you need to make changes.

Sharing this document with your research group or author team can help you get feedback on the extracted chunks of the review. This can be a valuable way to identify any confusion or disagreement and develop a plan for addressing reviewer requests.

5. Summarize, group, and synthesize reviewer issues

For each review, distill the key points into bullet points that would fit on a post-it note. This will help you to quickly identify and address the main issues raised by each reviewer. Reviewers often disagree on what constitutes a strength or weakness, so it is helpful to distill each point down to its essence. This will make it easier to understand when an issue is simply a matter of opinion versus when it points out something that needs to be addressed.

Tag each summary with the committee member [1AC], [2AC], [R3], [R4], etc., to specify its origin. Pay special attention to the points the 1AC lists in the meta-review, as these will likely be the most critical issues for your revision.

One of the best ways to synthesize reviewer feedback is by creating an affinity map. This is a technique where you take all the input and group it into groups or categories that each capture a specific idea. This can help you see patterns and identify areas where reviewers had similar concerns. Once you have your groups, you can start brainstorming ways to address each. This takes you from having a list of loosely connected points to a smaller list of clusters that all capture concrete, specific ideas. To make your sets most effective, try to group together reviewer requests that suggest a single solution (i.e., one solution to address different concerns).

In the end, these clusters are the to-do list for your paper revision. Next up, you want to order that to-do list by priority. We will use severity rankings (which you might know from usability reports).

6. Rank review issue clusters by severity

Add a number ranking for each reviewer comment, with the most severe issues ranked first. This will help you prioritize which areas to focus on in your revision. I would rank along 4 severity levels: Major, Important, Regular, and Minor.

Major: If the comment relates directly to an issue identified in the 1AC meta-review. If it does, then rank it as major.

Important: You should rank an issue as important if it was addressed in depth by the 2AC review.

Regular: If a reviewer brings up an issue in their comments, but it doesn’t seem like the 1AC addressed it in their meta-reviews, rank it as a regular issue.

Minor: You should rank a revision issue as minor if the comment concerns typos, table or figure fixes that are quickly addressed, or if the statement seems like a minor aside about something that could have been done differently.

Sort your review cluster to-do list from major to minor. Then brainstorm which issues you need to address and which you can rebut. Another alternative tagging/decision-making framework that can help you decide what changes actually to address is putting these issues (you could use post-it notes for each cluster) into a two-dimensional grid along the dimensions of importance (‘not’ to ‘extremely’) and revision difficulty (‘easy’ to ‘difficult’). Things that are not easy to revise should be rebutted. Although, for items that are important but not easy to revise, you have to make a thoughtful decision about how much this would improve the paper. Maybe it can be done in the time available. I prefer using the tagging above with four different severity levels, though.

7. Brainstorm review responses and make revisions

Now that you have a sorted list of the changes that need to be made to your paper begin working on them from the most important ones at the top of the list. For this, I would create a new separate document (and of course, a separate copy of your paper where you highlight the addressed changes). This new document contains your final responses to how you either rebutted or addressed the changes in your paper (a short statement clarifying how you addressed each review cluster).

Every time you address a change from the list to your satisfaction, write a brief paragraph in this document pointing out where in the paper you made the change. This will help you keep track of your progress and make sure that all of the most crucial changes are addressed. This document will also help you remember what was said about your paper and will make it easier for reviewers to understand why you made the changes that you did.

I would do a top-bottom-alternation strategy here. So, begin a revision from the top point of your list. Once you are done revising your paper to address that, take a break from revisions by writing a rebuttal paragraph or creating a fast fix to a minor issue before you tackle the next important point back on top of your list.

8. Refine and format your revision document

The main goal of this document is to keep the 1AC and 2AC content. You want to show the committee that you are making the right changes. It is essential to be reasonable and competent in your revision document. By doing this, you will be able to demonstrate that you are willing to work with the reviewer’s requests and make the necessary changes. By taking the time to respond to reviewers’ comments and make changes based on their feedback, you can ensure that your paper is as strong as possible. Addressing reviewer requests can also help build goodwill and improve relationships with potential future reviewers.

Make sure that the themes requested by the 1AC or 2AC form the majority of your response. This will show that you’ve taken their feedback seriously and addressed their concerns. Address every reviewer request cluster directly. Explain how you implemented each revision in your document. Be sure to include specific examples and details in your response. Revisions can be time-consuming, so plan ahead and give yourself plenty of time to make the changes requested by reviewers.