At the purest level of my psyche, I know that peer review comments lead me to better writing and better (humanities) scholarship. At the more pragmatic level, I know that giving reviews what they want gets the article into the journal and into my soon-due tenure packet. At the real, gnawing-at-my-gut level, every single less-that-positive comment in a review means I'm a lazy, stupid, and self-deluded fraud.
Yeah, OK. I'm printing out the comments, reading through them careful with my feet up, a cup of coffee in hand, and Abul Mogard playing at low volume. Then, I'm getting to work.
(And hi, y'all.)
Totally different field, but here's my process in case there is anything here that might help you get through this necessary process with less pain. Good luck and keep at it!
This is for an R & R. For outright rejections where I have to revise for another journal I'm much more selective in what reviewer comments I incorporate.
1. Skim through them once as soon as I get the reviews. Allow myself to feel whatever I'm feeling.
2. Put them away for at least a day or two.
3. Copy them into a Word file. Break up and number the comments for those annoying reviewers who don't already submit their reviews in bullet point format.
3. Go through reading carefully, and write notes to myself/my co-authors under each reviewer comment about how we can address it. These are often quite snarky and definitely not what will end up in the final version. I like to use red font for this.
4. Put it way again for at least a day or two.
5. Go back and turn the red notes into draft cover letter responses*. Here is where I start really thinking through how I will revise the text to address it, and how to word this to show the reviewer we're being responsive to their comment. The snarkiness goes at this point. This text is in blue until I've actually made the changes in the paper.
6. Make the changes in the paper itself.
7. Read through the cover letter responses one more time to make sure you have really addressed each one clearly in both places, that these match, and to check tone for any remaining defensiveness/dismissiveness.
8. Resubmit!
* In the sciences, revisions often require extensive additional statistical analyses or even more data collection. In that case, this step is delayed considerably until that is done so we know what the response is.
I recently got a grant review back in which the reviewers crushed the proposal and my academic track record. That was for $10,000. A week later, I got all glowing reviews on a $50,000 grant that was related to the same project and reviewers highlighted my academic track record. Reviews can be pretty random and accepting the randomness for what it is makes life much less stressful.
Quote from: Puget on June 27, 2019, 08:49:39 AM
5. Go back and turn the red notes into draft cover letter responses. Here is where I start really thinking through how I will revise the text to address it, and how to word this to show the reviewer we're being responsive to their comment. The snarkiness goes at this point. This text is in blue until I've actually made the changes in the paper.
6. Make the changes in the paper itself.
This is the better order. Thanks for making that point.
Most people do the reverse, but it is more efficient to make the changes if you know that the resulting rationale is going to be persuasive.
A huge part of reviewers (at least, in my discipline) is giving an outside perspective, and helping you understand how an audience will react. I would think this is especially true in Humanities disciplines, where you might even be writing a paper without coauthors (something I've not done). I've been an author of thirty-ish papers, and only one was ever accepted without revisions (I wasn't remotely the first author, and deserve little credit for that) - it's just too easy to overlook a bit of the literature, or gloss over something your audience will want to see, or think something is obvious when it's really only obvious to you who've been working on this project for a year or more.
I have a decently thick skin against paper reviews by now, but some grant reviews still hurt. And yes, they are very random in terms of who does the reviewing - their background, their agenda (e.g. past mentees applying for the same award, feud with your past advisor), and other factors. Still, it's hard to not let it get to you sometimes.
I am at the point where I don't get mad anymore when the reviewers are idiots or have an agenda that they are acting out. I just look at what stupid hoops I have to jump through to get the prize, and continue. The more I do this the more I don't care - it's really all just a game.
I also just take whatever comments make sense, do those, and then tell the editor why I didn't do the others. The goal is to get the thing published - not so that anyone necessarily reads it, but to properly inflate the cv so as to possibly get to full. Going up next year. I'm a proper cynic, I am.
An approach: My colleagues, in a discipline where the reviewers are unnecessarily harsh, read and summarize comments for each other. Once the author has come to terms with the summary version, then the originals are read. Sometimes, though, the colleague will redact especially noisome comments from the originals.
1. I read the reviewers' comments carefully and calmly.
2. I write long angry, rebuttals in the margins to each asinine comment.
3. I fashion voodoo dolls, stick pins in various particularly painful locations, then burn them.
4. I revise the article according to the reviewers' suggestions, insofar as I can while retaining a shred of integrity.
5. I resubmit.
At the beginning of my career I also used to take reviewer's criticism pretty defensively. Now with plenty of publishing and reviewing experiencing I am way more dispassionate about it.
Young authors should keep in mind that while you have to address (as in "reply to") all reviewers' comments, you don't have to implement everything they say to get your paper accepted.
Quote from: youllneverwalkalone on June 28, 2019, 01:47:47 AM
Young authors should keep in mind that while you have to address (as in "reply to") all reviewers' comments, you don't have to implement everything they say to get your paper accepted.
Yes, this is important. A huge difference exists between "yes, that's a gap in explaining what I did and why" and "no, that's a completely different study and out of scope of what we're doing here".
One benefit of having several reviewers is being able to see the pattern of similar questions from everyone, which probably do need to be addressed, and that doggone Reviewer 2 who wants a completely different paper that only exists in their mind.
Perhaps useful is "Don't Be Reviewer 2: Improving the Academic Journal Process" (https://www.youtube.com/watch?v=oUqHSgVTEN0).
Thanks, everyone. As it turns out, reviewer two makes pointed and valuable comments but, in a few instances, claims I did not consider specific pieces of scholarship that I used, cited, and have in the bibliography. The biggest problem is that adding everything reviewer two wants will expand the article by about 25%, while reviewer two, who is in the main much more positive, wants the article to be shorter. The extremely helpful editor did warn me that I should negotiate between the two as best I can, and I'm starting to feel good about it as an intellectual puzzle.
Quote from: glowdart on June 27, 2019, 07:19:42 PM
An approach: My colleagues, in a discipline where the reviewers are unnecessarily harsh, read and summarize comments for each other. Once the author has come to terms with the summary version, then the originals are read. Sometimes, though, the colleague will redact especially noisome comments from the originals.
I have done this for folks and recommend it highly, especially for writers who can't help but take the barbs seriously.
I just submitted a manuscript to a journal that has an "interactive review forum". The idea is that you can go back and forth with the reviewers. This will require a change in the usual process of thinking about and responding to their comments. That process is the one Aside and Puget describe above.
Any suggestions on preparing for this interactive forum?
Quotein a few instances, claims I did not consider specific pieces of scholarship that I used, cited, and have in the bibliography.
That's super-easy. You just respond saying "We added a consideration of those specific pieces of scholarship at places X, Y, and Z", and allow the reviewer to (wrongly) assume you're implying that happened chronologically after they wrote their review. ;)
Above all, I always to respond with something that sounds like "I totally agree with the reviewer, and have remedied the situation", even when that's not really an accurate description of the situation.
Quote from: Hibush on June 28, 2019, 12:28:58 PM
I just submitted a manuscript to a journal that has an "interactive review forum". The idea is that you can go back and forth with the reviewers. This will require a change in the usual process of thinking about and responding to their comments. That process is the one Aside and Puget describe above.
Any suggestions on preparing for this interactive forum?
I've been on both the author and reviewer sides of this system a number of times now, and aside from the rounds of review generally going faster and not having to wait on the editor (which is nice), it's really not any different than the usual process in practice. The original idea may have been for it to be more informal/interactive but people are comfortable with the old model and basically have replicated that in my experience-- i.e., you write the same sort of cover letter response you usually would and paste it into the forum. It does give you a chance to ask for clarification on a reviewer point if necessary before making manuscript revisions, which can be helpful. Otherwise same old same old.
Quote from: Puget on June 28, 2019, 02:07:46 PM
Quote from: Hibush on June 28, 2019, 12:28:58 PM
I just submitted a manuscript to a journal that has an "interactive review forum". The idea is that you can go back and forth with the reviewers. This will require a change in the usual process of thinking about and responding to their comments. That process is the one Aside and Puget describe above.
Any suggestions on preparing for this interactive forum?
I've been on both the author and reviewer sides of this system a number of times now, and aside from the rounds of review generally going faster and not having to wait on the editor (which is nice), it's really not any different than the usual process in practice. The original idea may have been for it to be more informal/interactive but people are comfortable with the old model and basically have replicated that in my experience-- i.e., you write the same sort of cover letter response you usually would and paste it into the forum. It does give you a chance to ask for clarification on a reviewer point if necessary before making manuscript revisions, which can be helpful. Otherwise same old same old.
Thanks for the answer from experience.
I'm glad to hear that a lot of the old paradigms are still in place. I'm still going to have to think differently about--that idiot--reviewer #2. Perhaps the chance to get clarification will change that dynamic somewhat.
Another thought--you don't read your critical reviews--delete them unread.
They will annoy you and in my situation, many comments are useless, nit picky. Every reviewer has their own biases positive and negative but we do not know what they are. I suspect that if you and I knew this, we would realize that we cannot take some comments seriously. Some critics in my field look for any excuse or reason to criticize and rewrite. Example.If the author you are discussing says that all human actions in the world are self centered and you say this includes Chinese, the critic could state that the author did not mention the Chinese. This is a petty dumb remark. I get a lot of these.
Quote from: Myword on June 29, 2019, 08:49:11 AM
Another thought--you don't read your critical reviews--delete them unread.
They will annoy you and in my situation, many comments are useless, nit picky. Every reviewer has their own biases positive and negative but we do not know what they are. I suspect that if you and I knew this, we would realize that we cannot take some comments seriously. Some critics in my field look for any excuse or reason to criticize and rewrite. Example.If the author you are discussing says that all human actions in the world are self centered and you say this includes Chinese, the critic could state that the author did not mention the Chinese. This is a petty dumb remark. I get a lot of these.
Huh? I think you may have misunderstood the topic here-- this is not about reviews someone publishes of your book, it's about peer review. If you get an R&R on a journal article you can't delete reviewer comments unread-- you need to address each one in your cover letter and revision.
I feel you. My first article ever was rejected by the first journal I have submitted to. I am actively working on addressing the comments and resubmit it to another journal. I would need feedback prior to submitting, however, I feel like everyone I know including other graduate students, supervisors and professors are busy...However, I know having feedback would help me to get published. How typical is it to ask people to read our work-in-progress research paper for free?
Quote from: adel9216 on June 29, 2019, 07:55:39 PM
How typical is it to ask people to read our work-in-progress research paper for free?
In my experience, it is not uncommon. I am often asked to read my colleagues' work before submission. I've never expected to be paid, though some folks will take you to lunch or give a small gift. Most have just offered to read something of mine in return.
Quote from: adel9216 on June 29, 2019, 07:55:39 PM
I feel you. My first article ever was rejected by the first journal I have submitted to. I am actively working on addressing the comments and resubmit it to another journal. I would need feedback prior to submitting, however, I feel like everyone I know including other graduate students, supervisors and professors are busy...However, I know having feedback would help me to get published. How typical is it to ask people to read our work-in-progress research paper for free?
It is very common in my world. I still exchange paper drafts with grad school friends, and then I have a professional writing circle that does draft exchanges too. Do you have people you can build a circle with or otherwise just ask?
Quote from: fast_and_bulbous on June 27, 2019, 12:59:45 PM
I am at the point where I don't get mad anymore when the reviewers are idiots or have an agenda that they are acting out. I just look at what stupid hoops I have to jump through to get the prize, and continue. The more I do this the more I don't care - it's really all just a game.
100%
In all my years of research... over 100 papers, 7 books... I think I got one all-positive review. You just let it go and move forward. Make the changes they want to see then move on. Don't take it personally.
Quote from: Myword on June 29, 2019, 08:49:11 AM
Another thought--you don't read your critical reviews--delete them unread.
They will annoy you and in my situation, many comments are useless, nit picky. Every reviewer has their own biases positive and negative but we do not know what they are. I suspect that if you and I knew this, we would realize that we cannot take some comments seriously. Some critics in my field look for any excuse or reason to criticize and rewrite. Example.If the author you are discussing says that all human actions in the world are self centered and you say this includes Chinese, the critic could state that the author did not mention the Chinese. This is a petty dumb remark. I get a lot of these.
I have one pretty gruesome review on a grant proposal. (Two reviews came back 'good'--about right, I think: it was mediocre for various reasons, including time constraints--but one was a straight-out 'poor'.) I still can't bear to read that 'poor' review, and I think I may never get around to it.
Quote from: fast_and_bulbous on June 27, 2019, 12:59:45 PM
I am at the point where I don't get mad anymore when the reviewers are idiots or have an agenda that they are acting out. I just look at what stupid hoops I have to jump through to get the prize, and continue. The more I do this the more I don't care - it's really all just a game.
It's (probably) not personal. It's (probably) just what they think/know/wish/ate last might. Take it with a grain (or a truckload) of salt and do what you can to get the article published.
It's just part of the job. I have received scathing reviews and I have written them. It's just part of our scientific (and sometimes not so scientific) process for furthering knowledge.
Quote from: Hibush on June 28, 2019, 12:28:58 PM
I just submitted a manuscript to a journal that has an "interactive review forum". The idea is that you can go back and forth with the reviewers. This will require a change in the usual process of thinking about and responding to their comments. That process is the one Aside and Puget describe above.
Any suggestions on preparing for this interactive forum?
Assuming you are talking about Frontiers? I have been using their system pretty much the old fashioned way, like preparing reply to each reviewers' point and pasting it in there. But basically I have waited until I had done all the revisions and all the replies to submit anything so I did not really take advantage of the potential "interactivity".
Quote from: youllneverwalkalone on July 03, 2019, 05:58:49 AM
Quote from: Hibush on June 28, 2019, 12:28:58 PM
I just submitted a manuscript to a journal that has an "interactive review forum". The idea is that you can go back and forth with the reviewers. This will require a change in the usual process of thinking about and responding to their comments. That process is the one Aside and Puget describe above.
Any suggestions on preparing for this interactive forum?
Assuming you are talking about Frontiers? I have been using their system pretty much the old fashioned way, like preparing reply to each reviewers' point and pasting it in there. But basically I have waited until I had done all the revisions and all the replies to submit anything so I did not really take advantage of the potential "interactivity".
Can you tell from the reviewers comments whether they are expecting interactivity? Or are they behaving like "old-style" reviewers? This format means that htey could ask an open-ended question rather than saying that a particular statement was wrong.
Quote from: Hibush on July 03, 2019, 12:29:05 PM
Quote from: youllneverwalkalone on July 03, 2019, 05:58:49 AM
Quote from: Hibush on June 28, 2019, 12:28:58 PM
I just submitted a manuscript to a journal that has an "interactive review forum". The idea is that you can go back and forth with the reviewers. This will require a change in the usual process of thinking about and responding to their comments. That process is the one Aside and Puget describe above.
Any suggestions on preparing for this interactive forum?
Assuming you are talking about Frontiers? I have been using their system pretty much the old fashioned way, like preparing reply to each reviewers' point and pasting it in there. But basically I have waited until I had done all the revisions and all the replies to submit anything so I did not really take advantage of the potential "interactivity".
Can you tell from the reviewers comments whether they are expecting interactivity? Or are they behaving like "old-style" reviewers? This format means that htey could ask an open-ended question rather than saying that a particular statement was wrong.
From the tenor of the comments they pretty much look like "old-style" in my experience, which I guess makes sense seeing that most reviewers (and most journals) use the traditional peer-review model.
In my understanding the interactivity from the author part means you could in theory submit separate replies as you go along, rather than waiting to have all your replies and submitting them all at once (which is what I did). On the other hand I could see their replies in real time rather than waiting for both of them to finish, and for the editor to send me a decision letter. For example, when one reviewer accepts the paper ("endorse publication" in the Frontiers parlance) you can see that right away, so you can get that cool "hell yeah, one down!" feeling without having to wait for the editor to write the usual letter to say that "Dear author, after one round of review reviewer 1 is satisfied, while reviewer 2 is still being a whining little b*tch. Please find their comments below".
QuoteAt the beginning of my career I also used to take reviewer's criticism pretty defensively. Now with plenty of publishing and reviewing experiencing I am way more dispassionate about it.
I am hoping to get that thick of a skin eventually. As it is, I'm waiting on tender-hooks to hear back from a pretty good journal (impact factor > 1.5, which is higher than any other journal I've published in) that I've already sent
two previous manuscripts to. I'm basically sending them the
third draft of a manuscript based on my dissertation and trying not to cringe at what remarks may come!
I do my best to take comments as constructive criticism. Likewise, when I write reviews, I try make sure they sound more constructive than not. I have heard horror stories a la "no intellectual merit" though.
QuoteI am hoping to get that thick of a skin eventually. As it is, I'm waiting on tender-hooks to hear back from a pretty good journal (impact factor > 1.5, which is higher than any other journal I've published in) that I've already sent two previous manuscripts to. I'm basically sending them the third draft of a manuscript based on my dissertation and trying not to cringe at what remarks may come!
It's an archaic term but it's "tenterhooks".
Good luck.
Quote from: risenanew on July 24, 2019, 10:51:22 PM
QuoteAt the beginning of my career I also used to take reviewer's criticism pretty defensively. Now with plenty of publishing and reviewing experiencing I am way more dispassionate about it.
I am hoping to get that thick of a skin eventually. As it is, I'm waiting on tender-hooks to hear back from a pretty good journal (impact factor > 1.5, which is higher than any other journal I've published in) that I've already sent two previous manuscripts to. I'm basically sending them the third draft of a manuscript based on my dissertation and trying not to cringe at what remarks may come!
The best advice I can give you is to forget all about this paper and get busy with your next one(s) while you wait. My advisor told me a long time ago that you should always have at least 1 paper under review and at least 1 paper currently working on, and it's one of the best advice I ever got.
I emphasize celebrating submissions and not just acceptances to my grad students. Celebrate getting it out there and then forget about it until you get reviews back. Academic life entails a lot of rejection-- you can't publish (or get grants, or jobs, or. . .) without rejection. Consider starting a Rejection Collection: https://sarneckalab.blogspot.com/2017/12/rejection-collection.html
(Also check out her writing workshop blog and materials, they are fantastic! https://sarneckalab.blogspot.com/2018/07/writing-workshop-table-of-contents.html)
Quote from: youllneverwalkalone on July 25, 2019, 07:05:07 AM
The best advice I can give you is to forget all about this paper and get busy with your next one(s) while you wait. My advisor told me a long time ago that you should always have at least 1 paper under review and at least 1 paper currently working on, and it's one of the best advice I ever got.
Yes, keep the pipeline full! I set up a paper tracking board (in Asana, which is awesome btw, but you can use anything), with columns for: Preregistrations, Analyzing, Drafting/Revision for new journal, Under review, R&R revising, R&R under review, Accepted. (I also recently added a "???" column for middle-authored papers that seem to have disappeared into the void but may resurface some day).
I try to keep all the columns populated, and notice when a paper has been languishing in one column too long.
Moving a paper over is cause for celebration--
Quote from: Puget on July 25, 2019, 07:28:15 AM
Yes, keep the pipeline full! I set up a paper tracking board (in Asana, which is awesome btw, but you can use anything), with columns for: Preregistrations, Analyzing, Drafting/Revision for new journal, Under review, R&R revising, R&R under review, Accepted. (I also recently added a "???" column for middle-authored papers that seem to have disappeared into the void but may resurface some day).
I try to keep all the columns populated, and notice when a paper has been languishing in one column too long.
Moving a paper over is cause for celebration--
I need to do something like this to keep everything sorted. Sometimes I can't decide on what to work on at any given time (which is a problem - I have at least 5 half-finished papers languishing because I get bored). But maybe if I can see it as progression it will feel more satisfying to move things over, kind of like crossing things off a to do list.
Puget, if I wasn't pretty sure we were in different fields, I might think you are my PhD supervisor - his method of responding to reviewers was almost identical to what you described on the first page of this thread, down to the color coding of responses. I mostly do the same and find it very helpful.
Quote from: scamp on July 25, 2019, 07:46:09 AM
Quote from: Puget on July 25, 2019, 07:28:15 AM
Yes, keep the pipeline full! I set up a paper tracking board (in Asana, which is awesome btw, but you can use anything), with columns for: Preregistrations, Analyzing, Drafting/Revision for new journal, Under review, R&R revising, R&R under review, Accepted. (I also recently added a "???" column for middle-authored papers that seem to have disappeared into the void but may resurface some day).
I try to keep all the columns populated, and notice when a paper has been languishing in one column too long.
Moving a paper over is cause for celebration--
I need to do something like this to keep everything sorted. Sometimes I can't decide on what to work on at any given time (which is a problem - I have at least 5 half-finished papers languishing because I get bored). But maybe if I can see it as progression it will feel more satisfying to move things over, kind of like crossing things off a to do list.
Puget, if I wasn't pretty sure we were in different fields, I might think you are my PhD supervisor - his method of responding to reviewers was almost identical to what you described on the first page of this thread, down to the color coding of responses. I mostly do the same and find it very helpful.
Well, not a he so pretty sure you're not my student ;) -- Some things just work and spread.
It sounds like you need a term plan and a weekly writing plan -- I refer you once more to Barbara Sarnecka's awesome writing workshop materials linked in my last post, which explains how they work and has links to google sheet templates. They really have revolutionized how we work on my lab--it has been super helpful for both me and my students. Members of two other labs now participate in our writing group. If you're not in a lab field, consider starting a department group.
One huge upside in addition to increased productivity is I feel much less guilty about everything I
should be doing, because I either do everything in the plan for that week, or recognize that it was unrealistic and re-plan so I can still get it done that term. Once I've done everything on that week's plan I can relax without feeling guilty. I also don't have to spend any mental energy figuring out what to prioritize that day/week, because I do it once at the beginning of the term (with some adjustments as needed) and week then just follow the plan.