question about being a reviewer for a journal for the 1st time

[adel9216]

Hello!

I have an opportunity to review an article for an academic journal. I have never been a reviewer before...is it worth it for me to give my time to this? Does it add value to my CV? (it's also unpaid work).

[Ancient Fellow]

If it's a good journal, then, yes, it's absolutely worth the investment of time to add evidence to your claim of being a recognized expert. The exception would be if you're struggling to finish the monograph or journal article you need for tenure, which should take priority over everything else at work.

[Ancient Fellow]

...or, looking at a few of your other posts, if it looks like it will be an obstacle to the thesis chapter you're struggling to finish, then do reconsider.

[ergative]

It's useful for your own publishing to see what it looks like from the other side. If you've never reviewed a paper before, then it's a good idea to do it.

Regarding the unpaid work: Yes, it is, but then every time you submit a paper you generate unpaid work for your colleagues. One general guideline I've seen is that you should agree to review papers in proportion to the number of reviewers you require. So if you submit two papers that get three reviews each, then you should say yes to six review requests. I myself have never been asked to review that many, but it makes it a lot easier for me to say yes when I think about it that way.

(I should add that I'm in a sub-field of my discipline where the reviewers I get tend to sign their names, and I'm quite friendly with some of them, and have starting signing my own reviews in response, so it's usually a collegial process all around. But we can't all be that lucky.)

[science.expat]

It's useful for your own publishing to see what it looks like from the other side. If you've never reviewed a paper before, then it's a good idea to do it.

Regarding the unpaid work: Yes, it is, but then every time you submit a paper you generate unpaid work for your colleagues. One general guideline I've seen is that you should agree to review papers in proportion to the number of reviewers you require. So if you submit two papers that get three reviews each, then you should say yes to six review requests. I myself have never been asked to review that many, but it makes it a lot easier for me to say yes when I think about it that way.

(I should add that I'm in a sub-field of my discipline where the reviewers I get tend to sign their names, and I'm quite friendly with some of them, and have starting signing my own reviews in response, so it's usually a collegial process all around. But we can't all be that lucky.)

I agree with all of this. And I'm also in a field where we generally sign our reviews.

[octoprof]

Reviewing manuscripts for journals is just part of our job as scholars in most fields. Don't accept too many at a time, but always do one a year or so at a minimum. You will build a wider knowledge of what is being done, create relationships with editors/journals, and generally improve your career in ways that are hard to measure.

[polly_mer]

On the unpaid work front, the review should not take you huge amounts of time and energy.  Several hours spread over a week is reasonable.  Several hours per day for two weeks is well beyond the expectations for a paper that's only 20 pages of text and another 20 pages of figures/tables/references.  If you find you have to do too much background work yourself to understand the paper, then perhaps it's time to contact the editor to suggest a different reviewer.

If the paper needs significant work, then don't comment on every paragraph.  Instead, think hard (perhaps let it percolate in the back of your mind for a couple days) and ask a few big picture questions.  In my fields, variants of these questions are often explicitly stated in instructions to the reviewers as goals to get feedback that will help improve papers.

0) Can you read the paper and make any sense of it or should a copy editor who is fluent in the relevant language be a first stop?  Ideally, these papers would never be sent to reviewers, but I've had a handful over the past 20 years where the response was, "This is not good enough English for me to comment on the science".

1)  Is a scientific question or hypothesis clearly stated in the introduction?  Related questions in the same vein are what were the researchers trying to learn and what information were the researchers trying to gather.

2) Are the conclusions clearly stated?   The related question are did the researchers answer whatever question, successfully gather the data, or address the hypothesis.

3) Do the data support the conclusions?  Many papers I review that need work have a clear scientific question, definite conclusions, clearly stated methods, and just a pile of random figures/charts/tables/numbers that do not directly connect the question and the conclusions.  It's clear that someone did a lot of work, but it's not clear that the work actually answers the question or, in some cases, could possibly answer the question. 

For example, having a detailed description of every square inch under the 3rd streetlight on the left may or may not actually help anyone find the lost keys.  This is especially true if a good literature review indicates the keys were last used just before going on a camping trip with a starting point as a house on the right near the 6th streetlight.  Sure, it's possible the keys were thrown under the 3rd streetlight, but it's more probable the keys are either somewhere near the house, in the car, or near the campsite.

4) Is there a coherent narrative to the paper or does it look like some headings were put in a file, text was written, and yet bears no resemblance to the headings?  Related questions are is the paper logically organized and can the paper be shortened without detriment.

Even though I'm wordy on these fora, I tend to point out that certain tangents in a paper are either better in an appendix, a different and separate paper, or just left out.  To continue on the streetlight analogy, I will point out that if the paper reports the results of searches near the house, in the car, and in the campsite, then the 3rd streetlight on the left is irrelevant and can be left out.  This is especially true if the keys were found in the car.

5) Is the literature review appropriate to the paper as written?  Something I see frequently is an introduction that has exquisite detail on the material that is generally taught in a 300-level class and then basically ignores the actual published research relevant to understanding why this problem is still outstanding. 

For example, someone will put a couple paragraphs regarding the importance of keys.  The authors will explain in great detail the history of streetlights and why streetlights are used. Then, there's a hypothesis and statement of why a particular method was employed: light is likely helpful as we look for keys; a streetlight provides a reasonable amount of light for other applications.

However, there's no page or two of discussion on what others found when they looked in the bushes around the house and along the 3 miles of trail near the campsite.  There's no indication of why anyone expects the keys to be under this particular streetlight and why we're not either looking in the daylight in other areas or taking a high-powered flashlight.  There's no indication of why a problem that is so important remains unsolved after all these decades and sometimes the answer is already in the literature as "we changed the locks and got new keys".

[Conjugate]

I always worry when I review a paper, even one in my own narrow area of specialization.  I can go through the paper and advise on poor English word choice, punctuation, and so forth, and I can point out awkward phrasing and typos, but beyond that I'm at a loss.

Partly this is because I feel I should respond with a statement as to whether the work is appropriate for the journal (I have no idea, except, you know, the journal has that discipline's name in its title), whether the work is of interest to the mathematical community (who the hell knows? A lot of what the 'mathematical community' is interested in, to me, seems to be garbage, and I'm sure the community feels that way about lots of work that I find interesting), and about the work's "significance," which I can't define.

Maybe I should just, you know, turn down reviewing gigs?

[Parasaurolophus]

That's exciting! I wasn't asked to referee until after I got my PhD, and even then, I only started getting more than one invitation a year this year. It takes a while for editors and associate editors to notice newcomers, and it's great that someone has noticed you and reached out. That's well worth a pat on the back!

That said, I don't think you should do it. Writing a report is a lot of work, and it requires you to be pretty familiar (1) with the subfield, (2) with the journal, and (3) with publishing norms in your discipline. As a student fresh out of your master's, and just trying to publish for yourself, you don't really tick those boxes yet. Besides which, it's a lot easier to model your referee report on the good ones you've received in the past--which you don't have yet.

[Kron3007]

It's useful for your own publishing to see what it looks like from the other side. If you've never reviewed a paper before, then it's a good idea to do it.

Regarding the unpaid work: Yes, it is, but then every time you submit a paper you generate unpaid work for your colleagues. One general guideline I've seen is that you should agree to review papers in proportion to the number of reviewers you require. So if you submit two papers that get three reviews each, then you should say yes to six review requests. I myself have never been asked to review that many, but it makes it a lot easier for me to say yes when I think about it that way.

(I should add that I'm in a sub-field of my discipline where the reviewers I get tend to sign their names, and I'm quite friendly with some of them, and have starting signing my own reviews in response, so it's usually a collegial process all around. But we can't all be that lucky.)

I agree with all of this. And I'm also in a field where we generally sign our reviews.

I agree in principle, but think the math is a little dodgy in many fields.  In my field, we generally co-author, so if I were to submit 2 papers that have three reviewers, that means that among me and my co-authors, we should review at least 6.  This is really splitting hairs I know...

Back to the OP, yes you should have some reviewing activity on your CV to show that 1) you are recognized by the journal as a qualified reviewer, and 2) that you are contributing to the field.

[pigou]

I highly recommend doing it. In my field, it's common to give papers to one's graduate students to review for practice. Doesn't absolve you from doing your own review, but it's excellent practice for students to critically engage with a paper and compare their comments with yours.

You learn a lot from reading a paper for the purpose of reviewing: it forces you to think about perhaps unsubstantiated assumptions, issues with the study, etc. It's also easier to see mistakes in others' writing than in your own. I'm convinced that reviewing makes you a better writer, too. It's also easy to dismiss other research, but reviewing forces you to look for the value the paper might have -- and how potential issues can be fixed.

The thing to keep in mind that as a junior researcher, you're not going to get sent outstanding papers. You'll be reviewing for low to mid tier journals and you're going to get papers that likely will be rejected (at least in my field). This is the editor giving the authors (hopefully) useful comments and you some practice with how to review. It also shows the editor if you're the kind of reviewer they should continue to reach out to.

Submit your review on time, take it seriously, but don't invest a ridiculous number of hours on it. When I started reviewing, I'd put a full day into it. Then I'd re-read the paper and my review a week later to see if it still made sense. Now, I spend maybe 3-4 hours and rarely re-read the paper after. As with everything, you get better the more often you do it.

The format I've been taught and believe in:

1) A paragraph summarizing the topic the paper contributes to, what the main contributions are, and what kind of methods the paper uses. Imagine explaining the paper favorably to someone who hasn't read it. This helps the editor, who probably hasn't read it, and the authors: sometimes, reviewers miss the point of the paper, which suggests the writing can be improved. This is also an opportunity to be nice to the authors and give them some recognition for the effort they have put into the paper. That is, this isn't the place to be critical.

2) Major issues. This includes all the things you think prevent the paper from being published -- and here it's your job to be a critical reader. Even (and perhaps especially) if you're favorable to the argument they make, you should look for everything that is unclear or could be wrong. This could point to seriously flawed arguments/evidence, which cannot easily be fixed, or to things that can be done. E.g. in experimental research, they may have used an incorrect analysis or interpreted something incorrectly. That's fixable. Or their design might actually not identify what they claim to be testing, which probably can't be fixed.

3) Minor issues. Things that you wouldn't reject a paper over, but that you want to see changed. This could be clarifying labels on graphs, or improving the clarity of some passage. I never offer much detail here (I'm not a co-author), but something like "the labels on Figure 1 are confusing." Your job is not to proofread (the journal has editors), but to point out things that won't be evident to someone who's just looking for typos without necessarily understanding what the research is about.

[nescafe]

From my perspective, it's both excellent to receive these requests and good citizenship to accept them when possible. In my tiny historical subfield, it would be impossible to get good peer review without occasionally asking grad students for assistance. I would hesitate to accept to review a manuscript if you have not yet published anything yourself (have you?). Otherwise, unless the topic of the article is distant to your own focus, I'd take it on. You're never the only referee, and it's a good experience.

As for how to review the manuscript, keep everything you say in the vein of improving the piece for publication. The review isn't about editing or in-fine changes, and it isn't just an assessment. It's a chance to offer ideas for how to ready the piece for publication, whether at the journal of choice or somewhere else. What are the article's central goals within its stated literature, and does it accomplish them? If not, what might help the author to reach those goals?

[adel9216]

Hello! I was not the sole reviewer so that helped. I gave the feedback and the editor in chief was satisfied with my comments. I'm glad I was able to help!

[fast_and_bulbous]

I always worry when I review a paper, even one in my own narrow area of specialization.  I can go through the paper and advise on poor English word choice, punctuation, and so forth, and I can point out awkward phrasing and typos, but beyond that I'm at a loss.

Partly this is because I feel I should respond with a statement as to whether the work is appropriate for the journal (I have no idea, except, you know, the journal has that discipline's name in its title), whether the work is of interest to the mathematical community (who the hell knows? A lot of what the 'mathematical community' is interested in, to me, seems to be garbage, and I'm sure the community feels that way about lots of work that I find interesting), and about the work's "significance," which I can't define.

Maybe I should just, you know, turn down reviewing gigs?

Replying a bit late to this thread, just discovered it.

That's a tough situation. I have turned down reviewing papers after realizing I was in over my head. It's better to do that, in my opinion, than to essentially let stuff slide because it's beyond your expertise, if that is the case. I'd just be as honest as I could be on the questions about relevancy etc. I'm in STEM and a lot of the papers I see are not really moving the field forward. Simulation is a big part of my field and a lot of people just like to simulate something and describe it and call it a day without any meaningful analysis. Those papers usually end up in the low tier journals. I would say about half of my recommendations are reject, with encouragement for resubmission. "Accept with major revisions" is recommended all the time in my field and I think it's a terrible thing because "accept" doesn't always happen after the "major revisions." I've seen that happen more than a few times. Sometimes after you wash all the grime away it reveals a clearly flawed object.

I take my responsibility as a reviewer extremely seriously and I think the majority of people in my field do as well. Yes, it sucks that we don't get paid for it. As a soft funded scientist I can spread it out among grants but for sure a lot happens on my free time. In a way the reviewers are as important as the authors in making sure good science gets published, and that due diligence is observed on both sides.

I dread reviewing NSF proposals because they take even more work and the stakes are higher. So I have to be careful not to bite off more than I can chew.

[ergative]

Sometimes after you wash all the grime away it reveals a clearly flawed object.

F&B, buddy, we talked about this. Quit looking over my shoulder when I'm working.