academics, as seen from vegreville. it can be cold here. and it is flat.

Email me

Previous Posts Site Feed


What the hell have I been doing this (academic) year?

  • Manuscripts accepted: 2
  • Manuscripts under review: 1
  • Revise and resubmits to do: 3
  • Working papers: 3
  • New projects: 2
  • Conference presentations: 2
  • Seminars given: 1
  • PhD students in progress: 5
  • PhD students completed: 0
  • Other students supervised: 2
  • Courses taught: 0
  • Courses scheduled: 4
  • Referee reports to write: 2
  • Referee reports completed: 17
  • Committees: 3
  • Angry co-authors: 0
  • Angry students: ?

powered by Blogger

designed by mela and modified by vegreville

Creative Commons License
The contents of this web site are licensed under a Creative Commons License.

Monday, February 06, 2006

This paper kicks ass.

It's great to referee a manuscript in which I can say that. The more reports I have written, the better the papers that I get to review.

Here is how I write a report.

  • Try to figure out the point of the paper. Sometimes it's easy. Sometimes it's hard. If it's too hard, I may give up.
  • If I understand the point, then I decide if it's new or not (to me and the literature) and if the analysis in the paper is correct. If the point is new, correct, and interesting, then see if I can help the author improve the exposition, results, and scope of the paper. If the point is not new, or incorrect, or uninteresting, then clearly explain my position in the report.
  • If the writing is bad, I won't get the point. If the technical material is wrong or sloppy, I won't get the point. If the point is correct but the technical material is sloppy or the writing is bad, then it is going to take me a lot of work. I need to decide if the point is worth the work that has to be put in.
  • The subjective part here is deciding if it is an interesting point or not. The editor wants my opinion, or I would not be chosen to be the referee.
  • Sometimes what the author thinks the point is turns out to be different than what I think the point should be. Sometimes the author puts many points in the paper, so I need to help the author focus on a few of them.
  • I always try to help the author improve the paper, with specific suggestions, even when I think the paper should be rejected from the journal. This involves going through the manuscript carefully a few times with a pencil and pad.
  • I am always clear to the editor about what I think the outcome should be, and if it's a revise and resubmit I tell the editor and author exactly what I think would lead me to suggest acceptance.
  • I am never wish-washy about what I think should happen. That doesn't mean I rewrite the paper, but it does mean that I try to be specific as possible. I have been burned a few times myself by wishy-washy reviewers who don't tell me what is required to have a successful outcome. In all such cases, the paper has gone more than one round only to be rejected in the end. Probably my fault, but it did waste everyone's time.
  • I always try to take the work seriously.
  • I try never, never, never to be sarcastic or mean. Of course trying may be be the same as achieving. But that being said, the most useful report I ever got was full of sarcasm--intended or not. The report forced me to work on being precise and clear in that paper and in future research. Helpful in the long run, even though the paper did not get accepted at that journal.
Many of the journals I review for have double blind refereeing. But since I see so many papers at seminars and conferences, I always wonder about how blind it is. I try hard not to let the author's identity influence my refereeing.

I find that the best way to get a report done is to read the paper, think about it carefully for a while, and then start writing. I usually start by writing a short paper summary, and then by going through the manuscript with a pencil or my text editor open, noting unclear and incorrect things. I then write a few paragraphs about why I or do not think the paper is successful. Rinse, lather, and repeat a few times until I am happy with the report.

link | posted by vegreville at 9:21 PM |


Anonymous Anonymous commented at 8:07 PM~  

Interesting. I wish I could draw you from the referee distribution instead of the people I seem to get.

I would add one thing. I try to communicate to the author that I have read the paper thoroughly and understand the main point of the paper (if one is articulated) as the author intended. I do this by starting every report with a one paragraph summary of the paper. I get too many reports back that appear to be about a paper different than the one I wrote, or worse, about the paper the referee wanted me to write. I don't know if this improves the refereeing process, but at least it makes me feel better.

Want to Post a Comment?