I'm certainly not an expert at reviewing papers for publication in academic journals, but I've done it a few times and have another one on my desk to review now. It's a pretty time intensive task. For me, I usually have to read the paper 3 times:
- To figure out the gist of the whole thing and to try and pick out the structure, methods, evidence, claims, etc
- In order to make specific notes on the above stuff so you can comment to the authors
- One last time to look for the details and comment on them- things like grammar, reference consistency, etc. I do a lot of this stuff in the previous reads, but I always have to do one more focused just on these tasks.
Once I've done this and made my notes, then I have to write them up in a coherent fashion (beginning with an outline) that will be interpretable and useful for the authors and journal editors. Then I make a decision to report to the editors (accept, revise and re-submit, reject, etc) and present it in a separate cover letter.
Like I said- I'm not an expert at this, but it is a useful and educational service to perform. Sometimes doing this makes me feel good about my own writing, and other times it really humbles me. It also helps me keep in touch with what others are doing in a different way than just reading the final version in publication.
Now back to the one on my desk, which I've only read through once so far.