Peer Reviewed Publishing

What sorts of methods are used by non-science journals to evaluate manuscripts submitted for publication? I'm already pretty familiar with the science/health review process, but don't know a thing about the procedures in the humanities or social sciences.

Are they sent to independent reviewers?

Are the reviews blinded (reviewer doesn't know identity of author, author doesn't know identity of reviewer)?

If blinded, are there usually clues in the paper itself that would give the reviewer a good idea of who might have written the paper?

Does an editor at the journal decide who should review a particular article? If so, how?

Are there so widely divergent that anyone from school X would always pan anything from school Y, and vice versa? How are such situations handled? -- [mailto:beisbol@ALUMNI.PITT.EDU Neal Traven], words-l, 20010802

Some potentially useful online resources:


 * http://www.alpsp.org.uk/pub4.htm (and the PDF linked therein)


 * http://www.exploit-lib.org/issue5/peer-review/


 * http://www.firstmonday.dk/issues/issue4_4/proberts/


 * http://www.ama-assn.org/public/peer/arevi.htm


 * http://www.amacad.org/publications/trans5.htm

Tangentially: there is some discussion in there about the difference widespread low-cost electronic publishing/interaction makes. At the limit, the current avant-garde new-age geek thinking is that things like collaborative filtering and reputation-based systems will provide something functionally similar to peer review. Extant prototypical reputation-based systems include eBay and slashdot. An alternative to reputation-based systems is an evaluative system such as Idea Futures --Peter Kaminski, words-l, 20010802

Electronic publication has opened up a discussion that it's possible now to publish a lot more than is practical with paper journals, with subsequent discussion about how refereeing fits into that (see Hanard's "subversive proposal", for instance).

--Actually, while the above-cited reference is an interesting discussion of scholarly publishing issues, the actual "subversive proposal" from Steven Harnad is one part of an excellent document put out theby the Association of Research Libraries (ARL). (added by Liz Lawley, 8 March 03)

For me, this segues into the reputation systems and collaborative filtering work that's been done recently online.

So my epiphany was that peer review sort of fits into a continuum that includes collaborative filtering, and even idea futures. The continuum runs from tightly-controlled content to chaotic, market-based control:


 * editoral board review [professional editors]
 * peer review [knowledgable editors]
 * reputation-based systems [democratic editing]
 * collaborative filtering [communal editing]
 * idea futures [market-based evaluation of key theses]

-- Peter Kaminski, 20010802

 Peter Kaminski points to a continuum of idea review from controlled to chaotic: editorial board review by professional editors, peer review by knowledgeable editors, reputation-based systems with democratic editing, collaborative filtering with communal editing, idea futures based on market evaluation of key theses.