Recent Posts

The Bertrand Russell Show

Feminist Philosophers

fragments of consciousness

Gender, Race and Philosophy: The Blog


Leiter Reports: A Philosophy Blog

Long Words Bother Me

semantics etc. highlights

Thoughts Arguments and Rants



Sunday, February 03, 2008

Refereeing Practices: Single- or Double-Blind?

A recent study conducted by Budden et al indicates that double-blind refereeing helps to increase the representation of women in ecology journals. The researchers compared Behavioral Ecology, which implemented double-blind refereeing in 2001, to Behavioral Ecology and Sociobiology, which remains single-blind refereed. Following the introduction of double-blind refereeing there was a 33% percent increase in the number of women represented in BE.

So, why is blind refereeing not standardly employed in ecology? As the article points out, the following four reasons are frequently cited:

1) Increased admistrative burden.
2) Referees can determine author identify in other ways.
3) The decreased potential for more feedback to junior people.
4) Harder to "detect publication of the same data across multiple papers"

But none of them survives closer scrutiny.

Ad 1) If the journal asks authors to prepare their papers for blind review, double-blind refereeing does not increase the work load for the editor. And there certainly shouldn't be an increased burden on the reviewer, as we should expect the reviewer to apply the same high standards in both cases.
Ad 2) Guesses tend to be inaccurate. Referees make correct guesses only in 25% - 42% of the cases. A related concern is that referees might google the paper, which would make double-blind refereeing redundant. But, as not every author posts their work in progress, this is not a foolproof method for determining author identity either.
Ad 3) If this is a real concern, the editor (who knows the author's identity) could ask the referee for a written report, rather than a 'yes' or 'no' assessment.
Ad 4) This may be a genuine concern in the sciences. But I doubt that it generalizes to other areas. It certainly is not a concern in philosophy, as far as I can tell.

So what are the lessons (if any) for philosophy? Well, most philosophy journals are already double-blind refereed, but the data can perhaps explain the underrepresentation of women in edited volumes (as inclusion is determined prior to refereeing). It might also give reason to implement tripple-blind refereeing (i.e., neither editor nor referee knows the author's identity).

(Thanks to Claire Horisk for sending the link)


Horisk said...

Hi Brit, I'm glad to see you discussing this study. The findings are highly relevant to discussions like this one on Thoughts, Arguments and Rants --
about the merits of anonymous reviewing for journals; the findings are also relevant for journals such as Analysis where the first cut is made by the editor with no anonymity. It also seems to me that this study should make us worry about biases in grant reviews and in grading student papers.
Claire Horisk

Brit said...

Yeah, you're right. We should worry about biases in grant reviews, student papers and single-blind reviews of journal submissions. The findings are probably also relevant for assessments of book manuscripts, book proposals and writing samples (for graduate school and the job market).