August 23, 2010
Scholars Test Web Alternative to Peer Review
By PATRICIA COHEN
For professors, publishing in elite journals is an unavoidable part of
university life. The grueling process of subjecting work to the up-or-down
judgment of credentialed scholarly peers has been a cornerstone of
academic culture since at least the mid-20th century.
Now some humanities scholars have begun to challenge the monopoly that
peer review has on admission to career-making journals and, as a
consequence, to the charmed circle of tenured academe. They argue that in
an era of digital media there is a better way to assess the quality of
work. Instead of relying on a few experts selected by leading
publications, they advocate using the Internet to expose scholarly
thinking to the swift collective judgment of a much broader interested
audience.
"What we're experiencing now is the most important transformation in our
reading and writing tools since the invention of movable type," said
Katherine Rowe, a Renaissance specialist and media historian at Bryn Mawr
College. "The way scholarly exchange is moving is radical, and we need to
think about what it means for our fields."
That transformation was behind the recent decision by the prestigious
60-year-old Shakespeare Quarterly to embark on an uncharacteristic
experiment in the forthcoming fall issue — one that will make it, Ms. Rowe
says, the first traditional humanities journal to open its reviewing to
the World Wide Web.
Mixing traditional and new methods, the journal posted online four essays
not yet accepted for publication, and a core group of experts — what Ms.
Rowe called "our crowd sourcing" — were invited to post their signed
comments on the Web site MediaCommons, a scholarly digital network. Others
could add their thoughts as well, after registering with their own names.
In the end 41 people made more than 350 comments, many of which elicited
responses from the authors. The revised essays were then reviewed by the
quarterly's editors, who made the final decision to include them in the
printed journal, due out Sept. 17.
The Shakespeare Quarterly trial, along with a handful of other
trailblazing digital experiments, goes to the very nature of the scholarly
enterprise. Traditional peer review has shaped the way new research has
been screened for quality and then how it is communicated; it has defined
the border between the public and an exclusive group of specialized
experts.
Today a small vanguard of digitally adept scholars is rethinking how
knowledge is understood and judged by inviting online readers to comment
on books in progress, compiling journals from blog posts and sometimes
successfully petitioning their universities to grant promotions and tenure
on the basis of non-peer-reviewed projects.
The quarterly's experiment has so far inspired at least one other journal
— Postmedieval — to plan a similar trial for next year.
Just a few years ago these sorts of developments would have been
unthinkable, said Dan Cohen, director of the Center for History and New
Media at George Mason University. "Serious scholars are asking whether the
institutions of the academy — as they have existed for decades, even
centuries — aren't becoming obsolete," he said.
Ms. Rowe, who served as guest editor for The Shakespeare Quarterly's
special issue devoted to Shakespeare and new media, said: "The traditional
process is not so much a gold standard but an effective accommodation to
the needs of the field. It represents a settlement for a particular
moment, not a perfect ideal."
Each type of review has benefits and drawbacks.
The traditional method, in which independent experts evaluate a
submission, often under a veil of anonymity, can take months, even years.
Clubby exclusiveness, sloppy editing and fraud have all marred peer review
on occasion. Anonymity can help prevent personal bias, but it can also
make reviewers less accountable; exclusiveness can help ensure quality
control but can also narrow the range of feedback and participants. Open
review more closely resembles Wikipedia behind the scenes, where anyone
with an interest can post a comment. This open-door policy has made
Wikipedia, on balance, a crucial reference resource.
Ms. Rowe said the goal is not necessarily to replace peer review but to
use other, more open methods as well.
In some respects scientists and economists who have created online
repositories for unpublished working papers, like repec.org, have more
quickly adapted to digital life. Just this month, mathematicians used
blogs and wikis to evaluate a supposed mathematical proof in the space of
a week — the scholarly equivalent of warp speed.
In the humanities, in which the monograph has been king, there is more
inertia. "We have never done it that way before," should be academia's
motto, said Kathleen Fitzpatrick, a professor of media studies at Pomona
College.
Ms. Fitzpatrick was a founder of the MediaCommons network in 2007. She
posted chapters of her own book "Planned Obsolescence" on the site, and
she used the comments readers provided to revise the manuscript for NYU
Press. She also included the project in the package she presented to the
committee that promoted her to full professor this year.
Many professors, of course, are wary of turning peer review into an
"American Idol"-like competition. They question whether people would be as
frank in public, and they worry that comments would be short and episodic,
rather than comprehensive and conceptual, and that know-nothings would
predominate.
After all, the development of peer review was an outgrowth of the
professionalization of disciplines from mathematics to history — a way of
keeping eager but uninformed amateurs out.
"Knowledge is not democratic," said Michèle Lamont, a Harvard sociologist
who analyzes peer review in her 2009 book, "How Professors Think: Inside
the Curious World of Academic Judgment." Evaluating originality and
intellectual significance, she said, can be done only by those who are
expert in a field.
At the same time she noted that the Web is already having an incalculable
effect on academia, especially among younger professors. In her own
discipline, for instance, the debates happening on the site
Sociologica.mulino.it "are defined as being frontier knowledge even though
they are not peer reviewed."
The most daunting obstacle to opening up the process is that peer-review
publishing is the path to a job and tenure, and no would-be professor
wants to be the academic canary in the coal mine.
The first question that Alan Galey, a junior faculty member at the
University of Toronto, asked when deciding to participate in The
Shakespeare Quarterly's experiment was whether his essay would ultimately
count toward tenure. "I went straight to the dean with it," Mr. Galey
said. (It would.)
Although initially cautious, Mr. Galey said he is now "entirely won over
by the open peer review model." The comments were more extensive and more
insightful, he said, than he otherwise would have received on his essay,
which discusses Shakespeare in the context of information theory.
Advocates of more open reviewing, like Mr. Cohen at George Mason argue
that other important scholarly values besides quality control — for
example, generating discussion, improving works in progress and sharing
information rapidly — are given short shrift under the current system.
"There is an ethical imperative to share information," said Mr. Cohen, who
regularly posts his work online, where he said thousands read it. Engaging
people in different disciplines and from outside academia has made his
scholarship better, he said.
To Mr. Cohen, the most pressing intellectual issue in the next decade is
this tension between the insular, specialized world of expert scholarship
and the open and free-wheeling exchange of information on the Web. "And
academia," he said, "is caught in the middle."
____________________________________________________________
Dit bericht is verzonden via NEDBIB-L@nic.surfnet.nl
Aanmelden? Afmelden? Info? Vacature plaatsen?
Zie de NEDBIB-L website op http://nedbib.reuser.biz/
Lijstbeheerder: Arno H.P. Reuser nedbib@reuser.biz
EEN ANTWOORD GAAT STANDAARD NAAR ALLE LIJSTDEELNEMERS
Wellicht is uw antwoord alleen zinvol voor de vragensteller
Houd hier a.u.b. rekening mee!
Freek Zegers
