Academia and “The Business of Evaluation”

A funny thing happened to me on the way to DH2013…

I missed my connecting flight in Charlotte. This delay meant not only re-booking my flight (and re-routing through Atlanta), it meant re-scheduling my reserved shuttle from the Omaha airport to the conference hotel in Lincoln. The shuttle company was very accommodating and squeezed me into a sedan with three other passengers for the 50-min. ride, where I ended up next to a woman who was obviously a fellow DH2013 attendee. In our introductory small talk it became clear that she was also a senior literature scholar of considerable regard, though she was endearingly modest and declined to pitch her paper during the ride. While we ended up having a lovely informal chat about travel, kids, careers (and careers with kids), I spoke at length about the roundabout way I ended up in librarianship and academia in general. Part of this was sharing my belated realization that much of the everyday task of an academic is being a writer. (My younger self had naively assumed that college professors were primarily concerned with teaching.)

Now, where I grew up, being a historian was tantamount to being an astronaut. Not likely. So part of my entry into academia has been consumed with figuring out the system—the behaviors, the etiquette, the peculiarities of “publish or perish” and a prestige economy replete with strict hierarchies, where people mingling over wine and cheese will literally turn away from you at the mention that you are a graduate student, not someone important enough to have a conversation with. Yet here I was, chatting away with a senior scholar who responded genuinely and sincerely, without disdain and without the need to showcase her elevated status.

And when I explained that I hadn’t realized how much academics were in the business of writing, she responded quickly: “No. We’re in the business of evaluation.” From grading, to letters of recommendation, to peer review, to visiting other university departments to advise on a new program/initiative, academics spend most of their time evaluating people, projects, ideas, scholarship. The business of evaluation.

Throughout the conference, I came back to this idea. In a session introducing Anvil Academic, participants discussed the prestige economy of academic publishing and remarked that a large role of publishers was to confer an imprimatur to the work.

And it occurred to me that part of the reason why a published monograph by an academic publisher matters so much for P&T is that it is a shortcut for evaluation. The crossover between academia and university presses is such that academics trust the work that is published has already been evaluated by their own standards. That imprimatur is not based on status or reputation, but on a practical arrangement between scholars and publishers known as peer review. And yet. The publication of a print monograph confers an additional weight because academic publishers also exist (albeit, nominally, it seems) outside of academia. That is, they decide whether scholarship is worth disseminating to the public, through one of the most permanent means available: the printed book. Here, academics are presented not only with a system that allows them a shortcut for their overburdened plate of responsibilities, but also validation that the work has relevance and longevity outside of the academy, thereby attaining the ideal of public scholarship that most (public) universities pay lip service to. Until we are able to create parallel systems that accomplish both of these goals while evaluating the work on its scholarly merits, we will be unable to gain the trust (and, crucially, the participation) of mainstream academics.

The reality, of course, is that the system is broken. Academic publishers do not reach a wide public audience, and are no longer the most visible means of dissemination. The reliance on journal publishing and the “serials crisis” in academic libraries has been a failed attempt to adapt a system of scholarly expedience to a format without any real public audience. Unlike a monograph, which could be collected by elite (even pseudo-) intellectuals as part of their private libraries, scholarly journals are hardly the permanent tokens of scholarship and erudition that books once were. Now, faced with the demise of print, scholars are grappling with the transition to the digital, and attempts to recreate a system of evaluation for digital scholarship based on an arrangement from the print world will necessarily fall short.

We call this scholarly communication. But it’s more than that. It’s a system of outsourcing scholarly labor that has finally outrun its course. If academics are really in the business of evaluation, they need to revisit their business models. And if that means restructuring the labor of evaluation to be more explicit, more open, more communal, and less commercial, sign me up. But it must be done in a way that includes the “public” aspect of scholarship as a gauge of academic success.