On Monday the Duke Libraries celebrated Open Access week with a talk by Jason Priem that was ostensibly about alternative metrics for measuring scholarly impact – so-called AltMetrics.  Jason is a Ph.D. student at the University of North Carolina School of Library and Information Science, a co-author of the well-regarded AltMetrics Manifesto, and one of the founders of the Web tool called ImpactStory.  In addition to his public talk, Jason gave a workshop for a small group of people interested in how ImpactStory works; you can read about the launch of that tool in this article.

So Jason is as qualified as anyone to talk about AltMetrics, and he did so very well.  But his talk was much broader than that subject implies; he really gave a superb summary of the current state of scholarly communications and a compelling vision of where we could go.  For me the most memorable quote was when he said, toward the end of his talk, that the Web had been created to be a tool for scholarly communications, yet while it had dramatically changed many industries, from bookselling to pornography, it had not yet revolutionized scholarly publishing as it should.  The problem is that publishers, and, to some extent, authors, are treating the Web as simply “a faster horse” and not truly exploiting the possibilities it offers to change the way scholarship is done.

Jason began with some history, pointing out the the earliest forms of scholarly communications were simply direct conversations, carried out through letters.  The first scholarly journals, in fact, were simply compilations of these sorts of letters, and you can still see that venerable tradition reflected in some modern journal titles, like “Physical Review Letters.”  But the modern journal, with its submission process, peer review, and extremely delayed publication schedules, was a revolution and a dramatic re-visioning of how scholars communicated with one another.  Certainly there were significant gains in that new technology for communication, but things were lost as well.  The sense of conversation was lost, as was immediacy.  Now, according to Jason, we have the ability to recapture some of those values from the older model.

Another dramatic change in scholarly communications was the movement called “bibliometrics,” which led to the creation, in the early 1960s, of the citation index and the journal impact factor.  Like the journal itself, the impact factor is so ingrained in our current thinking that it is hard to remember that it too was once a new technology.  And it is a system with significant problems.  As Jason said, the impact factor can track only one kind of person, doing one kind of research by making one kind of use.  The impact factor cannot track non-scholarly uses of scholarly works, or even scholarly uses that are not reflected in another journal article.  Also, true social impact , the kind of policy-changing impact that many scholars would see as an important goal, is seldom reflected in an impact factor.  The problem we face, Jason argued, is that we have confused the kind of use we can track with use itself.  In the process we often miss the real heart of scholarship, the conversation.

In the digital age, however, we can begin to track that conversation, because much of it is taking place online. AltMetrics, by which we teach computers how to look for a variety of article-level citations and discussions, offers the chance to analyze the scholarly environment much more thoroughly, and give individual scholars a much clearer and more comprehensive story of their real impact.

One connection that was hard for me in Jason’s talk, but ultimately persuasive, was his discussion of why Twitter is important.  I admit to being a reluctant and unenthusiastic Twitter user.  This blog post will be distributed via Twitter, and most of my readers seem to find what I write that way.  But still I was startled when Jason compared Twitter to that earliest form of scholarly communications, the conversation.  What was new to me was to think of Twitter as an opportunity to have a preselected jury suggest what is important to read.  If I follow people whose work is interesting and important to me, and they are all reading a particular article, isn’t it extremely likely that that article will be interesting and important to me as well?  And isn’t that peer review?  We sometimes hear that peer review is professional evaluation while Twitter is merely a popularity contest.  But Jason challenged that distinction, pointing out that if we follow the right users, the people whose work we know and respect, Twitter is a scholarly tool in which popularity becomes indistinguishable from professional evaluation.  Since many scholars already use Twitter, as well as blogs an e-mail lists, in this way, it is fair to say that new forms of peer-review have already arrived.  The AltMetrics movement aims to track those other forms of scholarly impact.

Jason ended his talk with a proposal to “decouple” the scholarly journal, to recognize that journals have traditionally performed several different functions — often identified as registration, certification, dissemination and archiving.  Some of those functions are now trivial; why pay anyone for dissemination in an age when an article can be distributed to millions with the click of a mouse?  Other functions, especially certification (peer-review) are changing dramatically.  Jason suggested that peer-review should be a service which could be offered independently of how an article was to be disseminated.  Scholarly societies especially are in a good possession to provide peer-review as a service for which scholars and their institutions could pay when it was felt to be necessary.  but in an age when so much peer-review is already happening outside the structure of journal publication, it is clear that not all scholarship will require that formal service.  So in place of the rigid structure that we have now, Jason suggests, illustrates, and enables a more flexible, layered system of scholarly networks and services.

As should be obvious by now, I found Jason’s talk for Open Access Week provocative and thought -provoking.  I hope I have represented what he said fairly. I have tried to indicate where I am paraphrasing Jason directly, and he should not be blamed for the conclusions I draw from his comments.  But for those who would like to hear from Jason directly, and I highly recommend it, he and several other leaders in the area of AltMetrics will take part in a webinar sponsored by NISO on November 14, which you can read about and register for here.  You can also finds slides and a video from a presentation similar to the one he gave at Duke here.

 

5 Responses to Is the Web just a faster horse?

  1. Ian says:

    Priem and Hemminger wrote an excellent article on the “decoupled journal” in Frontiers in Computational Neuroscience. In the same journal there is now a research topic devoted to new ways of thinking about the journal. Highly recommended.

  2. Philip Bolton, Jr says:

    I was listening in on an event via the Birkbeck University of London and Melissa Terras talked in depth about augmenting the impact of her green OA material by blogging and tweeting about the works. She found when she blogged and tweeted about her work, the work received many more downloads. She purposely kept quiet about some articles, which received few downloads. She even mentions that a publisher asked her to submit a blog post to a peer reviewed publication because she had tweeted about it. So I’d have to agree with Jason that it’s possible to create social media communities that can share and in a sense vet materials.

  3. Bob Smith says:

    Interesting! I’ve begun to use pinterest as a scholarly tool as well. I have boards for articles and blog posts of interest, of archeological evidence that affirms the accuracy of Biblical passages, etc. To my great surprise, people are following it.

  4. [...] basic research for the advancement of economic prosperity and human health and this is the kind of impact we should be trying to assign metrics to. [...]

  5. Kate says:

    Thanks for capturing your thoughts about Priem’s presentation, Kevin. It has helped me envision ways to connect my PLN to my library work and scholarship.