Assessing for Usage

There is a tendency to compare the usage of ebooks with what we know about print circulations, but this is not exactly a fair comparison. There are several well-trodden ideas about print collections that may be true for some collections but do not apply generally. One of these is the idea that 40% of library print materials never circulate, popularized by the Pittsburgh study, and another is Richard Trueswell’s 1969 observation that 20% of print collections are responsible for 80% of the circulations. As usage began to take center stage with the rise of electronic materials, these studies were frequently cited (Fischer, Wright, Clatanoff, Barton, Shreeves, 2012; Nixon, Freeman, & Ward, 2010; Tyler, Melvin, Yang, Epp, & Kreps, 2011) as evidence of progress in the switch from print to digital access. There are compelling arguments to refute this line of thinking (Fry, 2015), but the literature on the subject is very mixed. There is some evidence of low print circulations in traditionally selected print collections, some evidence that electronic demand-driven acquisitions (DDA) collections have better circulations, and plenty of inconclusive and contrary evidence for both these claims.

For instance, the University of Tennessee was able to substantiate Trueswell’s findings for their collections as a whole, but there were deviations in particular Library of Congress subclasses like government documents (J) and education topics and reports (L). These had a much more dismal breakdown: only 6% of items in education topics and reports accounted for 80% of the circulations and only 1.5% of the items in government documents. They also found disciplines with very broad use, gynecology and obstetrics (RG) and pediatrics (RJ) showed both high circulation and a high percentage of titles used in the collection. The researchers recommended expansion in areas with broad use and restriction or changing strategy in areas with narrow use (Britten, 1990). This situation, with inconsistent use across disciplines but a need for representation from the community, is an ideal setting for discipline-specific DDA or even a broad DDA strategy across all disciplines. This way purchasing could follow use while allowing for interdisciplinary or trend-based spikes in the usage of traditionally underused areas.

Indiana University also found inconsistencies in usage between publishers (Adams & Noel, 2008). This is a compelling analysis to conduct whether or not a DDA program is in place because it could potentially reveal publishers that could be weeded from the discovery system to improve overall usage. Duke University’s duplicate print and ebook study found a slightly elevated circulation rate for ebooks (11% higher) but a lot of consistency within disciplines and 39% overlap of titles in the checkout group for both print and digital (Littman & Connaway, 2004). This type of study reinforces the idea that we might need to look deeper than the whole collection and cross formats when we are assessing usage.

This inconsistent use across collections by publisher or discipline is probably partially to blame for low total circulations at institutions like the University of Colorado at Boulder, which recorded a 33% circulation of items between Jan. 1998 and Dec. 2002 (Knievel, Wicht, & Connaway, 2006). Brigham Young University Library also found that only 16% of the books purchased with traditional collection development methods circulated in the first year and only half circulated within 10 years (Schroeder, 2012).

Another piece in the puzzle is the use of circulation statistics to gauge usage in print collections. In open but noncirculating print collections, circulation numbers are either totally absent or not really that meaningful. Anecdotally we know that even if libraries have clearly posted policies asking patrons not to reshelve materials, some (and possibly many) users still do (Blake & Schleper, 2004). Ebooks give us a window into unrecorded “trivial” uses like picking up the book and reading the back cover, searching something in the index, looking at the preface, or reading the table of contents. It is likely that most of these uses go unrecorded in physical libraries and that certainly contributes to some of the elevated numbers we are able to observe in ebooks. That said, the comprehensive data that librarians are able to gather from ebook DDA programs is one of their greatest advantages. The possibility that we can “prove” our collections are being used and are useful to researchers is a powerful tool that has real implications in the funding, promotion, and status of libraries regardless of whether or not we are comfortable with that idea. It is absolutely true that the utmost care should be taken when we are studying the use of our collections, and also absolutely true that providing quick and convincing information about our usage is beneficial for libraries.

The DDA literature can explain some methods for assessing collections by use. The first is comparing materials from different collection strategies to one another, but it is important to keep in mind that this is not always a fair method of comparison. The University of Kansas Libraries had a print DDA program that showed an 88% initial circulation rate compared to 60% for their traditionally selected books (Currie & Graves, 2012). The University of Nebraska-Lincoln examined 63,732 purchases made through interlibrary loan-to-purchase, approval plan, and librarian selections. They subtracted the initial circulations of the DDA books but found that they still outperformed traditionally acquired books, nearly doubling the average circulations of the other materials. They found that while the interlibrary loan-to-purchase model was the best at predicting which materials would circulate, the librarian selections also performed well. The librarians at the University had very strong liaison relationships, so many of the purchases through this method probably came from informal patron recommendations. They found that approval plans were the weakest at predicting later use (Tyler, Falci, Melvin, Epp, & Kreps, 2013).

The University of Wisconsin-Madison also found that interlibrary loan-to-purchase programs were good at predicting use. Seventy-three percent of items purchased through the interlibrary loan-to-purchase program circulated two or more times in the first two years, while only 6% of traditionally developed collections circulated two or more times. Purdue found that 57% of DDA titles circulated at least once after their initial circulation, while only 31% of traditionally developed collections circulated. This study is an example of two materials that shared the same format, but were selected using different criteria. The interlibrary loan-to-purchase items were put through what was certainly a more rigorous vetting process than some of the items that were purchased through traditional methods. Both universities placed restrictions by year on interlibrary loan-to-purchase programs, so these were newer materials that fit into the collection’s guidelines (Ward, Wray, & Debus-López, 2003).

Open Polytechnic University in New Zealand found evidence that students were treating the electronic materials in the way they assumed students were using reference materials. The average session user looked at 45.6 pages in a session. They also looked into which pages the students were using and found that they skipped around a lot from section to section in the book. Though students may not have been using resources as deeply as the librarians had assumed, they used them often, the average item was used 9.2 times during the year including the three triggering uses. They compared this to their print materials, which they projected had an average of 0.62 uses per item. The researchers determined that short-term loans were generally cheaper than interlibrary loan and that autopurchases were roughly the same price as purchasing a new book via other methods, so DDA was clearly the most economical strategy for them (Kelly, 2010).

The University of Vermont adopted a DDA program because of dismal circulation statistics. They found that 40% of their physical titles had never been checked out. They worked with three vendors to upload catalog records for items they did not own, with the option for users to click to order the books. In 2007, they were one of the first libraries to offer a catalog-integrated DDA program. Their previous method of acquisitions was approval profile and they offered data from their pilot year in 2009 to back up the efficiency of the DDA method. During the year they purchased 505 items from the discovery pool. If they would have purchased all the books in their approval profile, they would have purchased 1759 titles at a cost of $142,231.12. They calculated that they saved $109,081.48 and 120 feet of shelf space in just one year by offering DDA instead of approval profile spending. It is likely that some of the books on the shelves would have been checked out, but they faced their previously calculated 40% inaction rate on these titles. In 2010, they purchased 590 books, but would have received 1502 under the old plan. Also, they found that DDA books had a better circulation rate, going out twice on average, while traditionally selected books only went out 1.18 times on average (Spitzform, 2011).

The Ontario Council of University Libraries began investigating a patron-driven acquisitions model in 2010. They set up a payment system based on student population for each school and loaded the discovery records into each individual catalog. The Consortium does not have a union catalog, so catalogers had to be in communication about loads and purchases. One of their colleges, Western University, which is a large medical and postdoctoral institution, found that more than half of the consortial DDA purchases duplicated items that they had in their collection in either print or electronic format. Of the unique titles purchased (169) 63% were used at least once and 17% were used 10 times or more. Another college in the consortium, Ryerson University, has large graduate and undergraduate populations in an urban setting. Students there are used to electronic resources because the college has always focused on acquiring these types of resources because of space concerns. Seventy-one percent of the titles purchased in the consortium were unique to Ryerson and 43% had at least one use in the first 6 months of the trial. Interestingly, titles that were duplicated between print and electronic did not see a reduction in print circulations as electronic editions were purchased, but rather circulations on the whole increased. They also conducted a survey and librarians at the other institutions were concerned about duplication in collections and the breadth of the collection (Davis, Lei, Neely, & Rykse, 2012).

The University of Nebraska-Lincoln Libraries did a short-term assessment of their interlibrary loan-to-purchase items to see if they really did circulate more than books obtained through other channels. They found that even with subtracting the initial circulation for interlibrary loan-to-purchase titles, they still circulated more than traditionally acquired books in 83 out of 134 LC subclasses. Only 1.3% of the items acquired through the interlibrary loan-to-purchase program had not circulated, while over 70% of approval plan items had not circulated (Tyler, Xu, Melvin, Epp, & Kreps, 2010). The University of Tennessee at Chattanooga had similar results with their “you ask, we buy” mediated print purchase program in 2010. They found that items from the first year of the program, which had more time to circulate, went out between 4.6 to 5.8 times more than items acquired through other means during this time (Dunn & Murgai, 2014).

It is clear that institutions that value increased usage of materials find DDA programs help them accomplish that goal. Though comparing different formats or acquisitions strategies is not always possible, when we can make thoughtful comparisons it can help make the case for experimental acquisitions strategies. Several studies have also found that DDA programs can help them target increased usage programs towards particular disciplines.

Rutgers University attempted a patron-driven acquisitions program in 2010 which was initially confined to their math and computer science collections. They felt that these programs were accustomed to using ebooks already and they wanted to start with a small pilot so that they could iron out any technical service workflow issues before they brought the program to the library as a whole. The math library’s print circulation numbers were also dismal: 79% of newly published titles had never circulated and 41% of titles published in the past several years had never circulated. They had also conducted a user survey which found that there was a demand in the community for professional programming books and that subject selectors for the discipline had a difficult time keeping up with trends in the field, so demand-driven acquisitions would be a good fit for them. Rutgers got on a monthly PDA discovery record cycle that let subject selectors review each month’s publications before they were added to the catalog to eliminate anything that did not fit collection goals and standards (De Fino & Lo, 2011).

The University of Mississippi evaluated their DDA program and found a huge cost per use benefit to their engineering and computer science acquisitions through this program. Only 21% of traditionally acquired titles in this subject circulated in the fiscal year and items in this group had a cost per use of $396. The DDA program, including both short-term loans and purchases, had a cost per use of $22. They found a similar breakdown in their business and economics profile. Traditionally acquired titles had only a 6% circulation rate and their cost per use was $363, while DDA titles had a cost per use of $36. Their Ebrary subscription collection was even more cost-effective in this discipline, with a cost per use of $7, but a similar circulation rate at 6% (Herrera, 2012).

Analyzing different acquisitions strategies can also provide a foundation for DDA programs. The University System of Maryland and Affiliated Institutions consortium, made up of 16 institutions serving over 160,000 students, began a DDA program in 2013. The Consortium chose EBL who offered two different lending models: a limited-access lending model that restricted the number of users who could access the resources at the same time, but did not include a multiplier and a model in which each library would have a separate DDA profile that they had unlimited access to and browsing access to the libraries of other institutions that they would have to pay to access. The consortium opted to go with the limited-access model and began creating a profile that all the libraries would share. They opted to add short-term loans to this profile. They found that users were 56% undergraduate, 27.3% graduate, 9.8% faculty, and 3.4% staff. Due to their assessment they found that 92.9% of titles were used three times or fewer, leading to the decision that the short-term loan-to-purchase cap should be decreased from six to three to encourage more purchasing and attract new publishers (Lowe & Aldana, 2015).

Policy making for new DDA programs can also be useful for making adaptations to existing programs. The University of Texas Libraries cited both a desire for experimentation and a quantifiable increase in the acceptance of ebooks in their student population for their 2009 exploration of pay-per-view ebooks. Like DDA, this program puts materials in the hands of users quickly and at a lower cost, but instead of becoming a part of the collection indefinitely, these items are licensed to the library for a designated period of time. In the case of the University of Texas, they had analyzed their statistics from their existing subscription ebooks and had determined that only a small percentage of titles had four or more uses, while the majority of items had three or fewer uses. By this count a short-term loan strategy would be far more effective (Macicak & Schell, 2009).

The University of Iowa used sessions per title for each publisher they purchase DDA from to estimate the value of each of these publishers to the program (Fischer & Diaz, 2014). Publishers with a lot of sessions per title might be books that users glance at, but low sessions per title might be things that patrons deeply engage with. Libraries could use an assessment strategy like this to pick good publishers for bulk purchases or DDA discovery pools.

University of Hong Kong Libraries initiated an interlibrary loan-to-purchase program in 2002 in order to purchase materials that the library would ordinarily need to borrow from overseas. They found that purchasing actually took longer on average than borrowing. This may not be true anymore, since the library chose slower shipping options through Amazon in order to cut costs and Amazon’s shipping speed has improved greatly since 2002. They also found that purchasing was on average twice as expensive as borrowing. Despite these drawbacks, 55% of the purchased items were used more than once, meaning that they were probably still a worthwhile purchase (Chan, 2004). Assessing materials for cost will determine how many uses it should receive before it becomes cost-effective for the institution. Particularly with ebook programs and when comparing programs, cost per use is an important metric.

There are also negative analyses which can be done on interlibrary loan transactions. These may represent an unmet demand in the community. The B.L. Fisher Library at Asbury Theological Seminary evaluated their interlibrary loan request subjects against the same subject headings in their holdings to reveal areas that needed to be developed more robustly. The most significant items to pay attention to when doing this are the newly published titles. They have two campuses, one in Kentucky and the other in Florida and they found that the subjects that were revealed in this area were different for each campus (Danielson, 2012).

Many institutions want to create collections that their patrons use broadly and deeply. Assessment for use is a way to experiment with strategies that can help find better workflows for each individual library. Many libraries have found cost reductions and circulation improvements through both DDA programs and interlibrary loan-to-purchase programs.