Category Archives: Assessment

Data Sciencing our Journal Subscriptions

The ongoing tensions between academic institutions and publishers have been escalating the last few months, but those tensions have existed for many years. The term “Big Deal” has been coined to describe a long-standing, industry-wide practice of journal bundling that forces libraries to subscribe to unwanted and unneeded publications rather than paying more for a limited number of individual subscriptions. This is a practice you see in other industries – for example, cable packages that provide hundreds of channels, even if you only want one or two specific channels.

What is especially problematic in higher education is that academics produce and review the content that gets published in the journals (for free), and then the universities have to pay the publishers a subscription fee to access the content. Imagine if YouTube required a subscription fee to watch any videos, including the ones you had posted. It’s a system that makes research harder to access and inhibits global scientific progress, all so publishers can earn an enormous profit margin.

Right now, academic publishing is controlled by five publishers (the “Big Five”) – a monopoly that makes it very difficult for libraries to negotiate better deals. Only very large organizations or consortia, like the University of California, have been able to start pushing back against the system. It will likely take large shake-ups like this for any large changes to take hold, but it in the meantime there may be ways to situate ourselves for making better purchasing decisions.

At Duke, we often review our usage of specific journal titles as we prepare to make purchasing decisions. Usage data comes in a variety of forms, but the most popular are counts of Duke views and downloads that come directly from the publishers and the number of times Duke authors publish in or cite a particular journal. There are many other kinds of data that might be of interest, however, including Duke participation on editorial boards, usage differences across disciplines, and even whether or not the journal is fully open access. Blending various data sources and optimizing the search decisions for a given budget cycle can be overwhelming.

Last fall, Duke University Libraries decided to propose a project for Duke’s Data+ summer program – a summer research experience in data science for undergraduate students. Our project, “Breaking the Bundle: Analyzing Duke’s Journal Subscriptions“, focuses on Duke’s subscriptions to journals published by Elsevier. The program is in its third week, and our team of two incredibly-sharp undergraduates has been hard at work building and blending our datasets. Our goal by the end of summer is to have a proof-of-concept dashboard that lets collection managers adjust the weights of various usage measures to generate an ideal collection of journals for a particular budget.

It is still very early in the process, but the students have been hard at work and have made great progress. We decided it would be best to develop the analysis software and dashboard using R, a statistical computing project with a rich history and many helpful development tools. In addition to publisher-provided views and downloads, the students have been able to use websites and APIs to collect data on journal open access status, editorial boards, numbers of publications, and numbers of citations. All Data+ teams present publicly on the projects twice during the summer, and we hope to schedule a third talk for a library audience before the end of the program on August 2.

Sample R code from the project
Just one of many files of R code generated for the project so far.

We look forward to seeing what the summer will bring! While this project is just one small step, automating the collection and analysis of journal usage will position us well, both for responsible purchases and for a hopefully-changing publishing landscape.

Is there an app for that? The seemingly endless quest to make discovery easier for users

Contributed by Assessment & User Experience Department Practicum Students Amelia Midgett-Nicholson and Allison Cruse 

Duke University Libraries (DUL) is always searching for new ways to increase access and make discovery easier for users. One area users frequently have trouble with is accessing online articles. Too often we hear from students that they cannot find an article PDF they are looking for, or even worse, that they end up paying to get through a journal paywall. To address this problem, DUL’s Assessment and User Experience (AUX) Department explored three possible tools: LibKey Discovery, Kopernio, and Lean Library. After user testing and internal review, LibKey Discovery emerged as the best available tool for the job.  

LibKey Discovery logo

LibKey Discovery is a suite of user-friendly application programming interfaces (APIs) used to enhance the library’s existing discovery system.  The APIs enable one-click access to PDFs for subscribed and open-source content, one-click access to full journal browsing via the BrowZine application, and access to cover art for thousands of journals.  The tool integrates fully with the existing discovery interface and does not require the use of additional plug-ins.

According to their website, LibKey Discovery has the potential to save users thousands of clicks per day by providing one-click access to millions of articles.  The ability to streamline processes enabling the efficient and effective discovery and retrieval of academic journal content prompted the AUX department to investigate the tool and its capabilities further.  An internal review of the system was preceded by an introduction of the tool to Duke’s subject librarians and followed with a preliminary round of student-based user testing.

Current DUL interface
Current DUL discovery interface
LibKey interface
LibKey discovery interface

Pros

  • One-Click Article and Full Journal Access

Both the AUX staff and the subject librarians who performed an initial review of the LibKey Discovery tools were impressed with the ease of article access and full journal browsing.  Three members of the AUX department independently reviewed LibKey’s features and concluded the system does provide substantial utility in its ability to reduce the number of clicks necessary to access articles and journals.

  • Streamlined Appearance

The tool streamlines the appearance and formatting of all journals, thus removing ambiguity in how to access information from different sources within the catalog.  This is beneficial in helping to direct users to the features they want without having to search for points of access. The AUX department review team all found this helpful.

  • Seamless Integration

LibKey Discovery’s APIs integrate fully into the existing DUL discovery interface without the need for users to download an additional plug-in.  This provides users the benefit of the new system without asking them to go through extra steps or make any changes to their current search processes.  Aside from the new one-click options available within the catalog’s search results page, the LibKey interface is indistinguishable from the current DUL interface helping users to benefit from the added functionality without feeling like they need to learn a new system.  

Cons

  • Cost

LibKey Discovery carries a relatively hefty price tag, so its utility to the end-user must be weighed against its cost.  While internal review and testing has indicated LibKey Discovery has the ability to streamline and optimize the discovery process, it must be determined if those benefits are universal enough to warrant the added annual expenditure.

  • Inconsistency in Options

A potential downside to LibKey Discovery is lack of consistency in one-click options between articles.  While many articles provide the option for easy, one-click access to a PDF, the full text online, and full journal access, these options are not available for all content.  As a result, this may cause confusion around the options that are available for users and may diminish the overall utility of the tool depending on what percentage of the catalog’s content is exempt from the one-click features.

LibKey Discovery User Testing Findings

An initial round of user testing was completed with ten student volunteers in the lobby of Perkins Library in early April.  Half of the users were asked to access an article and browse a full journal in the existing DUL system; the other half were asked to perform the same tasks using the LibKey Discovery interface.

Initial testing indicated that student users had a high level of satisfaction with the LibKey interface; however, they were equally satisfied with the existing access points in the DUL catalog.  The final recommendations from the user testing report suggest the need for additional testing to be completed. Specifically, it was recommended that more targeted testing be completed with graduate-level students and faculty as a majority of the original test’s participants were undergraduate students with limited experience searching for and accessing academic journal issues and articles.  It was concluded that testing with a more experienced user group would likely produce better feedback as to the true value of LibKey Discovery.

LibKey Summary

LibKey Discovery is a promising addition to Duke’s existing discovery system.  It allows for streamlined, one-click article and full journal access without disrupting the look and feel of the current interface or requiring the use of a plug-in.  Initial reviews of the system by library staff have been glowing; however, preliminary user testing with student participants indicated the need for additional testing to determine if LibKey’s cost is sufficiently offset by its utility to the user.

Kopernio logo

Kopernio is a free browser plug-in which enables one-click access to academic journal articles. It searches the web for OA copies, institutional repository copies, and copies available through library subscriptions. The tool is designed to connect users to articles on and off campus by managing their subscription credentials and automatically finding the best version of an article no matter where a user is searching.

Given the potential of this tool to help increase access and make discovery easier for students, the AUX department initiated an internal review process. Four members of the department independently downloaded the Kopernio plug-in, thoroughly tested it in a variety of situations, and shared their general and specific notes about the tool.

Pros

  • OA Content + Library Subscription

By its design, Kopernio has an advantage over other plug-in tools that serve a similar function (i.e. Unpaywall). When users first download Kopernio they are asked to register their subscription credentials. This information is saved in the plug-in so users can automatically discover articles available from OA sources, as well as library subscriptions. This is an advantage over other plug-ins that only harvest from freely available sources.

Screenshot: Kopernio sign-in page
Kopernio sign-in page
  • Branding

Kopernio has highly visible and consistent branding. With bright green coloring, the plug-in stands out on a screen and attracts users to click on it to download articles.

  • One-Click

Kopernio is advertised as a “one-click” service, and it pays off in this respect. Using Kopernio to access articles definitely cuts down on the number of clicks required to get to an article’s PDF. The process to download articles to a computer was instantaneous, and most of the time, downloading to the Kopernio storage cloud was just as fast.

Cons

  • Creates New Pain Points

Kopernio’s most advertised strength is its ability to manage subscription credentials. Unfortunately, this strength is also a major data privacy weakness. Security concerns ultimately led to the decision to disable the feature which allowed users to access DUL subscriptions via Kopernio when off-campus. Without this feature, Kopernio only pulls from OA sources and therefore performs the same function that many other tools currently do.

Similar to data privacy concerns, Kopernio also raises copyright concerns. One of Kopernio’s features is its sharing function. You can email articles to anyone, regardless of their university affiliation or if they have downloaded Kopernio already. We tested sending DUL subscription PDFs to users without Duke email addresses and they were able to view the full-text without logging in. It is unclear if they were viewing an OA copy of the article, or if they were seeing an article only meant for DUL authenticated users.

Screenshot: Sharing articles through Kopernio
Sharing an article through Kopernio

Running the Kopernio plug-in noticeably slowed down browser speed. We tested the browser on several different computers, both on campus and off, and we all noticed slower browser speeds. This slow speed led Kopernio to be occasionally buggy (freezing, error messages etc.).

Screenshot showing a buggy screen from Kopernio
Buggy screen while using Kopernio
  • Many Features Don’t Seem Useful

When articles are saved to Kopernio’s cloud storage, users can add descriptive tags. We found this feature awkward to use. Instead of adding tags as you go along, users have to add a tag globally before they can tag an article. Overall, it seemed like more hassle than it was worth.

Kopernio automatically imports article metadata to generate citations. There were too many problems with this feature to make it useful to users. It did not import metadata for all articles that we tested, and there was no way to manually add metadata yourself. Additionally, the citations were automatically formatted in Elsevier Harvard format and we had to go to our settings to change it to a more common citation style.

Lastly, the cloud storage which at first seemed like an asset, was actually a problem. All articles automatically download to cloud storage (called the “Kopernio Locker”) as soon as you click on the Kopernio button. This wouldn’t be a problem except for the limited storage size of the locker. With only 100MB of storage in the free version of Kopernio, we found that after downloading only 2 articles the locker was already 3% full. To make this limited storage work, we would have to go back to our locker and manually delete articles that we did not need, effectively negating the steps saved by having an automatic process.

Lean Library Logo

Lean Library is a similar tool to Kopernio. It offers users one-click access to subscription and open access content through a browser extension. In Fall 2018, DUL staff were days away from purchasing this tool when Lean Library was acquired by SAGE Publishing. DUL staff had been excited to license a tool that was independent and vendor-neutral and so were disappointed to learn about its acquisition. We have found that industry consolidation in the publishing and library information systems environment has lowered competition and resulted in negative experiences for researchers and staff. Further, we take the privacy of our users very seriously and were concerned that Lean Library’s alignment with SAGE Publishing will compromise user security. Whenever possible, DUL aims to support products and services that are offered independently from those with already dominant market positions. For these reasons, we opted not to pursue Lean Library further.

Conclusion

Of the three tools the AUX Department explored, we believe LibKey Discovery to be the most user-friendly and effective option. If purchased, it should streamline journal browsing and article PDF downloads without adversely affecting the existing functionality of DUL’s discovery interfaces.

Wrangling Messy Data with Airtable

The Assessment & User Experience department at Duke University Libraries keeps the libraries’ physical and virtual spaces responsive to user needs by constantly gathering feedback. In additional to our biennial user satisfaction survey, we run usability tests, hold focus groups, and host meetings of our student advisory boards, all in an effort to keep a finger on the pulse of the DUL patrons.

These activities can generate a lot of unstructured data! For example, in a typical meeting of our undergraduate advisory board, we might collect feedback from a dozen or more students, generating seven or more pages of notes and covering a range of topics. We review and act upon some of these comments immediately, but others may influence longer-term planning. As library staff, we know how important it is to store information in a way that promotes future access. This year we decided to pilot a new system for storing and describing our unstructured data.

Airtable logoEnter Airtable. If you’re not familiar, Airtable is a cloud-based database solution. Similar to Google Sheets, Airtable lets you enter and share data in your web browser, but it also offers more powerful features for projects that have messy data or interconnected components. There are many Airtable templates to show off the different features, including project trackers, event planners, and even product catalogs.

For our messy data, we built a simple spreadsheet that was general enough to collect data from a variety of sources. We included columns like basic demographics, the feedback provided, the original question or prompt, the date when feedback was provided, and how we collected the feedback. Then we took advantage of Airtable’s special features to create a column for topical tags. One of the column types in Airtable is called “multiple select“, which means you can add multiple tags to a single comment. Other spreadsheets can’t understand a list of tags in a single cell, but Airtable treats each tag separately and allows us to group and filter comments by each individual tag.

a screenshot of an Airtable database with tagged comments

The ability to look at comments across different feedback channels in one central location has enormous potential. Instead of having to hunt through old Word documents or emails, we have a single database that can be searched, sorted, or filtered to explore trends in comments over time. When a question comes up about how patrons feels about a particular service or space, we can compile data much more easily, and we no longer have to rely on our memory of what feedback we’ve received and when.

screenshot of Airtable tutorial instructions

Airtable’s free accounts have a limited number of rows allowed in each database,  but they do offer a discount on paid plans to educational institutions. We’re only just starting to explore the potential of Airtable, but so far we’ve been happy with the ability to collect our messy data in one place and organize comments with tags.

Want to learn more? Take a look at our recent tutorial on using Airtable for coding survey data, originally offered at the Designing For Digital 2019 conference.

Find your haven at Oasis Perkins

(Thanks to Assessment and User Experience Intern Brenda Yang for this post and for her amazing work on Oasis Perkins!)

What if it was possible to unwind – color, do a jigsaw puzzle, meditate – without leaving the Libraries?

It is at Oasis Perkins! This high-ceilinged refuge is tucked into the fourth floor of Perkins in room 418. It’s a perfect place to escape any finals-related tension palpable in study spaces this time of year.

A floor plan showing the location of Oasis Perkins on the 4th Floor of Perkins LibraryYou’ll find:

  • Yoga mats and meditation cushions
  • A jigsaw puzzle table
  • Coloring books, logic puzzles, and sudoku pages
  • Origami paper and instruction books
  • A quiet nook
  • A white noise machine
  • Plenty of natural light (during the day)
  • And more to explore!

Unlike other fourth floor spaces in Perkins and Bostock meant for silent study, feel free to chat and connect with a friend or strangers, or simply sit and reflect quietly.

How did Oasis Perkins come to be?

A photo of different types of teas from the Tea-laxation event.

One major motivation was direct feedback from students. Comments from our 2018 Student Library Satisfaction Survey made clear that while study spaces at the Duke Libraries are a keystone of many students’ academic lives, it can be a stressful place, especially during the exam season: “I love coming to the library during most of the semester… particularly during finals, there is an overwhelming sense of stress that emanates from the other students at the library.” A few students explicitly requested “a room to relax,” a place to have have a “refreshing study break without leaving the library somehow,” or a “stress-relief room.”

We hope that Oasis Perkins can serve as a dedicated place for students to nurture their well-being, fitting into the ecosystem of Oasis West and Oasis East (which are managed by Duke Wellness). However, Oasis Perkins is located, of course, right inside of Perkins Library – and its doors don’t close.

You’ll also find occasional events hosted in Oasis Perkins, from Koru Meditation classes to “Tea-laxation” events. Check out the Oasis Perkins webpage to stay up to date on events, or be in touch if your organization would like to host a relevant get together in this space!

You can find a smaller space on the second floor at the Prayer and Meditation Room in Perkins 220.

Other Wellness Resources at Duke

For other tips and events to help you end the semester strong, check out the Duke Libraries End of Semester Survival Guide. There are also a number of resources right on Duke’s campus to support your mental health, which include:

Oasis Perkins has existed in its current form for only one short semester! Is there something that could change about the Oasis Perkins that would help you re-charge? Our team at the Libraries would love to make it better for you. Fill out a feedback from in the suggestion box in Oasis Perkins, or reach out to brenda.yang@duke.edu with your comments or suggestions.

Understanding the experiences and needs of 1G students at Duke

How can the Duke Libraries support the needs of first-generation (1G) college students at Duke?

A team of library staff became interested in this question after noticing that 1G students’ responses to a survey question about the Libraries were different from those of continuing-generation students. While many 1G students are successful in and out of Duke classrooms, we wondered how their experiences might differ from those of continuing-generation students.

To begin our project, we read existing research on academic libraries’ support of 1G students and spoke  with offices on campus that support 1G students, such as the Duke Office of Access & Outreach. Then, we conducted six focus groups with 1G students, in addition to analyzing responses from the Libraries’ 2018 user survey (which included 2,381 student responses) for 1G and continuing generation students. Our full report discusses this process and our findings in more detail, including concrete recommendations for improving library services.

While the experiences of 1G students are not monolithic, we identified nine core findings, which speak to challenges students experience and suggest specific points for intervention and support. One important overall finding is that 1G challenges are student challenges: support or expansions of campus and library services targeted toward 1G students will help all students succeed.

Finding 1. 1G students perceive a dearth of academic and social information capital.

We asked all focus group participants the following, “Have you ever felt like other people around you know things about college that you don’t know about?” Each time, the response from the group was an overwhelming expression of, “Yes, of course, all the time.” One student captured the experience of her continuing-generation:

“Who told you that? Have you been told your whole life you have to do this? Was there an info session I missed?”

Students repeatedly referred to Duke’s demanding academic environment and the abrupt transition from their high school habits to the expectations of Duke classrooms. At times Duke staff may also take for granted how much knowledge incoming students have.

Information capital is not limited to classrooms; it is also used in social contexts and in navigating college life. 1G experiences are diverse: while some students reported feelings of isolation, others described feeling supported through orientation programs and a lively community on Duke’s East Campus, where many undergraduates live.

While 1G students perceive that continuing generation students are able to rely on family to guide them through the myriad of informational and financial challenges encountered in college, 1G students do not have access to this information from their parents. In addition, they sometimes feel unable to share the stress of college with their parents.

“You have the pressure of pretending ‘I’m okay.’  My parents are so proud of me that I can’t tell them what’s really going on.”

Finding 2. Finances are stressful, and an early source of feeling unwelcome.

Our past research suggested that feeling that one doesn’t belong is a global concern for first-years adjusting to life on a college campus, and one particularly poignant for 1G students. Duke is no exception. Several focus group participants shared comments they received from their peers after revealing that they were the first in their families to attend college:

“Oh you’re smart for a first-generation student. I never would have known!”

First-year focus group participants quoted other early encounters with roommates or colleagues that continued to sting. Many of these comments reflect the fact that financial security is one of the starkest differentiators between many 1G students and their peers at Duke.

“There’s definitely a mentality that exists at Duke that middle class is poor and lower class is even worse. Not that everybody is like that, but it certainly exists.”

When 1G students reveal aspects of their own financial circumstances to their peers, they receive blowback in several ways. For example, a few students shared that their financial aid was stigmatized, with other students suggesting that those who receive aid are very “lucky” to pay so little, or stating explicitly their own significant costs of attendance, possibly to engender shame or guilt. These experiences are formative, alienating, and angering.

Finding 3. An ecosystem of supportive offices and people on campus is critical, but knowledge of and willingness to access resources takes time.

The landscape described above is important to understand because it is the one 1G students step into when they arrive at Duke. However, peer attitudes and financial impediments are difficult intervention points. In contrast, faculty, older peers, and staff are better positioned to be support systems native to the institution:

“When you go to Duke resources, people are more than happy to help you. Adults at Duke are much more receptive and much more understanding of our issues as first-generation students.”

In general, students spoke warmly of the many services, programs, and offices offered on campus. This included the Office of A&O, resident assistants (RAs), peer advisors, a close community on East Campus as freshmen, pre-orientation, the Women’s Center, the Financial Aid office, Counseling & Psychological Services (CAPS), Duke Reach, and cultural student groups.

Word cloud of campus spaces that 1G focus group respondents identified as safe and welcoming

The staff in the Office of A&O were mentioned frequently. Students praised staff members, citing their open door policy, knowledge, and willingness to offer genuine and consistent support.

Unfortunately, some students reported feeling stigmatized by others when they were known to have used campus resources such as CAPS, the Academic Resource Center (ARC), the Women’s Center, and identity centers.

Finding 4. The cost of textbooks is a special pain point.

In most focus groups, students shared the challenge, stress, and fear of purchasing expensive textbooks. This anxiety about textbooks rests on top of an ongoing concern about finances. Students described extensive efforts to find affordable copies, taking great pains to maintain their workbooks so they could re-sell them at the end of the semester, and to locate upper-level textbooks that were not available through Textbook on Reserve.

“[Laughter] I’ve never researched so hard as when I’m looking for a digital version of a textbook!”

Students who knew about and utilized the Libraries Textbooks on Reserve made special note of its impact and importance in alleviating some financial burden.

“The textbook rental program has been really important and impactful for me… To get them here and be able to rent them out for 3 hours has been perfect. It’s really important to have that.”

Results from the Libraries biennial student survey also support the idea that the Textbooks on Reserve program is particularly important for 1G students.

Finding 5. 1G challenges are challenges common to many Duke students.

Broadly speaking, 1G students’ responses to our 2018 survey did not differ from those of continuing-generation students. Both groups are generally confident in their ability to use library resources and report that they have successfully used the library and/or the library website to find research articles and books for class assignments. They find the website easy to use, believe that the library is welcoming, that library staff are helpful, and that the library is an important part of their experience at Duke.

1G students were more likely to report that expansion of the Textbooks on Reserve program and the device-lending program (for borrowing equipment such as laptops or cameras) would improve their library experience a great deal.

Overall, the four areas in which more than 50% of all undergraduates responded that expanded services would improve their library experience “a lot” include:

  • More spaces for quiet/individual study
  • More textbooks to check out for my classes
  • Additional specialized spaces for honors researchers, graduate students, or other student populations
  • More spaces for collaborative study

Survey data indicated a few differences between the services that 1G and continuing generation students believe to be important to their academics. The chart below shows services that 1G students more frequently listed as “important” than continuing generation students.

Survey responses of services that are important to students. Items shown are those that reflect a greater than 10%-pt difference between 1G and continuing generation students

Finding 6. It is sometimes hard to find existing resources at the Libraries.

When asked about services they wish they had known about earlier, our 1G focus group participants mentioned the Textbooks on Reserve program, library workshops (e.g., Matlab workshops), subject librarians, lockers available for short-term use, and the ability to reserve study rooms. Students described numerous library resources they have discovered seemingly by chance or long after their first semester at Duke. At multiple points during the focus groups students expressed that important services are not adequately marketed or shared with all 1G students. First-year 1G students reiterated the feeling of “unknown unknowns”: of understanding that many resources are available, but often finding it difficult to locate specific points of access.

Finding 7. Getting help from experts at the Libraries is important, but difficult.

1G students indicated that reaching out to library staff can be intimidating or even frightening. They described an initial barrier to asking for help, even while knowing it is likely the best way to receive assistance. Students noted feeling that their questions are “silly,” and they believe they have “gaps” in their knowledge. Students also reflected that it would be helpful for the person providing guidance to understand students’ lack of familiarity with library resources and services.

“We have a librarian for an English department, and for the Linguistics department… but it would be awesome to have a 1G librarian. Just someone who already knows that we don’t know anything, and it’s okay.”

Finding 8. Checking out books using call numbers is daunting.

Focus group participants frequently noted difficulty finding and checking out books using call numbers. 1G students did not pin this difficulty on library staff, but rather on their own lack of knowledge.

“The assumption is that we’ve been in libraries before.  They [library staff] were helpful after I admitted I didn’t know my way around.”

The stacks are an understandably daunting environment, especially for those unfamiliar with academic libraries. Students reported feeling supported once they made their confusion clear. Previous assessments have shown that the difficulty of understanding call numbers and finding materials in the Libraries is one experienced by many students, regardless of 1G status.

Finding 9. While students generally view the Libraries as a safe space, 1G students feel less strongly that this is true.

Some of the most striking differences in responses between 1G and continuing-generation students relate to the survey question asking the extent to which both Duke Campus and the Libraries feel like a safe space. For the purposes of the user survey, a “safe space” was defined as a place in which people can feel safe from discrimination, harassment, and any other emotional or physical harm.

Only 20% of 1G students “strongly agree” that Duke Campus is a safe space for them, compared to 36% of continuing-generation students. While it is a small percent, four times as many 1G students (4% compared to 1%) “strongly disagree” that campus is a safe space for them. Duke University has some work to do before all students, and especially 1G students, feel that it is a safe space.

Both 1G and continuing generation students feel strongly that the Libraries are more of a safe space than Duke University. This is encouraging, as a major goal of the Libraries is to provide a welcoming space for all. Differences in feelings about the Libraries as a safe space between 1G and non-1G are less stark but still present: 52% of 1G students “strongly agree” that the Libraries are a safe space compared to 61% of continuing-generation students.

Summary

First-generation students are resilient and successful members of the Duke community. The early years on campus, which involve finding the right communities for support and learning new academic skills, can be a difficult transition for some. While all students experience challenges in college, 1G students may not have access to certain sources of information capital and can have significant financial stressors that are difficult for many peers to understand. The Duke University Libraries are well poised to support the success of 1G students on campus. Library staff can help reduce the burdens associated with transitioning from high school to college by making academic and research support known to students early and often, providing access to cost-prohibitive textbooks, and continuing to make the Duke Libraries a welcoming space for all students.

What’s next?

These findings became the basis of the 19 recommendations outlined in the research team’s full report. For example, one important recommendation was to expand the Textbooks on Reserve program. Though the library already had a pilot program, it became clear that all students would benefit from expanding the program to include more textbooks and increasing marketing of the program. This fall the program expanded to include textbooks from the 100 largest courses on campus, and the Libraries has already seen an increase in student use of these books. There was also a recommendation that a librarian be designated as a 1G Student Success Librarian as a way to build the ecosystem of supportive offices and people described in focus groups. Arianne Hartsell-Gundy is currently serving in this role as a way to coordinate the libraries’ efforts, make connections with other programs and departments providing support, and serve as a point of contact for 1G students.

Additionally, The Libraries formed a 1G Study Recommendations Implementation Team (headed by the 1G Student Success Librarian) to prioritize recommendations and work across the Libraries to improve services, library instruction, and marketing/outreach to 1G students. One of the team’s first projects was to increase the library presence during the Rubenstein Scholars summer program. In addition to providing a library instruction session and one-on-one appointments with the students in this program, librarians attended a poster session and a mixer as a way to increase their presence. Also, the team is engaging with the staff dedicated to working on our service desks to find ways to help students feel more comfortable asking questions and navigating our book stacks. The team is pleased with their progress thus far and looks forward to finding new ways to connect with and support 1G students.

By: Joyce Chapman, Brenda Yang, Arianne Hartsell-Gundy, Emily Daly

Textbooks and Bean Bags: The 2018 Student Library Satisfaction Survey

This spring, Duke University Libraries conducted the 2018 biennial user satisfaction survey, a large survey of students and faculty at Duke. The goal of the survey is to gauge overall user satisfaction and to gather specific ideas for improvements to DUL materials, services, and spaces. In this post, we’ll share some of the trends within the student responses.

Survey methodology

Since 2013, DUL has created custom surveys rather than use generic survey products, allowing us to customize questions to different patron groups and even different parts of the campus libraries system. Developing and analyzing the results of a customized survey, however, is no small feat! The survey is run every two years, in part because the full cycle of survey development, dissemination, analysis, and follow-up takes the entire two years.

The 2018 survey was deployed in January 2018. A sample of students and faculty received personal invitations over email, but the survey was also advertised on the DUL website and open to anyone. We received responses from 2,610 students. We don’t have full demographic information for everyone, but approximately 54% of the students for whom we have demographics were undergraduates. The survey took approximately five to seven minutes to complete.

Two pie charts. One pie chart shows the distribution of the total 2,610 student participants (48% undergraduate, 41% graduate, 12% unknown). The second pie chart shows the distribution of the 2,307 participants that don't include unknown (54% undergraduate, 46% graduate).
After the survey closed, a group of seven staff at DUL divided up approximately 3,600 free-text responses and manually coded them for topic and, where appropriate, whether they were a request for a new service or change in existing policy or a compliment. The survey data have been visualized in a series of public dashboards. To gather additional information about some of the results, the Assessment & User Experience department also hosted several follow-up focus groups with both students and faculty. The focus group results, while not incorporated into the survey dashboards, have been incorporated into summary reports and recommendations.

The good

“I think the library is one of the places of greatest mutual respect on campus. There is less social stratification and freer flow of interaction. I enjoy my time in the library quite a lot.”

The survey included questions that everyone answered and questions that were specific to different libraries. All survey participants identified which library they visited most frequently. For students, 77% selected Perkins & Bostock as their primary libraries. Only 3% (76 students) reported that they don’t physically visit a library.

A bubble chart showing the libraries visited most frequently by students. Perkins & Bostock Libraries are highest with 77%, followed by Lilly Library with 11% and Divinity Library with 6%.

The libraries are considered an important part of the Duke experience by over 80% of participants. Focusing on the students who picked Perkins & Bostock as their primary libraries, we can look at usage of and satisfaction with the library. Of the 1,978 students who responded, over 80% visit Perkins & Bostock at least once a week. And by and large, students are quite satisfied with Perkins & Bostock. Less than 1% of responses fall in the “not satisfied at all” or “not very satisfied” categories, and the vast majority are very satisfied.

Three related bar charts. The first bar chart shows responses to a question asking students to agree that the library is an important part of their experience. 30% of students selected "somewhat agree," and 51% selected "strongly agree." A second chart shows that, for the Perkins & Bostock Libraries, 18% of students visit once a week, 38% visit more than once a week, and 25% visit daily. The third chart shows that for overall satisfaction with Perkins & Bostock, 14% are somewhat satisfied, 65% are very satisfied, and 21% are extremely satisfied.

The Duke University Libraries value diversity of thought, perspective, experience, and background and are actively committed to a culture of inclusion and respect. Beyond gauging user satisfaction, this year we also asked students about their impressions of Duke and DUL as safe spaces. (In the survey, “safe space” was defined as “a place in which people can feel safe from discrimination, harassment, and any other emotional or physical harm.”) We were excited to find that overall students agree that DUL is a safe space (92% respond with “agree” or “strongly agree”), even more than they agree that Duke University as a whole is a safe space (78% response with “agree” or “strongly agree”). Similarly, when asked if the library is a welcoming place, almost 90% agreed. Despite these encouraging numbers, we are committed to continuing to improve in this area wherever we can.

Three related bar charts. The first chart shows that when asked if Duke University is a safe space, 44% somewhat agree and 34% strongly agree. The second chart shows that when asked if the Duke Libraries are a safe space, 32% somewhat agree and 60% strongly agree. The third chart shows that when asked if the library is a welcoming place, 35% somewhat agree and 53% strongly agree.

The So-So

“I use the libraries a lot to study (esp Bostock) with friends, which is both helpful for me academically and comforting for me socially. The libraries fills up pretty often during busy times, so I wonder if more chairs would help accommodate more students (not even more tables, just more seating). Thanks!”

Even though by-and-large students are satisfied with the libraries, they were not afraid to let us know what areas could be improved! They gave us their constructive criticism in a few ways. First, we asked students to offer their opinions on the possibility of expanding different types of library services. Next, we asked how important specific services, materials, and spaces were, as well as how they were meeting the students’ needs. Finally, we gave them the opportunity to offer additional comments about DUL and suggestions on how to make DUL more of a safe space.

When we asked students what services should be expanded, students were most likely to vote for more spaces for individual study, more spaces for collaborative study, and more textbooks to check out. A second tier of requests include better signage, delivery of items between campuses, lockers, and help with digital scholarship.

A stacked bar chart showing the results to a question about the desirability of various specific services. The most desirable service is "more spaces for quiet or individual study," which 88% of responding students said would improve their experience either a little or a lot. Second is "more spaces for collaborative study" with 80% of responding students. Third is "more textbooks to check out for my classes" with 71% of responding students.

Looking at library-specific responses, we can find a bit more detail about these requests. When looking for services that are both important and not meeting students’ needs, we can see that reservable project/study rooms, a variety of seating options,  adequate quiet study space, and textbooks on reserve all appear in the high quadrant for both importance and not meeting students’ needs.

A scatterplot showing that four services are high on importance and on the percentage of students whose needs are not met: adequate quiet study space; variety of seating options; reservable study/project rooms; and print book, textbooks and articles on reserve for classes.

While not every student followed up on these questions with free-text explanations, the analysis of the free-text comments are consistent with these results. Of the 769 student comments that included requests for new services or a change in policy (rather than compliments), the top code was study/research space, which accounts for approximately 12% of the total requests. The second most frequent code was noise (about 9.5% of the requests), clarifying some of the complaints about “adequate quiet study space.” Requests often include a desire for the Libraries’ quiet space policies to be better enforced. The third most frequent code was atmosphere/sense of welcome – e.g., how inviting the library feels, feelings of “stress in the air.” This code was applied to just over 8% of the requests.

A bar chart showing the top 10 topical tags for requests, including study/research space (about 12%), noise (about 9.5%), and atmosphere/sense of welcome (just over 8%)

Security, furniture, advertising, and signage also ranked highly among requests. Students seem especially desirous of “comfortable” seating; write-in comments mention several types of comfortable seating by name, including couches and bean bags.

The Unknown

“Having taken this survey, I have realized that there are many things which the Duke University Libraries offer which I am not currently taking advantage of…”

While student needs and reactions change over time, one thing remains the same: they unknowingly request services we already offer. Sometimes the survey itself alerts students to particular services.

When we ask students how certain services are meeting their needs or which services should be expanded, we offer a choice labelled “I didn’t know the the library provided this.” Here are some of our most pressing “marketing opportunities,” according to the number of people who were unaware of the service.

A dumbbell plot showing the percentage of students who didn't know about a service, split into those who did find the service important to their research, coursework or teaching and those who did not. Regardless of importance, the service with the highest percentage of students who did not know about it was "support for using, analyzing, and visualizing data." Second highest was "self-checkout stations," followed by "scheduled assistance from library staff."

For each service, there are two values – one for the students who marked the service as important and another for those who didn’t. As might be expected, awareness is always lower among students who don’t find the service important, but there are also services that have lower awareness overall. Services like support analyzing data, self-checkout stations, meetings with library staff, and reservable interview rooms may be good candidates for increased marketing. (If you look at the previous scatterplot, you’ll see that reservable interview rooms also had a high value for students whose needs weren’t being met, even though it’s not rated very highly on importance.)

Another good indicator of marketing opportunities is our analysis of the students’ free-text comments. Some of the major requests from students actually match up well with some of our existing but possibly under-advertised services.

We already know that students are always on the lookout for quiet study spaces. This need is especially pronounced for graduate students, who seem to feel outnumbered by undergraduates, who need quiet space for long periods to work on independent research projects, and who don’t always have private office space elsewhere on campus. When we asked students about services they would like us to expand, we offered them the opportunity to comment on “Additional specialized spaces for honors researchers, graduate students, or other student populations.” Out of 281 total comments on additional specialized spaces, 142 (or almost 51%) mentioned graduate students. In analyzing the comments and in follow-up focus groups with graduate students, however, it appears that many are not aware of either one or both of the dedicated graduate student spaces in Perkins Library.

Riess Graduate Student Reading Room

a photo from outside a room, showing 2 large tables with 8 chairs each and outlets along the top. The room has windows along the back wall and a keypad on the door.

The graduate reading room is a shared reading space for graduate students on the 2nd floor of Perkins. It has a key pad entry code that can be obtained from the Library Service Desk. The room is has good natural lighting and is an “absolutely quiet” zone. Some of the requests indicate that students would like more individual desks, however, so some students may be unsatisfied with this as the only dedicated space open to all graduate students.

Graduate Research Commons

a photo of a room with many cubicles and lockers and a few windows in the back

In the spring of 2016, a large room on the second floor of Perkins was converted into the Graduate Research Commons. The space has 27 individual cubicles of two different heights, adjustable sit-stand desks, and dedicated lockers for all users. The room also includes a technology center with an e-Print terminal, a scanner, and a desktop computer with the Adobe software suite.

Unlike the Graduate Reading Room, however, students must apply for access to the Graduate Research Commons. Despite its many features, the space has been underutilized, and it appears that many students are not familiar with it and have never tried to apply for access. A review of this space could reveal ways to market and set policies for the space.

Next Steps

To determine the most needed and feasible improvements for follow-up, the Assessment & User Experience department will host a DUL-wide staff workshop in July to review the results and make specific recommendations to improve the experience of all of our users. Contact us if you would like information more about this workshop.

We look forward to sharing more of our progress on this and other assessment projects for DUL in the future!