Hang in there, the migration is coming

Detail from Hugh Mangum photographs - N318
Wouldn’t you rather read a post featuring pictures of cats from our digital collections than this boring item about a migration project that isn’t even really explained? Detail from Hugh Mangum photographs – N318.

While I would really prefer to cat-blog my merry way into the holiday weekend, I feel duty-bound to follow up on my previous posts about the digital collections migration project that has dominated our 2016.

Since I last wrote, we have launched two more new collections in the Fedora/Hydra platform that comprises the Duke Digital Repository. The larger of the two, and a major accomplishment for our digital collections program, was the Duke Chapel Recordings. We also completed the Alex Harris Photographs.

Meanwhile, we are working closely with our colleagues in Digital Repository Services to facilitate a whole other migration, from Fedora 3 to 4, and onto a new storage platform. It’s the great wheel in which our own wheel is only the wheel inside the wheel. Like the wheel in the sky, it keeps on turning. We don’t know where we’ll be tomorrow, though we expect the platform migration to be completed inside of a month.

hang-in-there-baby-kitten-poster
A poster like this, with the added phrase “Friday’s coming,” used to hang in one of the classrooms in my junior high. I wish we had that poster in our digital collections.

Last time, I wrote hopefully of the needle moving on the migration of digital collections into the new platform, and while behind the scenes the needle is spasming toward the FULL side of the gauge, for the public it still looks stuck just a hair above EMPTY. We have two batches of ten previously published collections ready to re-launch when we roll over to Fedora 4, which we hope will be in June – one is a group of photography collections, and the other a group of manuscripts-based collections.

In the meantime, the work on migrating the digital collections and building a new UI for discovery and access absorbs our team. Much of what we’ve learned and accomplished during this project has related to the migration, and quite a bit has appeared in this blog.

Our Metadata Architect, Maggie Dickson, has undertaken wholesale remediation of twenty years’ worth of digital collections metadata. Dealing with date representation alone has been a critical effort, as evidenced by the series of posts by her and developer Cory Lown on their work with EDTF.

Sean Aery has posted about his work as a developer, including the integration of the OpenSeadragon image viewer into our UI. He also wrote about “View Item in Context,” four words in a hyperlink that represent many hours of analysis, collaboration, and experimentation within our team.

I expect, by the time the wheel has completed another rotation, and it’s my turn again to write for the blog, there will be more to report. Batches will have been launched, features deployed, and metadata remediated. Even more cat pictures will have been posted to the Internet. It’s all one big cycle and the migration is part of it.

 

1940s & 1950s Chronicles Are Live!

The Digital Projects and Production Services is excited to announce that the 1940s and 1950s Chronicle are now digitized and accessible online at the Duke Chronicle Digital Collection.  These two new decades represent the next installment in a series of releases, which now completes a string of digitized Chronicles spanning from 1940 to 1989.

header
Headline from December 9, 1941
Army-soldiers
Army Finance Officers living at Duke, September 16, 1942

The 1940s and 1950s took Americans from WWII atrocities and scarcities to post-war affluence of sprawling suburbias, mass consumerism, and the baby boom.  It marked a time of changing American lifestylesa rebound from the Great Depression just ten years before.  At Duke, these were decades filled with dances and balls and Joe College Weekends, but also wartime limitations.  

ODK
Omicron Delta Kappa Fraternity Symbol, November 22, 1940

A year before the Japanese bombed Pearl Harbor, Duke lost its president of thirty years, William Preston Few.  The Chronicle reported Few to be “a remarkable man” who “worked ceaselessly towards [Duke University’s] growth” during a time when it was “a small, practically unheard-of college.”  While Duke may have been relatively small in 1940, it boasted a good number of schools and colleges, and a lively social scene.  Sorority and fraternity events abounded in the 1940s and 1950s.  So, too, did fights to overhaul the fraternity and sorority rushing systems.  Social organizations and clubs regularly made the Chronicle’s front page with their numerous events and catchy names, like Hoof ‘n’ Horn, Bench ‘n’ Bar, and Shoe ‘n’ Slipper.  These two decades also saw milestone celebrations, like the Chronicle’s 50th anniversary and the 25th Founders’ Day celebration.

wrong-ram
Duke captures the wrong ram, November 13, 1942

Sports was another big headliner.  In 1942, Duke hosted the Rose Bowl.  Usually played in Pasadena, California, the game was moved to Durham for fear of a Japanese attack on the West Coast during World War II.  The 1940s also saw the rivalry between Duke and UNC escalate into violent outbursts.  Pranks became more destructive and, in 1945, concerned student leaders pleaded for a “cease-fire.”  Among the pranks were cases of vandalism and theft.  In 1942, Duke “ramnappers” stole what they believed to be Carolina’s ram mascot, Rameses.  It was later discovered they heisted the wrong ram.  In 1949, unknown assailants painted the James B. Duke statue in Carolina blue, and Duke administration warned students against retaliation.  As one article from 1944 informs us, the painting of Duke property by UNC rivals was not a new occurrence, and if a Carolina painting prankster was captured, the traditional punishment was a shaved head.  In an attempt to reduce the vandalism and pranks, the two schools’ student governments introduced the Victory Bell tradition in 1948 to no avail.  The pranks continued into the 1950s.  In 1951, Carolina stole the Victory Bell from Duke, which was returned by police to avoid a riot.  It was again stolen and returned in 1952 after Duke’s victory over Carolina.  That year, the Chronicle headline echoed the enthusiasm on campus:  BEAT CAROLINA!  I urge you to explore the articles yourself to find out more about these crazy hijinks!

The articles highlighted here are only the tip of the iceberg.  The 1940s and 1950s Chronicles are filled with entertaining and informative articles on what Duke student life was like over fifty years ago.  Take a look for yourself and see what these decades have to offer!

Communication in Practice

The SNCC Digital Gateway is a collaborative, Mellon-funded project to document the history and legacy of the Student Nonviolent Coordinating Committee on a digital platform. One of the challenges of this undertaking is the physical distance between many of the project partners. From Washington, D.C. to St. Cloud, MN and Durham, NC to Rochester, NY, the SNCC veterans, scholars, librarians, and staff involved in the SNCC Digital Gateway Project are spread across most of the country. We’ve had collaborators call in anywhere from grocery stores in Jacksonville to the streets of Salvador da Bahia. Given these arrangements and the project’s “little d” democracy style of decision-making, communication, transparency, and easy access to project documents are key. The digital age has, thankfully, given us an edge on this, and the SNCC Digital Gateway makes use of a large selection of digital platforms to get the job done.

Trello

Say hello to Trello, an easy-to-use project management system that looks like a game of solitaire. By laying cards in different categories, we can customize our to-do list and make sure we have a healthy movement between potential leads, what’s slated to be done, and items marked as complete. We always try to keep our Trello project board up-to-date, making the project’s progress accessible to anyone at anytime.

While we use Trello for as a landing board for much of our internal communication, Basecamp has come in handy for our work with Digital Projects and our communication with the website’s design contractor, Kompleks Creative. Basecamp allows us to have conversations around different pieces of project development, as we provide feedback on design iterations, clarify project requirements, and ask questions about the feasibility of potential options. Keeping this all in one place makes this back-and-forth easy to access, even weeks or months later.

Screen Shot 2016-05-10 at 11.01.54 AM

 

Much of the project’s administrative documents fall into Box, a platform available through Duke that is similar to Dropbox but allows for greater file security. With Duke Toolkits, you can define a project and gain access to a slew of handy features, one of which is a project designation within Box (giving you unlimited space). That’s right, unlimited space. So, apart from allowing us to organize all of the many logistical and administrative documents in a collective space, Box is able to rise to the challenge of large file sharing. We use Box as a temporary landing platform through which we send archival scans, videos, audio recordings, and other primary source material to project partners.

Screen Shot 2016-05-10 at 11.02.41 AM

With the student project team, we’re also producing hundreds of pages worth of written content and look to Google Drive as our go-to for organization, access, and collaborative editing. Upon the completion of a set of drafts, we hold a workshop session where other members of the project team comment, critique, and contribute their knowledge. After a round of edits, drafts then go to SNCC veteran and former journalist Charlie Cobb, who puts red pen to paper (figuratively). With one more round of fact-checking and source logging, the final drafts are ready for the website.

And who doesn’t like to see the face of who they’re talking to? We make good use of Skype and Google Hangouts for long distance calls, and Uber Conference when we need to bring a lot of people into the conversation. And finally, an ongoing volley of e-mails, texts, and phone calls between individual project partners helps keep us on the same page.

While non-exhaustive, these are some of the digital platforms that have helped us get to where we are today and maintain communication across continents in this intergenerational and interdisciplinary collaboration.

Preservation Architecture: Phase 2 – Moving Forward with Duke Digital Repository

 

DukeSpace circa 2013
DukeSpace circa 2013

 

In 2013, the average price for a gallon of gas was $3.80, President Obama was inaugurated for a second term, and Duke University Libraries offered DukeSpace as an institutional repository.  Some things haven’t changed much, but the preservation architecture protecting the digital materials curated by the Libraries has changed a lot!

We still provide DukeSpace, but are laying the foundation to migrate collections and processes to the Duke Digital Repository (DDR).  The DDR was conceived of and developed as a digital preservation repository, an environment intended to preserve and sustain the rich digital collections; university scholarship and research data; purchased collections, and history of Duke far into the future.  Only through the grace of our partnership with Digital Projects and Production Services has the DDR recently also become a site that no longer hurts the eyes of our visitors.

The Duke Digital Repository endeavors to protect our assets from a large and diverse threat model. There are threats that are not addressed in the systems model presented here, such as those identified in the SPOT Model for Risk Assessment, of course. We formally consider these baseline threats to include:

  • Natural disasters including accidents at our local nuclear power station, fire, and hurricanes
  • Data degradation also known as bit rot or bit decay
  • External actors or threats posed by people external to the DDR team including those who manage our infrastructure
  • Internal actors including intentional or unintentional security risks and exploits by privileged staff in the libraries and supporting IT organizations

Phase 1 of our ingress into digital preservation established that DSpace, the software powering DukeSpace, was not sufficient for our needs, which led to an environmental scan and pilot project with Fedora and then Fedora and Hydra. This provided us with some of the infrastructure to mitigate the threats we had identified, but not all.  In Phase 1 we were to perform some important preservation tasks including:

  • Prove authenticity by offering checksum fixity validation on ingest and periodically
  • Identify and report on data degradation
  • Capture context in the form of descriptive, administrative, and technical metadata
  • Identify files in need of remediation using file characterization tools

Phase 2 allows us to address a greater range of threats and therefore offer a higher level of security to our collections.  In Phase 2 we’re doing several concurrent migrations including migrating our archival storage to infrastructure that will allow for dynamic resizing, de-duplication, and block-level integrity checking; moving to a horizontally scaled server architecture to allow the repository to grow to meet increasing demands of size (individual file size and size of collection) and traffic; and adopting a cloud replication disaster recovery process using DuraCloud to replace our local-only disk/tape infrastructure.  These changes provide significant protection against our baseline threat model by providing geographic diversity to our replicas, allowing us to constantly monitor the health of our 3 cloud replicas, and providing administrative diversity to the management of our replicas ensuring no single threat may corrupt all 4 copies of our data.

More detail about the repository architecture to come.

 

Digitizing Divinity: New Duke Chapel Recordings Digital Collection

Allow me to introduce the new, and delightfully improved…

…Drumroll please!…

Duke Chapel Nave
Duke Chapel Nave

Duke Chapel Recordings Digital Collection!  

I think I speak for all of us in the Digital Collections Program when I say how excited we are to roll out this complex collection of digitized audio, video, and manuscripts that document sermons at Duke Chapel from the 1940s to early 2000s.  You can now watch, listen to, and read sermons given at the Chapel by an array of preachers, including Duke Divinity faculty, and notable female and African American preachers.  Many of the recordings contain full worship services complete with music by the Chapel’s 100-voice choir and four pipe organs.  There are also special services, such as Martin Luther King, Jr. memorials, Good Fridays and Christmas Eves, Baccalaureates, and Convocations.  

Digitization of this collection was made possible through our collaboration with Duke University’s Divinity School, Duke Chapel, University Archives, and Duke University Libraries’ Digital Collections Program.  In 2015, the Divinity School received a Lilly Endowment Grant that funded the outsourcing of A/V digitization through two vendors, The Cutting Corporation and A/V Geeks, and the in-house digitization of the printed sermons.  The grant will also support metadata enhancements to improve searchability and discovery, like tagging references within the recordings to biblical verses and liturgical seasons.  The Divinity School will tackle this exciting portion of the project over the next two years, and their hard work will help users search deeper into the content of the collection.

Duke Chapel, September 1950
Duke Chapel, September 1950

Back in 2014, digital collections program manager, Molly Bragg, announced the release of the first installation of digitized Duke Chapel Recordings.  It consisted of 168 audio and video items and a newly developed video player.  This collection was released in response to the high priority Duke Chapel placed on digitization, and high demand from patrons to digitize and view the materials.  Fast forward two years and we have upped our game by expanding the collection to over 1,400 audio and video items, and adding more than 1,300 printed sermon manuscripts.  Many of the printed sermons match up to a recording, as they are often the exact document the preacher used to deliver their sermon.  The online content now represents a large percentage of the original materials held in the Duke University Archives taken from the Duke University Chapel Recordings and Duke Chapel Records collections.  Many of the audio reels were not included in the scope of the project and we hope to digitize these in the near future.

Divinity student delivers practice sermon before faculty and students, undated
Divinity student delivers practice sermon before faculty and students, undated

The Lilly Grant also provided funding to generate transcriptions of the audio-visual items, which we outsourced to Pop Up Archive, a company that specializes in creating timestamped transcripts and tags to make audio text searchable.  Once the transcriptions are generated by Pop Up Archive and edited by Divinity students, they will be made available on the web interface alongside the recordings.  All facets of this project support Divinity’s Duke Preaching Initiative to enhance homiletical education and pedagogy.  With the release of the Duke Chapel Recordings Digital Collection, the Divinity School now has a great classroom resource to help students learn about the art of sermon writing and delivery.

The release of the Chapel Recordings marks yet another feat for the Digital Collections Program.  This is the first audio-visual collection to be published in the new Tripod3 platform in conjunction with the Digital Collections migration into the Duke Digital Repository (see Will Sexton’s blog posts on the migration).  Thanks to the hard work of many folks in the Digital Repository Services and Digital Projects and Production Services, this means for the user a new and squeaky clean interface to browse the collection.  With the growing demand to improve online accessibility of audio-visual materials, Chapel Recordings has also been a great pilot project to explore how we can address A/V transcription needs across all our digital collections.  It has presented us all with many challenges to overcome and successes to applaud along the way.

Chapel scene, 1985
Chapel scene, 1985

If you’re not intrigued by the collection already, here are some sermon titles to lure you in!

From contemplative, to entertaining, to historical, the sermons cover a broad range of topics. I encourage you to take a look and a listen!

Digitizing for Exhibits

While most of my Bitstreams posts have focused on my work preserving and archiving audio collections, my job responsibilities also include digitizing materials for display in Duke University Libraries Exhibits.  The recent renovation and expansion of the Perkins Library entrance and the Rubenstein Library have opened up significantly more gallery space, meaning more exhibits being rotated through at a faster pace.

gallery2

Just in the past year, I’ve created digital images for exhibits on Vesalius’s study of human anatomy, William Gedney’s photographs, Duke Chapel’s stained glass windows, and the 1793 Yellow Fever epidemic.  I also worked with a wide range of materials spanning “books, manuscripts, photographs, recordings and artifacts that document human aspirations” for the Dreamers and Dissenters exhibit celebrating the reopening of the newly renovated David M. Rubenstein Rare Book & Manuscript Library.  The digital images are used to create enlargements and facsimiles for the physical exhibits and are also used in the online “virtual exhibits.”

 

Working with such a variety of media spanning different library collections presents a number of challenges and necessitates working closely with our Exhibits and Conservation departments.  First, we have to make sure that we have all of the items listed in the inventory provided by the exhibit curator.  Secondly, we have to make sure we have all of the relevant information about how each item should be digitally captured (e.g. What image resolution and file specifications?  Which pages from a larger volume?  What section of a larger map or print?)  Next we have to consider handling for items that are in fragile condition and need special attention.  Finally, we use all of this information to determine which scanner, camera, or A/V deck is appropriate for each item and what the most efficient order to capture them in is.

All of this planning and preliminary work helps to ensure that the digitization process goes smoothly and that most questions and irregularities have already been addressed.  Even so, there are always issues that come up forcing us to improvise creative solutions.  For instance:  how to level and stabilize a large, fragile folded map that is tipped into a volume with tight binding?  How to assemble a seamless composite image of an extremely large poster that has to be photographed in multiple sections?  How to minimize glare and reflection from glossy photos that are cupped from age?  I won’t give away all of our secrets here, but I’ll provide a couple examples from the Duke Chapel exhibit that is currently on display in the Jerry and Bruce Chappell Family gallery.

angel

This facsimile of a drawing for one of the Chapel’s carved angels was reproduced from an original architectural blueprint.  It came to us as a large and tightly rolled blueprint–so large, in fact, that we had to add a piece of plywood to our usual camera work surface to accommodate it.  We then strategically placed weights around the blueprint to keep it flattened while not obscuring the section with the drawing.  The paper was still slightly wrinkled and buckled in places (which can lead to uneven color and lighting in the resulting digital image) but fortunately the already mottled complexion of the blueprint material made it impossible to notice these imperfections.

projection

These projected images of the Chapel’s stained glass were reproduced from slides taken by a student in 1983 and currently housed in the University Archives.  After the first run through our slide scanner, the digital images looked okay on screen, but were noticeably blurry when enlarged.  Further investigation of the slides revealed an additional clear plastic protective housing which we were able to carefully remove.  Without this extra refractive layer, the digital images were noticeably sharper and more vibrant.

Despite the digitization challenges, it is satisfying to see these otherwise hidden treasures being displayed and enjoyed in places that students, staff, and visitors pass through everyday–and knowing that we played a small part in contributing to the finished product!

 

Digital Resources: Allen Building Activism

allen bldg study-in 13Nov1967-2
Allen Building Study-In, 13 November 1967

Duke University has a long history of student activism, and the University Archives actively collects materials to document these movements.  With the administration’s offices residing in the Allen Building, this is not the first time it is the center of activism activity.  The Allen Building Study-In occurred November 13, 1967, the Allen Building Takeover occurred February 13, 1969, and the Allen Building Demonstration occurred in May 1970 to support the Vietnam Moratorium.  In light of the current occupation of the Allen Building, we’ve compiled some digital resources you can use to find out more about the history of activism in relation to the 1969 Allen Building Takeover.

Allen Building Demonstration, 6 May 1970
Allen Building Demonstration, 6 May 1970

The University Archives has a collection of materials from the 1969 Allen Building Takeover, which includes many digitized images available through the online finding aid.  This collection also has materials from the 2002 Allen Building lock-in that commemorated 1960s activism at Duke:  Guide to the Allen Building Takeover Collection, 1969-2002.

WDBS, Duke University’s campus radio station at the time of the 1969 Allen Building Takeover, also broadcasted reports on the event.  Listening copies of these recordings are in the Allen Building Takeover Collection, and a list of the broadcasts can be found in the WDBS Collection:  Guide to the WDBS Collection, 1949-1983.

The Duke Chronicle also documented the event, and these images are digitized and available through the Guide to the Allen Building Takeover Collection, and also online through the library’s digital collections.  Check out the week of February 13th:  1969 issues of Duke Chronicle Digital Collection.

Allen Building Takeover supporters being tear-gassed, 13 February 1969
Allen Building Takeover supporters being tear-gassed, 13 February 1969

Lastly, oral history interviews were conducted by Don Yannella for his 1985 senior honors thesis, Race Relations at Duke University and the Allen Building Takeover.  These now make up the Allen Building Takeover Oral History Collection in the University Archives:  Guide to the Allen Building Takeover Oral History Collection.

Allen Building Takeover, 13 February 1969
Allen Building Takeover, 13 February 1969

Currently, University Archives is documenting the present Allen Building occupation, and has captured over 7,000 tweets with #DismantleDukePlantation.  To ensure that Duke activism will continue to be represented in the archives, efforts will be made to collect additional materials related to the occupation.

Looking to the Future of the Duke Digital Repository: Defining a Program for Digital Preservation, Management & Access

Our modern day lives and professional endeavors are teeming with digital output.  We participate in the digital ecosystem every day, contributing our activities, our scholarship, and our work in new and evolving ways.  Some of that contribution gets lost in the Internet ether, and some gets saved, or preserved, in specific, often localized ways that are neither sustainable nor preservable for the long haul.  We here at the Duke University Libraries, want to be able to look to the future with confidence, knowing that we have a game plan for capturing and preserving digital objects that are necessary and vital to the university community.  Queue the new Duke Digital Repository.  

DDR

The Duke Digital Repository is a software development initiative undertaken by the Digital Repository Services department in the Duke University Libraries.  It is a preservation repository architected using the Fedora Open Source software project, which is intended to replace the current manifestation of our institutional repository, Duke Space.  It is a superior product that is provisioned specifically for the preservation, storage, and access of digital objects.  The Duke Digital Repository is fully operational; we are now in the process of refining user interfaces, ingesting new and varied collections, and assessing descriptive metadata needs for ingested collections.  

fed

So what’s next?  Well we’ve got the Duke Digital Repository as a platform, now we need the Duke Digital Repository as a program.  We need to clarify the services and support that we offer to the university community, we need to fully define its stakeholders, and we need to implement an organizational structure to support a robust service.  

Here are just a few things that we’re engaged in that are seeking to define our user groups and assess their needs in a preservation platform and digital support service.  Defining these expectations will allow us to take the next step in crafting a sustainable and relevant program to support the digital scholarship of the university.

  • ITHAKA Faculty Survey: In the Fall semester of 2015, the Libraries deployed the ITHAKA S+R Faculty Survey.  Faculty are considered a primary stakeholder of the repository, as it is well provisioned to meet their data management needs.  260 faculty members responded to the survey, sharing their thoughts on a variety of topics including scholarly communications services, research practices, data preservation and management needs, and much more.  There was a lot of valuable, actionable data contributed, which pertains directly to the repository as a preservation tool, and a service for data support.  The digital repository team is working through this data to identify and target needs and desires in a repository program.
  • Graduate & Undergraduate Advisory Boards:  The Digital Repository staff are also working with the Assessment & User Experience team within the library to reach out to graduate and undergraduate student constituents to capture their voice.  We have collectively identified a list of questions and prompts that will engage them in a discussion about their needs pertaining to the repository as a tool and a service.  From this discussion we are also gauging their understanding of ‘a repository’ and hoping to glean some information that will help us to understand how we might brand and market the repository more effectively.  
  • Fedora Community: Fedora is an open source software product developed and stewarded by the DuraSpace community.  The Duke University Libraries are active participants in the community which is essentially a consortium of academic institutions that are working toward a common goal of preserving intellectual, cultural, and scientific heritage.  We are reaching out to our community constituents to ask how other institutions similar to ours are supporting their repository  programs.  We’re assessing  various models of support and generating a discussion around repository support as a resourced program, rather than a simple software solution.  We are also working with Assessment & User Experience to conduct an environmental scan and literature review to gain greater insight and understanding of best practice.

special

In short, we want to make the repository special, and relevant to its users.  We want to feel confident that it provides a service that is valuable and necessary for our university community.  We invite your feedback as we embark on this effort.  For further information or to give us your feedback, please contact us: repository-help@duke.edu

Notes from the Duke University Libraries Digital Projects Team