We’re experimenting with changing our approach to projects in Software Development and Integration Services (SDIS). There’s been much talk of Agile (see the Agile Manifesto) over the past few years within our department, but we’ve faced challenges implementing this as an approach to our work given our broad portfolio, relatively small team, and large number of internal stakeholders.
After some productive conversations among staff and managers in SDIS where we reflected on our work over the past few years we decided to commit to applying the Scrum framework to one or more projects.
There are many resources available for learning about Agile and Scrum. The resources I’ve found most useful so far in learning about the framework include:
Scrum seems best suited to developing new products or software and defines the roles, workflow, and artifacts that help a team make the most of its capacity to build the highest value features first and deliver usable software on a regular and frequent schedule.
To start, we’ll be applying this process to a new project to build a prototype of a research data repository based on Hyrax. We’ve formed a small team, including a product owner, scrum master, and development team to build the repository. So far, we’ve developed an initial backlog of requirements in the form of user stories in Jira, the software we use to manage projects. We’ve done some backlog refinement to prioritize the most important and highest value features, and defined acceptance criteria for the ones that we’ll consider first. The development team has estimated the story points (relative estimate of effort and complexity) for some of the user stories to help us with sprint planning and release projection. Our first two-week sprint will begin the week after Thanksgiving. By the end of January we expect to have completed four, two-week sprints and have a pilot ready with a basic set of features implemented for evaluation by internal stakeholders.
One of the important aspects of Scrum is that group reflection on the process itself is built into the workflow through retrospective meetings after each sprint. Done right, routine retrospectives serve to reinforce what is working well and allows for adjustments to address things that aren’t. In the future we hope to adapt what we learn from applying the Scrum framework to the research data repository pilot to improve our approach to other aspects of our work in SDIS.
International Broadsides (added to migrated Broadsides and Ephemera collection): https://repository.duke.edu/dc/broadsides
Orange County Tax List Ledger, 1875: https://repository.duke.edu/dc/orangecountytaxlist
Radio Haiti Archive, second batch of recordings: https://repository.duke.edu/dc/radiohaiti
William Gedney Finished Prints and Contact Sheets (newly re-digitized with new and improved metadata): https://repository.duke.edu/dc/gedney
In addition to the brand new items, the digital collections team is constantly chipping away at the digital collections migration. Here are the latest collections to move from Tripod 2 to the Duke Digital Repository (these are either available now or will be very soon):
What we hoped would be a speedy transition is still a work in progress 2 years later. This is due to a variety of factors one of which is that the work itself is very complex. Before we can move a collection into the digital repository it has to be reviewed, all digital objects fully accounted for, and all metadata remediated and crosswalked into the DDR metadata profile. Sometimes this process requires little effort. However other times, especially with older collection, we have items with no metadata, or metadata with no items, or the numbers in our various systems simply do not match. Tracking down the answers can require some major detective work on the part of my amazing colleagues.
Despite these challenges, we eagerly press on. As each collection moves we get a little closer to having all of our digital collections under preservation control and providing access to all of them from a single platform. Onward!
At Duke University Libraries (DUL), we are embarking on a new way to propose digitization projects. This isn’t a spur of the moment New Year’s resolution I promise, but has been in the works for months. Our goal in making a change to our proposal process is twofold: first, we want to focus our resources on specific types of projects, and second, we want to make our efforts as efficient as possible.
Introducing Digitization Initiatives
The new proposal workflow centers on what we are calling “digitization initiatives.” These are groups of digitization projects that relate to a specific theme or characteristic. DUL’s Advisory Council for Digital Collections develops guidelines for an initiative, and will then issue a call for proposals to the library. Once the call has been issued, library staff can submit proposals on or before one of two deadlines over a 6 month period. Following submission, proposals will be vetted, and accepted proposals will move onto implementation. Our previous system did not include deadlines, and proposals were asked to demonstrate broad strategic importance only.
DUL is issuing our first call for proposals now, and if this system proves successful we will develop a second digitization initiative to be announced in 2018.
I’ll say more about why we are embarking on this new system later, but first I would like to tell you about our first digitization initiative.
Call for Proposals
Duke University Libraries’ Advisory Council for Digital Collections has chosen diversity and inclusion as the theme of our first digitization initiative. This initiative draws on areas of strategic importance both for DUL (as noted in the 2016 strategic plan) and the University. Prospective champions are invited to think broadly about definitions of diversity and inclusion and how particular collections embody these concepts, which may include but is not limited to topics of race, religion, class, ability, socioeconomic status, gender, political beliefs, sexuality, age, and nation of origin.
Proposals will be due on March 15, 2017 or June 15, 2017.
Proposing non-diversity and inclusion related proposal
We have not forgotten about all the important digitization proposals that support faculty, important campus or off campus partnerships, and special events. In our experience, these are often small projects and do not require a lot of extra conservation, technical services, or metadata support so we are creating an“easy” project pipeline. This will be a more light-weight process that will still requires a proposal, but less strategic vetting at the outset. There will be more details coming out in late January or February on these projects so stay tuned.
Why this change?
I mentioned above that we are moving to this new system to meet two goals. First, this new system will allow us to focus proposal and vetting resources on projects that meet a specific strategic goal as articulated by an initiative’s guidelines. Additionally, over the last few years we have received a huge variety of proposals: some are small “no brainer” type proposals while others are extremely large and complicated. We only had one system for proposing and reviewing all proposals, and sometimes it seemed like too much process and sometimes too little. In other words one process size does not not fit all. By dividing our process into strategically focussed proposals on the one hand and easy projects on the other, we can spend more of our Advisory committee’s time on proposals that need it and get the smaller ones straight into the hands of the implementation team.
Another benefit of this process is that proposal deadlines will allow the implementation team to batch various aspects of our work (batching similar types of work makes it go faster). The deadlines will also allow us to better coordinate the digitization related work performed by other departments. I often find myself asking departments to fit digitization projects in with their already busy schedules, and it feels rushed and can create unnecessary stress. If the implementation team has a queue of projects to address, then we can schedule it well in advance.
I’m really excited to see this new process get off the ground, and I’m looking forward to seeing all the fantastic proposals that will result from the Diversity and Inclusion initiative!
Made possible by the generous support of The Andrew W. Mellon Foundation, the SNCC Digital Gateway tells the story of how young activists in SNCC united with local people in the Deep South to build a grassroots movement for change that empowered the Black community and transformed the nation.
Using documentary footage, audio recordings, photographs, and documents, the site portrays how SNCC organizers, alongside thousands of local Black residents, worked so that Black people could take control of their lives. It unveils the inner workings of SNCC as an organization, examining how it coordinated sit-ins and freedom schools, voter registration and economic cooperatives, anti-draft protests and international solidarity struggles.
In this new documentary website, you’ll find:
Historic materials including documents, photographs, oral history interviews, and audiovisual material hosted in digital collections at repositories across the country
Profiles examining individuals’ contributions to the Movement
Inside SNCC pages unveiling the inner workings of SNCC as an organization
Perspectives presenting aspects of SNCC’s history from the eyes of the activists themselves
Map connecting users to the people who worked—and the events that happened—in a specific place.
In 2013, the SNCC Legacy Project (SLP) and Duke University formed a partnership to chronicle the historic struggles for voting rights and to develop ongoing programs that contribute to a more civil and inclusive democracy in the 21st century.
SNCC veterans shaped the vision and framework of the SNCC Digital Gateway. They worked collaboratively with historians of the Movement, archivists, and students to weave together grassroots stories, digitized primary source materials, and new multi-media productions to bring this history—and its enduring legacy—to life for a new generation.
The SNCC Digital Gateway is a work in progress. We will continue to add more stories and fill out its content in the year to come.
The story of the Movement told on this website is one of unsung heroes: domestic workers and sharecroppers, young organizers and seasoned mentors, World War II veterans and high school students. The SNCC Digital Gateway is here to share their story—and to help continue their legacy of organizing for self-determination and democracy in the generations to come. We feel certain that the site not only provides an unprecedented and valuable window onto past civil rights struggles, but a valuable tool for all those interested in social change today.
The 1905 to 1939 Chronicle issues are now live online at the Duke Chronicle Digital Collection. This marks the completion of a multi-year project to digitize Duke’s student newspaper. Not only will digitization provide easier online access to this gem of a collection, but it will also help preserve the originals held in the University Archives. With over 5,600 issues digitized and over 63,000 pages scanned, this massive collection is sure to have something for everyone.
The first two decades of the Chronicle saw its inception and growth as the student newspaper under the title The Trinity Chronicle. In the mid-1920s after the name change to Duke University, the Chronicle followed suit. In Fall of 1925, it officially became The Duke Chronicle.
The Nineteen-teens saw the growth of the university, with new buildings popping up, while others burned down – a tragic fire decimated the Washington Duke Building.
In the shadow of the Great Depression, the 1930s at Duke was a time to unite around a common cause – sports! Headlines during this time, like decades to follow, abounded with games, rivalries, and team pride.
Take the time to explore this great resource, and see how Duke and the world has changed. View it through the eyes of student journalists, through advertisements and images. So much occurred from 1905 to 1989, and the Duke Chronicle was there to capture it.
Post contributed by Jessica Serrao, former King Intern for Digital Collections.
The 1940s and 1950s took Americans from WWII atrocities and scarcities to post-war affluence of sprawling suburbias, mass consumerism, and the baby boom. It marked a time of changing American lifestyles—a rebound from the Great Depression just ten years before. At Duke, these were decades filled with dances and balls and Joe College Weekends, but also wartime limitations.
A year before the Japanese bombed Pearl Harbor, Duke lost its president of thirty years, William Preston Few. The Chronicle reported Few to be “a remarkable man” who “worked ceaselessly towards [Duke University’s] growth” during a time when it was “a small, practically unheard-of college.” While Duke may have been relatively small in 1940, it boasted a good number of schools and colleges, and a lively social scene. Sorority and fraternity events abounded in the 1940s and 1950s. So, too, did fights to overhaul the fraternity and sorority rushing systems. Social organizations and clubs regularly made the Chronicle’s front page with their numerous events and catchy names, like Hoof ‘n’ Horn, Bench ‘n’ Bar, and Shoe ‘n’ Slipper. These two decades also saw milestone celebrations, like the Chronicle’s 50th anniversary and the 25th Founders’ Day celebration.
Sports was another big headliner. In 1942, Duke hosted the Rose Bowl. Usually played in Pasadena, California, the game was moved to Durham for fear of a Japanese attack on the West Coast during World War II. The 1940s also saw the rivalry between Duke and UNC escalate into violent outbursts. Pranks became more destructive and, in 1945, concerned student leaders pleaded for a “cease-fire.” Among the pranks were cases of vandalism and theft. In 1942, Duke “ramnappers” stole what they believed to be Carolina’s ram mascot, Rameses. It was later discovered they heisted the wrong ram. In 1949, unknown assailants painted the James B. Duke statuein Carolina blue, and Duke administration warned students against retaliation. As one article from 1944 informs us, the painting of Duke property by UNC rivals was not a new occurrence, and if a Carolina painting prankster was captured, the traditional punishment was a shaved head. In an attempt to reduce the vandalism and pranks, the two schools’ student governments introduced the Victory Bell tradition in 1948 to no avail. The pranks continued into the 1950s. In 1951, Carolina stole the Victory Bell from Duke, which was returned by police to avoid a riot. It was again stolen and returned in 1952 after Duke’s victory over Carolina. That year, the Chronicle headline echoed the enthusiasm on campus: BEAT CAROLINA! I urge you to explore the articles yourself to find out more about these crazy hijinks!
The articles highlighted here are only the tip of the iceberg. The 1940s and 1950s Chronicles are filled with entertaining and informative articles on what Duke student life was like over fifty years ago. Take a look for yourself and see what these decades have to offer!
Digitization of this collection was made possible through our collaboration with Duke University’s Divinity School, Duke Chapel, University Archives, and Duke University Libraries’ Digital Collections Program. In 2015, the Divinity School received a Lilly Endowment Grant that funded the outsourcing of A/V digitization through two vendors, The Cutting Corporation and A/V Geeks, and the in-house digitization of the printed sermons. The grant will also support metadata enhancements to improve searchability and discovery, like tagging references within the recordings to biblical verses and liturgical seasons. The Divinity School will tackle this exciting portion of the project over the next two years, and their hard work will help users search deeper into the content of the collection.
Back in 2014, digital collections program manager, Molly Bragg, announced the release of the first installation of digitized Duke Chapel Recordings. It consisted of 168 audio and video items and a newly developed video player. This collection was released in response to the high priority Duke Chapel placed on digitization, and high demand from patrons to digitize and view the materials. Fast forward two years and we have upped our game by expanding the collection to over 1,400 audio and video items, and adding more than 1,300 printed sermon manuscripts. Many of the printed sermons match up to a recording, as they are often the exact document the preacher used to deliver their sermon. The online content now represents a large percentage of the original materials held in the Duke University Archives taken from the Duke University Chapel Recordings and Duke Chapel Records collections. Many of the audio reels were not included in the scope of the project and we hope to digitize these in the near future.
The Lilly Grant also provided funding to generate transcriptions of the audio-visual items, which we outsourced to Pop Up Archive, a company that specializes in creating timestamped transcripts and tags to make audio text searchable. Once the transcriptions are generated by Pop Up Archive and edited by Divinity students, they will be made available on the web interface alongside the recordings. All facets of this project support Divinity’s Duke Preaching Initiative to enhance homiletical education and pedagogy. With the release of the Duke Chapel Recordings Digital Collection, the Divinity School now has a great classroom resource to help students learn about the art of sermon writing and delivery.
The release of the Chapel Recordings marks yet another feat for the Digital Collections Program. This is the first audio-visual collection to be published in the new Tripod3 platform in conjunction with the Digital Collections migration into the Duke Digital Repository (see Will Sexton’s blog posts on the migration). Thanks to the hard work of many folks in the Digital Repository Services and Digital Projects and Production Services, this means for the user a new and squeaky clean interface to browse the collection. With the growing demand to improve online accessibility of audio-visual materials, Chapel Recordings has also been a great pilot project to explore how we can address A/V transcription needs across all our digital collections. It has presented us all with many challenges to overcome and successes to applaud along the way.
If you’re not intrigued by the collection already, here are some sermon titles to lure you in!
The 1970s are here! That is, in digital form. The Duke Chronicle digital collection now includes issues from the grooviest decade of the twentieth century.
The American memory of the 1970s is complex, wavering from carefree love to Vietnam and civil rights. As the social turmoil of the 1960s flowed into the 1970s, Terry Sanford was sworn in as president of Duke University. This marked the beginning of his sixteen-year term, but also marked the decade in which Sanford twice ran for president and partook in heated debates with Alabama governor George Wallace. He presided over the university In the midst of the Vietnam War and national protests, the Watergate scandal, and the aftermath of the Allen Building occupation in 1969.
In response to the demands from the Allen Building takeover, the Duke University community worked to improve social inequalities on campus. The 1972 incoming freshman class boasted more than twice as many black students than ever before in university history. Black Studies Program faculty and students struggle to create their own department, which became a controversial event on campus throughout the ‘70s. One Chronicle article even tentatively labeled 1976 as “The Year of the Black at Duke,” reflecting the strides made to incorporate black students and faculty into campus life and academics.
The 1970s was also a decade of change for women at Duke. In 1972, Trinity College and the Woman’s College merged, and not all constituents agreed with the move. Women’s athletics were also shaken by the application of Title IX implemented by the Department of Health, Education and Welfare (HEW), that prohibited discrimination on the basis of sex. This regulation significantly impacted the future of the Physical Education Department as well as women’s sports at Duke.
The addition of the 1970s to the Duke Chronicle digital collection marks a milestone for the Digital Projects and Production Services Department. We can now provide you with a complete run of issues from 1959 to 1989, and the 1950s will be heading your way soon! We invite you to explore the 1970s issues and see for yourself how history unfolded across the nation and across Duke campus.
Q: How is a silent H. Lee Waters film like an oral history recording?
A: Neither is text searchable.
But, leave it to oral historians to construct solutions for access to audiovisual resources of all stripes. No mistake, they’ve been thinking about it for a long time. Purposefully, profoundly non-textual at their creation, oral histories have since their postwar genesis contended with a central irony: as research they are exploited almost exclusively via textual transcription. Oral histories that don’t get transcribed get, instead, infamously ignored. So as the online floodgates have opened and digital media recorders and players have kept pace, oral historians have seen an opportunity to grapple meaningfully with closing the gap between the text and its source, and perhaps at the same time free the interview from the expectation that it should be transcribed.
Enter OHMS (http://www.oralhistoryonline.org/). In 2013, Doug Boyd at the University of Kentucky debuted the results of an IMLS-funded project to create the Oral History Metadata Synchronizer. A free, open-source tool, OHMS empowers even the smallest oral history archive to encode its media with textual information. The OHMS editor enables the oral historian to easily create item level metadata for an oral history recording, including an index or subject list that can drop a researcher into an interview at that selected point. OHMS can also timestamp an existing transcript, so that researchers can track the audio via the text. In its short life, OHMS has demonstrated a way to bridge the great divide among oral history theorists, which reads something like this: Should our focus be the audio or the transcript?
While it springs from the minds of oral historians, OHMS might more accurately be termed the Media Metadata Synchronizer. When I saw Doug’s presentation on OHMS at the Oral History Association meeting in 2013, two alternative uses immediately came to mind: OHMS had the potential to help us provide bilingual entry to the 3,500+ recordings in our Radio Haiti Collection (currently being digitzed), and it could dramatically enhance access to one of Duke’s great collections, the H. Lee Waters Films. Waters filmed his Movies of Local People in mostly smaller communities around North Carolina from 1936-1942, using silent reversal film stock. Waters’ effort to supplement his family’s income has over the intervening years become a major historical document of the state during the Great Depression. And yet as rich as the collection is, it is difficult for students, scholars, and filmmakers to find specific scenes or subjects among the thousands of two-second shots Waters put to film. Several years ago, an intern in the archive created shotlists for some of the films, but these existed independently of the films and were not terribly accurate in matching times since they were created using VHS tapes (and VHS players are notorious for displaying incorrect times). OHMS would give us the opportunity to update the shotlists we had and create some new ones, linking description to precise points within the films.
Implementing OHMS at Duke Libraries was a pleasure, mostly because I had the opportunity to work with my colleagues in Digital Projects and Production Services, an outstanding team that can do amazing things with our equally amazing archival resources. Recognizing the open-source spirit of OHMS, Sean Aery, Will Sexton, and Molly Bragg immediately saw how the system could help us get deeper into the Waters films without having to build out a complex infrastructure (or lay out lots of cash). And so, when the H. Lee Waters website went live last year with 35 hours of mostly undescribed digital video (although we did post those older shotlists too, where we had them), it was generally agreed that a phase two would happen sooner rather than later and include a pilot for OHMS shotlists. Rubenstein Audiovisual Intern Olivia Carteaux worked diligently through the spring to normalize existing shotlists and create new ones where possible. This necessitated breaking down the descriptive data we had into spreadsheets, so we could then “crosswalk” the description into the OHMS xml file that is at the heart of the system.
While the OHMS index viewer allows for metadata including title or description, partial transcript, segment synopsis, keywords, subjects, GPS coordinates and a link to a map, we concentrated on providing a descriptive sentence as the title and, where it was easy to find, the location of the action.
While on the face of it generating description for the H. Lee Waters films might seem fairly straightforward, we found a number of challenges in describing his silent moving images. For starters, given Waters’ quick edits, what would adequate frequency of description look like? A new descriptive entry at every cut would be extremely unwieldy. At the same time we recognized that without a spoken or textual counterpart to the image, every time we chose not to describe would deprive potential users of a “way in.” We settled on creating entries whenever the general scene or action changed; for instance, when Waters shifts from a scene on main street to one in front of a mill or school, or within the scene at a school when the action goes from schoolyard play to the pledge of allegiance. Sometimes the shifts are obvious, other times they are more subtle, so watching the action with a deep focus is necessary. We also created new entries whenever Waters created a trick shot, such as a split screen, a speed up or slow down of the action, a reverse shot, or a masking shot. Additionally, storefront signs, buildings, and landmarks also became good places to create entries, depending on their prominence; for these, too, we attempted to create GPS coordinates where we could easily do so.
Our second challenge was how much to invest in each description. “A picture is worth a thousand words” and “every picture tells a story” sum up much of the Waters footage, but brevity was of value to the workflow. One sentence, which did not have to be properly complete — a sort of descriptive bullet point — was decided on as our rule of thumb. In the next phase of this process I hope to use the keywords field more effectively, but that requires a controlled vocabulary, which brings me to our third challenge: normalizing description was the most difficult single piece of describing the films. Turns out there’s not a lot of library-based methodology for describing moving images, although there are general recommended approaches for describing images for the visually impaired. Then, of course, there’s the difficulty in deciding how to represent nuanced factors such as race, ethnicity, class, and gender. It is clear that in the event we undertake to create shotlists for all the Waters films, the first order of business will be to create a thesaurus of terms, to provide consistent description across the films.
When we felt like we had enough transformed shotlists for a pilot OHMS project for the Waters website, the OHMS player was loaded onto a server and the playlists uploaded. Links to the 29 shotlists were then placed below the video windows on their respective pages. To access the video and synchronized description, simply click on the link that says “Synchronized Shot List.” In this initial run we’re hoping to upload about 20 more shotlists, and at that point take a breath and see how we can improve on what we’ve accomplished. Given the challenges of presenting audiovisual resources online, there’s never really a “done,” only steady improvement. OHMS has provided what I believe is a clear step forward on access to the Waters films, and has the potential to help us transform other audiovisual collections into deeply mined treasures of the archive.
Post contributed by Craig Breaden, Audiovisual Archivist, Rubenstein Library
Yesterday was Back to the Future day, and the Internet had a lot of fun with it. I guess now it falls to each and every one of us, to determine whether or not today begins a new future. It’s certainly true for Duke Digital Collections.
The next few months will see a great deal more work on the project. We have new collections scheduled for December and the first quarter of 2016, we’ll gradually migrate the collections from our existing site, and we’ll be developing the features and the look of the new site in an iterative process of feedback, analysis, and implementation. Our current plan is to have nearly all of the content of Duke Digital Collections available in the new platform by the end of March, 2016.
The completion of the Tripod3 project will mean the end of life for the current-generation platform, which we call, to no one’s surprise, Tripod2. However, we have not set an exact timeline for sunsetting Tripod2. During the transitional phase, we will do everything we can to make the architecture of Duke Digital Collections transparent, and our plans clear.
After the jump, I’ll spend the rest of this post going into a little more depth about the project, but want to express my pride and gratitude to an excellent team – you know who you are – who helped us achieve this milestone.