Videotelephony, Better Late than Never

A technology allowing most of us to keep working effectively during the COVID-19 pandemic is called “videotelephony,” which is real-time, simultaneous audio-visual communication between two or more users. Right now, millions of workers and families are using Zoom, FaceTime, WhatsApp, WebEx, Skype and other software to see and hear each other live, using the built-in microphones and video cameras on our computers, tablets and mobile phones.

We take this capability for granted now, but it’s actually been over a century in the making. Generations of trial and error, billions in spent capital, technical brick walls and failed business models have paved the way to this morning’s Zoom meeting with your work team. You might want to change out of your pajamas, by the way.

AT&T’s Picturephone (Model 1) was introduced at the 1964 World’s Fair.

Alexander Graham Bell famously patented the telephone in 1876. Shortly after, the concept of not only hearing the person you are talking to, but also seeing them simultaneously, stirred the imagination of inventors, writers and artists. It seemed like a reasonably-attainable next step. Early terms for a hypothetical device that could accomplish this included the “Telephonoscope” and the “Telectroscope

Mr. Bell himself conceived of a device called an “electrical radiophone,” and predicted “the day would come when the man at the telephone would be able to see the distant person to whom he was speaking.” But that day would not come until long after Bell’s death in 1922.

The problem was, the transmission of moving images was a lot more complicated than transmitting audio. Motion picture film, also introduced in the late 1800s, was brought to life by chemicals reacting to silver-halide crystals in a darkroom, but unlike the telephone, electricity played no part in film’s construction or dissemination.

The telephone converted sound waves to electrical signals, as did radio station towers. Neither could transmit without electricity. And a telephone is “full-duplex,” meaning the data is transmitted in both directions, simultaneously, on a single carrier. The next challenge was to somehow electrify moving images, make them full-duplex, and accommodate their exponentially larger bandwidth.

The Picturephone (Model 2). Only a few hundred were sold in the 1970s.

It wasn’t until the late 1930s that cathode-ray-tube television sets were introduced to the world, and the concept of analog video began to gain traction. Unlike motion picture film, video is an electronic medium. Now that moving images were utilizing electricity, they could be transmitted to others, using antennas.

After World War II ended, and Americans had more spending money, black & white television sets became popular household items in the 1950s. But unlike the telephone, communication was still one way. It wasn’t full-duplex. You could see “The Honeymooners,” but they couldn’t see you, and it wasn’t live.  Live television broadcasts were rare, and still in the experimental phase.

In 1964, AT&T’s Bell Labs (originally founded by Alexander Graham Bell), introduced the “Picturephone” at the New York World’s Fair and at Disneyland, demonstrating a video call between the two locales. Later, AT&T introduced public videophone booths in New York City, Chicago and Washington, DC. If you were in the New York videophone booth, you could see and hear someone in the Chicago videophone booth, in real time, and it was two-way communication.

The problem was, it was outrageously expensive. A three-minute call cost $225 in today’s money. The technology was finally here, but who could afford it? AT&T poured billions into this concept for years, manufacturing “PicturePhones” and “VideoPhones” for home and office, all the way through 1995, but they were always hampered by the limitations of low-bandwidth telephone lines and very high prices, making them not worth it for the consumer, and never widely adopted.

AT&T’s VideoPhone 2500, released in 1992, priced at $1599.99.

It wasn’t until broadband internet, and high-compression video codecs became widespread in the new millennium, that videotelephony finally became practical, affordable and thus marketable. In recent years, electronics manufacturers began to include video cameras and microphones as a standard feature in CPUs, tablets and mobile phones, making external webcams obsolete. Services like Skype, FaceTime and WebEx were introduced, and later WhatsApp, Zoom and numerous others.

Now it’s simple, and basically free, to have a high-quality, full-color video chat with your friend, partner or co-worker, and a company like Zoom has a net worth of 40 billion. It’s amazing that it took more than 100 years since the invention of the telephone to get here. And just in time for a global pandemic requiring strict physical distancing. Don’t forget to update your clever background image!

ArcLight Migration: A Status Update After Three Months of Work

On January 20, 2020, we kicked off our first development sprint for implementing ArcLight at Duke as our new finding aids / collection guides platform. We thought our project charter was solid: thorough, well-vetted, with a reasonable set of goals. In the plan was a roadmap identifying a July 1, 2020 launch date and a list of nineteen high-level requirements. There was nary a hint of an impending global pandemic that could upend absolutely everything.

The work wasn’t supposed to look like this, carried out by zooming virtually into each other’s living rooms every day. Code sessions and meetings now require navigating around child supervision shifts and schooling-from-home responsibilities. Our new young office-mates occasionally dance into view or within earshot during our calls. Still, we acknowledge and are grateful for the privilege afforded by this profession to continue to do our work remotely from safe distance.

So, a major shoutout is due to my colleagues in the trenches of this work overcoming the new unforeseen constraints around it, especially Noah Huffman, David Chandek-Stark, and Michael Daul. Our progress to date has only been possible through resilience, collaboration, and willingness to keep pushing ahead together.

Three months after we started the project, we remain on track for a summer 2020 launch.

As a reminder, we began with the core open-source ArcLight platform (demo available) and have been building extensions and modifications in our local application in order to accommodate Duke needs and preferences. With the caveat that there’ll be more changes coming over the next couple months before launch, I want to provide a summary of what we have been able to accomplish so far and some issues we have encountered along the way. Duke staff may access our demo app (IP-restricted) for an up-to-date look at our work in progress.

Homepage

Homepage design for Duke’s ArcLight finding aids site.
  • Duke Branding. Aimed to make an inviting front door to the finding aids consistent with other modern Duke interfaces, similar to–yet distinguished enough from–other resources like the catalog, digital collections, or Rubenstein Library website.
  • Featured Items. Built a configurable set of featured items from the collections (with captions), to be displayed randomly (actual selections still in progress).
  • Dynamic Content. Provided a live count of collections; we might add more indicators for types/counts of materials represented.

Layout

A collection homepage with a sidebar for context navigation.
  • Sidebar. Replaced the single-column tabbed layout with a sidebar + main content area.
  • Persistent Collection Info. Made collection & component views more consistent; kept collection links (Summary, Background, etc.) visible/available from component pages.
  • Width. Widened the largest breakpoint. We wanted to make full use of the screen real estate, especially to make room for potentially lengthy sidebar text.

Navigation

Component pages contextualized through a sidebar navigator and breadcrumb above the main title.
  • Hierarchical Navigation. Restyled & moved the hierarchical tree navigation into the sidebar. This worked well functionally in ArcLight core, but we felt it would be more effective as a navigational aid when presented beside rather than below the content.
  • Tooltips & Popovers. Provided some additional context on mouseovers for some navigational elements.

    Mouseover context in navigation.
  • List Child Components. Added a direct-child list in the main content for any series or other component. This makes for a clear navigable table of what’s in the current series / folder / etc. Paginating it helps with performance in cases where we might have 1,000+ sibling components to load.
  • Breadcrumb Refactor. Emphasized the collection title. Kept some indentation, but aimed for page alignment/legibility plus a balance of emphasis between current component title and collection title.

    Breadcrumb trail to show the current component’s nesting.

Search Results

Search results grouped by collection, with keyword highlighting.
  • “Group by Collection” as the default. Our stakeholders were confused by atomized components as search results outside of the context of their collections, so we tried to emphasize that context in the default search.
  • Revised search result display. Added keyword highlighting within result titles in Grouped or All view. Made Grouped results display checkboxes for bookmarking & digitized content indicators.
  • Advanced Search. Kept the global search box simple but added a modal Advanced search option that adds fielded search and some additional filters.

Digital Objects Integration

Digital objects from the Duke Digital Repository are presented inline in the finding aid component page.
  • DAO Roles. Indexed the @role attribute for <dao> elements; we used that to call templates for different kinds of digital content
  • Embedded Object Viewers. Used the Duke Digital Repository’s embed feature, which renders <iframe>s for images and AV.

Indexing

  • Whitespace compression. Added a step to the pipeline to remove extra whitespace before indexing. This seems to have slightly accelerated our time-to-index rather than slow it down.
  • More text, fewer strings. We encountered cases where note-like fields indexed as strings by ArcLight core (e.g., <scopecontent>) needed to be converted to text because we had more than 32,766 bytes of data (limit for strings) to put in them. In those cases, finding aids were failing to index.
  • Underscores. For the IDs that end up in a URL for a component, we added an underscore between the finding aid slug and the component ID. We felt these URLs would look cleaner and be better for SEO (our slugs often contain names).
  • Dates. Changed the date normalization rules (some dates were being omitted from indexing/display)
  • Bibliographic ID. We succeeded in indexing our bibliographic IDs from our EADs to power a collection-level Request button that leads a user to our homegrown requests system.

Formatting

  • EAD -> HTML. We extended the EAD-to-HTML transformation rules for formatted elements to cover more cases (e.g., links like <extptr> & <extref> or other elements like <archref> & <indexentry>)

    Additional formatting and link render rules applied.
  • Formatting in Titles. We preserved bold or italic formatting in component titles.

ArcLight Core Contributions

  • We have been able to contribute some of our code back to the ArcLight core project to help out other adopters.

Setting the Stage

The behind-the-scenes foundational work deserves mention here — it represents some of the most complex and challenging aspects of the project.  It makes the application development driving the changes I’ve shared above possible.

  • Built separate code repositories for our Duke ArcLight application and our EAD data
  • Gathered a diverse set of 40 representative sample EADs for testing
  • Dockerized our Duke ArcLight app to simplify developer environment setup
  • Provisioned a development/demo server for sharing progress with stakeholders
  • Automated continuous integration and deployment to servers using GitLabCI
  • Performed targeted data cleanup
  • Successfully got all 4,000 of our finding aids indexed in Solr on our demo server

Our team has accomplished a lot in three months, in large part due to the solid foundation the ArcLight core software provides. We’re benefiting from some amazing work done by many, many developers who have contributed their expertise and their code to the Blacklight and ArcLight codebases over the years. It has been a real pleasure to be able to build upon an open source engine– a notable contrast to our previous practice of developing everything in-house for finding aids discovery and access.

Still, much remains to be addressed before we can launch this summer.

The Road Ahead

Here’s a list of big things we still plan to tackle by July (other minor revisions/bugfixes will continue as well)…

  • ASpace -> ArcLight. We need a smoother publication pipeline to regularly get data from ArchivesSpace indexed into ArcLight.
  • Access & Use Statements. We need to revise the existing inheritance rules and make sure these statements are presented clearly. It’s especially important when materials are indeed restricted.
  • Relevance Ranking. We know we need to improve the ranking algorithm to ensure the most relevant results for a query appear first.
  • Analytics. We’ll set up some anonymized tracking to help monitor usage patterns and guide future design decisions.
  • Sitemap/SEO. It remains important that Google and other crawlers index the finding aids so they are discoverable via the open web.
  • Accessibility Testing / Optimization. We aim to comply with WCAG2.0 AA guidelines.
  • Single-Page View. Many of our stakeholders are accustomed to a single-page view of finding aids. There’s no such functionality baked into ArcLight, as its component-by-component views prioritize performance. We might end up providing a downloadable PDF document to meet this need.
  • More Data Cleanup. ArcLight’s feature set (especially around search/browse) reveals more places where we have suboptimal or inconsistent data lurking in our EADs.
  • More Community Contributions. We plan to submit more of our enhancements and bugfixes for consideration to be merged into the core ArcLight software.

If you’re a member of the Duke community, we encourage you to explore our demo and provide feedback. To our fellow future ArcLight adopters, we would love to hear how your implementations or plans are shaping up, and identify any ways we might work together toward common goals.

Stay safe, everyone!

Labor in the Time of Coronavirus

The Coronavirus pandemic has me thinking about labor–as a concept, a social process, a political constituency, and the driving force of our economy–in a way that I haven’t in my lifetime. It’s become alarmingly clear (as if it wasn’t before) that we all need food, supplies, and services to survive past next week, and that there are real human beings out there working to produce and deliver these things. No amount of entrepreneurship, innovation, or financial sleight of hand will help us through the coming months if people are not working to provide the basic requirements for life as we know it.

This blog post draws from  images in our digitized library collections to pay tribute to all of the essential workers who are keeping us afloat during these challenging times. As I browsed these photographs and mused on our current situation, a few important and oft-overlooked questions came to mind.

Who grows our food? Where does it come from and how is it processed? How does it get to us?

Bell pepper pickers, 1984 June. Paul Kwilecki Photographs.
Prisoners at the county farm killing hogs, 1983 Mar. Paul Kwilecki Photographs.
Worker atop rail car loading corn from storage tanks, 1991 Sept. Paul Kwilecki Photographs.
Manuel Molina, mushroom farm worker, Kennett Square, PA 1981. Frank Espada Photographs.

What kind of physical environment do we work in and how does that affect us?

Maids in a room of the Stephen Decatur Hotel shortly before it was torn down, 1970. Paul Kwilecki Photographs
Worker in pit preparing to weld. Southeastern Minerals Co. Bainbridge, 1991 Aug. Paul Kwilecki Photographs
Loggers in the woods near Attapulgus, 1978 Feb. Paul Kwilecki Photographs.

How do we interact with machines and technology in our work? Can our labor be automated or performed remotely?

Worker signaling for more logs. Elberta Crate and Box. Co. Bainbridge, 1991 Sept. Paul Kwilecki Photographs.
Worker, Williamson-Dickie plant. Bainbridge, 1991 Sept. Paul Kwilecki Photographs.
Machine operator watching computer controlled lathe, 1991 July. Paul Kwilecki Photographs.
Worker at her machine. Elberta Crate and Box Co. Bainbridge, 1981 Nov. Paul Kwilecki Photographs.

What equipment and clothing do we need to work safely and productively?

Cotton gin worker wearing safety glass and ear plugs for noise protection. Decatur Gin Co., 1991 Sept. Paul Kwilecki Photographs.
Worker operating bagging machine. Flint River Mills. Bainbridge, 1991 July. Paul Kwilecki Photographs.
Worker at State Dock, 1992 July. Paul Kwilecki Photographs.

Are we paid fairly for our work? How do relative wages for different types of work reflect what is valued in our society?

No known title. William Gedney Photographs.

How we think about and respond to these questions will inform how we navigate the aftermath of this ongoing crisis and whether or not we thrive into the future. As we celebrate International Workers’ Day on May 1 and beyond, I hope everyone will take some time to think about what labor means to them and to our society as a whole.

In a (Temporary) Time of Remote Work, Duke’s FOLIO Implementation Continues

Duke University is an early adopter for FOLIO, an open source library services platform that will give us tools to better support the information needs of our students, faculty, and staff. A core team in Library Systems and Integration Support began forming in January 2019 to help Duke move to FOLIO. I joined that team in January 2019 and began work as an IT Business Analyst.

In preparation for going-live with FOLIO, we formally kicked off our local implementation effort in January 2020. More than 40 local subject experts have joined small group teams to work on different parts of the FOLIO project. These experts are invaluable to Library IT staff: they know how the library’s work is done, which features need to be prioritized over others, and are committed to figuring out how to transition their work into the FOLIO environment.

If you’re reading this in April 2020 and thinking “wasn’t January ten years ago?” you’re not alone. Because the FOLIO Project is international, with partners all over the world, many of us are used to working via remote tools like Slack, Microsoft Teams, and Zoom. But that is a far cry from doing ALL of our work that way, while also taking care of our families and ourselves. It’s a huge credit to all library staff that while the University was swiftly pivoting to remote work, we were able to keep our implementation work going.

One of the first big, messy areas that we knew we needed to work on was using locations.

Locations are essential to how patrons know where an item is at the Duke Libraries. When you look up a book in our catalog and the system tells you Where to Find It, it’s using location information from our systems. Library staff also use locations to understand how often items are borrowed, decide when to move items to our off-campus storage, and decide when we to buy new items to keep our collections up to date.

A group of FOLIO team members came together from different working areas, including public services, cataloging, acquisitions, digital resources and assessment. I convened those discussions as a lead for our Configurations team. Over the course of late February and March 2020, we met three times as a group using Zoom and delved deep into learning about locations in our current system and how they will work in FOLIO. Staff members shared their knowledge with each other about their functional areas, allowing us to identify potential gaps in FOLIO functionality, as well as things we could improve now, without waiting for FOLIO to deploy.

This team identified two potential paths forward – one that was straightforward, and one that was more creative and would adapt the FOLIO four-level locations in a new way.  In our final meeting – where we had hoped to decide between the two options, our subject experts grappled with the challenges, risks and rewards of the two choices and were able to recommend a path forward together. Ultimately, the team agreed that the creative option was the best choice, but both options would work – and that guidance helped us decide how to make a first pass on configuring locations and move the project forward.

The most important part of these meetings was valuing the expertise of our library staff and working to support them as they decided what would work the best for the library’s needs.  I am deeply appreciative of the staff who committed the time to these discussions while also figuring out how to move their regular jobs to remote work. Our FOLIO implementation is all the better because of their collaborative spirit.

DUCC, TUCC, and the origins of digital computing in North Carolina

The feature image is”Triangle University Computation Center IBM System/370 Hardware Configuration,” from Network Management Survey, published in 1974.

The Cut Study and DUCC

The Fall semester of 1958 saw deep concern among the Duke student body with a pressing issue – cutting class. The Undergraduate Faculty Council Committee had taken up a study of class attendance, and planned to issue recommendations for policies on “absence limitations.” Its chair was John Jay Gergen, who had been on the faculty at Duke more than 20 years at that point, serving most of them as head of the Mathematics Department. In September, the Chronicle urged him to “make a sincere effort to show the students the seriousness of the situation and to explain their findings.” They warned him not to “announce suddenly a new policy to the students,” which would be a form of “[t]actless communication” that might “breed discontent among the students.” By all indications, Gergen ignored them.

While the “Cut Study” may have seemed enormously consequential to the students at the time, Gergen was leading a different effort that would have far more lasting impact at Duke. By at least one account, he was a large and imposing man, which could also describe his influence on campus. He channeled some of that influence into his work as the senior faculty member and administrator who oversaw the effort to bring digital computing to the university. While Gergen acted mainly in an administrative role, it was a protege of his who authored the grants that brought in the funding, and did the legwork on setting up an operational computing center.

Continue reading DUCC, TUCC, and the origins of digital computing in North Carolina

Library Crisis: The Late Bronze Age Collapse of 1177 BC and the Coronavirus of 2020 AD

View of Hattusa

 

Late 13th century BC rock relief carving at the Yazılıkaya sanctuary

Located amongst agricultural fields, grazing cattle and goats, the quiet town of Boğazkale watches time pass.  It is a small town in the Black Sea region of Turkey, only about 1300 people live here.  You are more likely to encounter a tractor than a car and you get the sense that everyone knows everyone else.  If you walk about half a kilometer outside of this sleepy community, you will encounter the equally serene-looking archaeological site of Hattusa.  Although now abandoned, at its peak in the 14th century BC, Hattusa was a capital and home to nearly 50,000 Hittites.

 

Cuneiform tablet

Much of what is known about the Hittites is drawn from their own writings, or from later copies of their writings.  The Hittites have been described by some sources as warlike; they utilized chariots and steadily increased their kingdom.  They worked iron and had a fair amount of courtly intrigue.  Perhaps most importantly, the Hittites were literate, employing an Indo-European language with an Akkadian script.

Around 1177 BC, Hattusa fell as one of many victims of the Late Bronze Age Collapse.  There is archaeological evidence for extensive burning at the site and the written records at Hattusa stop.  It appears as though the site was gradually abandoned over time and never successfully reoccupied.  That turned out to be a stroke of luck because a German archaeological dig in the early 20th century (re)discovered the so-called Bogazköy Archive, which was determined initially to be a largely intact royal archive.  Thanks to more research and the decipherment of the Hittite language in 1915, it seems more likely that this assemblage of 30,000 cuneiform tablets is actually a library.  This is based largely on the argument that some of the tablets have colophons that provide an order hierarchy for multi-part works and the collection includes basic descriptive inventory lists of the tablets, instituting an early type of collection management.

The abandonment of Hattusa and its library makes one wonder:  What would it have been like for the scribes or library keepers to make the decision to leave the collection?  What would they have done to get the collection ready for their absence?  What would have the last person who left the collection felt like upon leaving?  Did they believe they would return?

It’s impossible to answer such questions, since we have no surviving first-person accounts from this event.  However, we can consider a current analogy and while it’s not a perfect correspondence, it may provide some emotional experiential equivalence.

With the advent of the 2019-2020 coronavirus pandemic, many universities and cultural heritage institutions were faced with the difficult challenge of abiding by official orders, trying to keep their faculty and staff safe and healthy, and attempting to continue with business as usual.  At Duke University Libraries, the decision-making process began publicly at the beginning of March 2020.  It started with one email, a few days’ pause, and then another email and the information about how the Libraries would ultimately face the pandemic trickled in.  Eventually, it became one or more emails everyday with updates.  The Libraries scaled back opening times and access to staff and eventually made the onerous decision to close the Libraries on March 20, 2020.

The last week that the Libraries were open, staff were encouraged to begin setting up to work from home and to come up with ideas of what work could be done remotely.  Colleagues began making the transition from being in the Library to being at home.  In the Digital Production Center (DPC), our small workgroup of 4 individuals slowly began to dwindle as one-by-one co-workers began working from home.

One of the last items digitized by the DPC prior to its March 20 closure

So, what did the DPC do to get ready for the Libraries’ closure?  Our primary remit as a department is to produce digital surrogates of Duke’s collection items both for online use and for library patrons.  So, we worked hard to get items digitized for online class use, patrons and online collections.  It was a race against time because we knew that whatever we didn’t have imaged by Friday at 5pm would have to wait.  And for online courses that were counting on the material, waiting wasn’t an option.  Normally, the DPC holds materials that we are working with in our vault.  However, with the Library closing down, those items needed to be secured in the David M. Rubenstein Rare Book and Manuscript Library.  We took several carts full of materials back for safe keeping, with the idea that we would retrieve them when the Libraries reopen.

As the number of staff physically onsite in the Library began to diminish, I started saying goodbye to more people than hello.  With every farewell, there was uncertainty:  When will we see each other again?  The Library became a truly quiet place.  Gone were the patrons and students, replaced by empty seats and a deafening silence.  Colleagues that I would normally pass in the Library several times a week suddenly began showing up on my computer screen for Zoom meetings.

Perkins Library on March 20, 2020
Last day at the DPC

On the final day that the Libraries were open to staff, the last batch of materials was returned to the Rubenstein Library for safekeeping and the Libraries themselves had transformed into very different, very empty spaces.  While the circumstances are certainly not the same as they were for the Hittites, there was a sense of uncertainty.  I’d like to believe that I was more assured of my return to the Library than that last scribe or keeper.

Spring blossoms on a eerily empty Duke University campus

As I left Perkins Library, I paused in the doorway and looked back over my shoulder.  I didn’t know when I might be back.  I turned and exited into the warm Spring day on an empty Duke campus, and just like the last scribe or keeper, I was stepping into an unfamiliar future.

Epilogue

Although as of March 27, 2020,  Duke University Libraries are currently closed, the Libraries anticipate reopening as soon as it is safe and prudent to do so.  In the meantime, you can find a list of services still available during this closure here.

Sources

Bryce, Trevor. The Kingdom of the Hittites. Rev. ed.  Oxford University Press, 2005.

Casson, Lionel.  Libraries of the Ancient World.  Yale University, 2001.

Roaf, Michael. Cultural Atlas of Mesopotamia and the Ancient Near East.  Facts on File, 1990.

van den Hout, Theo P.J.  “Miles of Clay: Information Management in the Ancient Near Eastern Hittite Empire.”  Fathom Archive, The University of Chicago, 26 October 2002, http://fathom.lib.uchicago.edu/1/777777190247/.

Further Reading

Cline, Eric H.  1177 BC: The Year Civilization Collapsed.  Princeton University Press, 2014.  An overview of the Late Bronze Age Collapse, with thanks to the author for his suggestion.

Sharing data and research in a time of global pandemic

[Header image from the New York Times Coronavirus Map, March 17th, 2020]

Just before Duke stopped travel for all faculty and staff last week, I was able to attend what will probably turn out to have been one of the last conferences of the spring in the Research Data Access and Preservation Association’s (RDAP) annual summit in Santa Fe, New Mexico. RDAP is a community of “data managers and curators, librarians, archivists, researchers, educators, students, technologists, and data scientists from academic institutions, data centers, funding agencies, and industry who represent a wide range of STEM disciplines, social sciences, and humanities,” and who are committed to creating, maintaining, and teaching best practices for the access and preservation of research data. While there were many interesting presentations and posters about the work being done in this area at various institutions around the country, the conference and RDAP’s work more broadly resonated with me in a very general and timely way, which did not necessarily stem from anything I heard during the week. 

In a situation like the global pandemic we are now facing, open and unfettered access to research data is vital for treating patients, attempting to stem the course of the disease, and potentially developing life-saving vaccines lives.

A recent editorial in Science, Translational Medicine, argues that data-driven models and centralized data sharing are the best way to approach this kind of outbreak, stating “[w]e believe that scientific efforts need to include determining the values (and ranges) of the above key variables and identifying any other important ones. In addition, information on these variables should be shared freely among the scientific and the response and resilience communities, such as the Red Cross, other nongovernmental organizations, and emergency responders” [1]. As another article points out, sharing viral samples from around the world has allowed scientists to get a better picture of the disease’s genetic makeup: “[c]omparing those genomes allowed Bedford and colleagues to piece together a viral family tree. ‘We can chart this out on the map, then, because we know that this genome is connected to this genome by these mutations,’ he said. ‘And we can learn about these transmission links'” [2].


We can chart this out on the map, then, because we know that this genome is connected to this genome by these mutations. And we can learn about these transmission links.


Scientists are also accelerating the research lifecycle by using preprint servers like arXiv, bioRxiv, and medRxiv to share their preliminary conclusions without waiting on the often glacial process of peer review. This isn’t a wholly unalloyed positive, and many preprints warrant the increased scrutiny that peer review represents. Moreover, scientific research often benefits from the kind of contextualization and unpacking that peer review and science journalism can occasionally provide. But in the acute crisis that the current outbreak presents, the rapid spread of information among scientific peer networks can undoubtedly save lives.

Continuing to develop and build the infrastructure—in terms of both technology and policy frameworks—needed to conduct the kind of data sharing we are seeing now remains a goal for the scientific community moving forward.

The Libraries, along with communities like RDAP, the Research Data Alliance, and the Data Curation Network, endorse and support this mission, and we will continue to play our role in preserving and providing persistent access to research data as best we can as we all move forward through this together. In the meantime, we hope everyone in the Duke community stays safe and healthy!

[1] Layne, S. P., Hyman, J. M., Morens, D. M., & Taubenberger, J. K. (2020, March 11). New coronavirus outbreak: Framing questions for pandemic prevention. Science Translational Medicine 12(534). https://doi.org/10.1126/scitranslmed.abb1469

[2] Sanders, L. (2020, February 13). Coronavirus’s genetic fingerprints are used to rapidly map its spread. Science News. https://www.sciencenews.org/article/coronavirus-genetic-fingerprints-are-used-to-rapidly-map-spread

The New Books & Media Catalog Turns One

It’s been just over a year since we launched our new catalog in January of 2019. Since then we’ve made improvements to features, performance, and reliability, have developed a long term governance and development strategy, and have plans for future features and enhancements.

During the Spring 2019 semester we experienced a number of outages of the Solr index that powers the new catalog. It proved to be both frustrating and difficult to track down the root cause of the outages. We took a number of measures to reduce the risk of bot traffic slowing down or crashing the index. A few of these measures include limiting facet paging to 50 pages and results paging to 250 pages, as well as setting limits on OpenSearch queries. We also added service monitoring so we are automatically alerted when things go awry and automatic restarts under some known bad system conditions. We also identified that a bug in the version of Solr we were running was vulnerable to causing crashes for queries with particular characteristics. We have since applied a patch to Solr to address this bug. Happily, the index has not crashed since we implemented these protective measures and bug fixes.

Over the past year we’ve made a number of other improvements to the catalog including:

  • Caching of the home page and advanced search page have reduced page load times by 75%.
  • Subject searches are now more precise and do not include stemmed terms.
  • CDs and DVDs can be searched by accession number.
  • When digitized copies of Duke material are available at the Internet Archive, links to the digital copy are automatically generated.
  • Records can be saved to a bookmarks list and shared with others via a stable URL.
  • Eligible records now have a “Request digitization” link.
  • Many other small improvements and bug fixes.

We sometimes get requests for features that the catalog already supports:

While development has slowed, the core TRLN team meets monthly to discuss and prioritize new features and fixes, and dedicates time each month to maintenance and new development. We have a number of small improvements and bug fixes in the works. One new feature we’re working on is adding a citation generator that will provide copyable citations in multiple formats (APA, MLA, Chicago, Harvard, and Turabian) for records with OCLC numbers.

We welcome, read, and respond to all feedback and feature requests that come to us through the catalog’s feedback form. Let us know what you think.

Check out “Search Tips” and “Expert Search Tips” for detailed information about how to get the most out of the new catalog.

Beyond One Thousand Words

There is a particular fondness that I hold for digital photograph collections. If I had to pinpoint when this began, then I would have to say it started while digitizing material on a simple Epson flatbed scanner as an undergraduate student worker in the archives.

Witnessing the physical become digital is a wonder that never gets old.

Every day we are generating digital content. Pet pics. Food pics. Selfies. Gradually building a collection of experiences as we document our lives in images. Sporadic born digital collections stored on devices and in the cloud.

I do not remember the last time I printed a photograph.

My parents have photo albums that I love. Seeing images of them, then us. The tacky adhesive and the crinkle of thin plastic film as it is pulled back to lift out a photo. That perfect square imprint left behind from where the photo rested on the page.

Pretty sure that Polaroid camera is still around somewhere.

Time bound up in a book.

Beyond their visual appeal, I appreciate how photos capture time. Nine months have passed since I moved to North Carolina. I started 2019 in Chicago and ended it in Durham. These photos of my Winter in both places illustrate that change well.

Sometimes I want to pull down my photos from the cloud and just print everything. Make my own album. Have something with heft and weight to share and say, “Hey, hold and look at this.” That sensory experience is invaluable.

Yet, I also value the convenience of being able to view hundreds of photos with the touch of a button.

Duke University Libraries offers access to thousands of images through its Digital Collections.

Here’s a couple photo collections to get you started:

Duke Digital Repository Evolution and a new home page

After nearly a year of work, the libraries recently launched an updated version of the software stack that powers parts the Duke Digital Repository. This work primarily centered around migrating the underlying software in our Samvera implementation — which we use to power the DDR — from ActiveFedora to Valkyrie. Moving to Valkyrie gives us the benefits of improved stability along with the flexibility to use different storage solutions, which in turn provides us with options and some degree of future-proofing. Considerable effort was also spent on updating the public and administrative interfaces to use more recent versions of blacklight and supporting software.

ddr admin interface
Administrative interface for the DDR

We also used this opportunity to revise the repository landing page at repository.duke.edu and I was involved in building a new version of the home page. Our main goals were to make use of a header implementation that mirrored our design work in other recent library projects and that integrated our ‘unified’ navigation, while also maintaining the functionality required by the Samvera software.

Old DDR Homepage
DDR home page before the redesign

We also spent a lot of time thinking about how best to illustrate the components of the Duke Digital Repository while trying to keep the content simple and streamlined. In the end we went with a design that emphasizes the two branches of the repository; Library Collections and Duke Scholarship. Each branch in turn links to two destinations — Digitized Collections / Acquired Materials and the Research Data Repository / DukeSpace. The overall design is more compact than before and hopefully an improvement aesthetically as well.

new DDR homepage
Redesigned DDR home page

We also incorporated a feedback form that is persistent across the interface so that users can more readily report any difficulties they encounter while using the platform. And finally, we updated the content in the footer to help direct users to the content they are more than likely looking for.

Future plans include incorporating our header and footer content more consistently across the repository platforms along with bringing a more unified look and feel to interface components.

Check out the new design and let us know what you think!

Notes from the Duke University Libraries Digital Projects Team