Two years ago, Duke Libraries Advisory Council for Digital Collections launched a new process for proposing digitization projects. Previously the group accepted new digitization proposals every month. We decided to shift to a “digitization initiative” approach where the Council issues a time-based call for proposals focusing on a theme or format. This new method has allowed staff across different departments to plan and coordinate their efforts more effectively.
This Fall we are inviting DUL staff to propose Audio and Video (A/V) based collections/items for digitization. DUL staff are welcome to partner with Duke Faculty on their proposals. We chose to focus on A/V formats this year due to the preservation risks associated with the material. Magnetic tape formats are especially fragile compared to film given their composition, and the low availability of players for accessing content.
The Advisory Council is working on another call for digitization proposals, which is intended to include non-A/V formats (manuscripts, photographs, and more). We should be able to announce the new call before the end of the calendar year. Stay tuned!
Last week, I went to go see the movie IT: Chapter 2. One thing I really appreciated about the movie was how it used a scene’s lighting to full effect. Some scenes are brightly lit to signify the friendship among the main characters. Conversely, there are dark scenes that signify the evil Pennywise the Clown. For the movie crew, no doubt it took a lot of time and manpower to light an individual scene – especially when the movie is nearly 3 hours long.
We do the same type of light setup and management inside the Digital Production Center (DPC) when we take photos of objects like books, letters, or manuscripts. Today, I will talk specifically about how we light the bound material that comes our way, like books or booklets. Generally, this type of material is always going to be shot on our PhaseOne camera, so I will particularly highlight that lighting setup today.
Before We Begin
It’s not enough to just turn the lights on in our camera room to do the trick. In order to properly light all the things that need to be shot on the PhaseOne, we have specific tools and products we use that you can see in the photo below.
We have 4 high-powered lights (two sets of two Buhl SoftCube SC-150 models) pointed directly in the camera’s field of view. There are two on the right and two on the left. These are stationed approximately 3.5 feet off the ground and approximately 2.5 feet away from the objects themselves. These lights are supported by Avenger A630B light stands. They allow for a wide range of movement, extension, and support if we need them.
But if bright, hot lights were pointed directly at sensitive documents for hours, it would damage them. So light diffusers are necessary. For both sets of lights, we have 3 layers of material to diffuse the light and prevent material from warping or text from fading. The first layer, directly attached to the light box itself, is an inexpensive sheet of diffusion fabric. This type of material is often made from nylon or silk, and are usually inexpensive.
The second diffusion layer is an FJ Westcott Scrim Jim, a similar thin fabric that is attached to a lightweight stand-up frame, the Manfrotto 156BLB. This frame can also be moved or extended if need be. The last layer is another sheet of diffusion fabric, attached to a makeshift “cube” held up by lightweight wooden rods. This cube can be picked up or carried, making it very convenient if we need to eventually move our lights.
So in total, we have 4 lights, 4 layers of diffusion fabric attached to the light boxes, two Scrim Jims, and the cube featuring 2 sides of additional diffusion fabric. After having all these items stationed, surely we can start taking pictures, right? Not yet.
Around the Room
There are still more things to be aware of – this time in the camera room itself. We gently place the materials themselves on a cradle lined with a black felt, similar to velvet. This cradle is visible in the bottom right part of the photo above. It is placed on top of a table, also coated in black felt. This is done so no background colors bounce back or reflect onto the object and change what it looks like in the final image itself. The walls of the camera room are also painted a neutral grey color for the same reason, as you can see in the background of the above photo. Finally, any tiny reflective segments between the ceiling tiles have been blacked out with gaffer tape. Having the room this muted and intentionally dark also helps us when we have to shoot multi-spectral images. No expense has been spared to make sure our colors and photos are correct.
With all these precautions in place, can we finally take photos of our materials? Almost. Before we can start photographing, we have to run some tests to make sure everything looks correct to our computers. After making sure our objects are sharp and in focus, we use a program called DTDCH (see the photo to the right) to adjust the aperture and exposure of the PhaseOne so that nothing appears either way too dim or too bright. In our camera room, we use a PhaseOne IQ180 with a Schneider Kreuznach Apo-Digitar lens (visible in the top-right corner of the photo above). We also use the program CaptureOne to capture, save, and export our photos.
Once the shot is in focus and appropriately bright, we will check our colors against an X-Rite ColorChecker Classic card (see the photo on the left) to verify that our camera has a correct white balance. When we take a photo of the ColorChecker, CaptureOne displays a series of numbers, known as RGB values, found in the photo’s colors. We will check these numbers against what they should be, so we know that our photo looks accurate. If these numbers match up, we can continue. You could check our work by saving the photo on the left and opening it in a program like Adobe Photoshop.
Finally, we have specific color profiles that the DPC uses to ensure that all our colors appear accurate as well. For more information on how we consistently calibrate the color in our images, please check out this previous blog post.
After all this setup, now we can finally shoot photos! Lighting our materials for the PhaseOne is a lot of hard work and preparation. But it is well worth it to fulfill our mission of digitizing images for preservation.
Duke Libraries has a large collection of analog videotapes, in several different formats. One of the most common in our archives is 3/4″ videotape, also called “U-matic” (shown above). Invented by Sony in 1969, U-matic was the first videotape to be housed inside a plastic cassette for portability. Before U-matic, videotape was recorded on very large reels in the 2″ format known as Quadruplex which required heavy recording and playback machines the size of household refrigerators. U-matic got its name from the shape of the tape path as it wraps around the video head drum, which looks like the letter U.
The format was officially released in 1971, and soon became popular with television stations, when the portable Sony VO-3800 video deck was released in 1974. The VO-3800 enabled TV crews to record directly to U-matic videotape at breaking news events, which previously had to be shot with 16mm film. The news content was now immediately available for broadcast, as opposed to film, which had to wait for processing in a darkroom. And the compact videocassettes could easily and quickly be transported to the TV station.
In the 1970’s, movie studios also used U-matic tapes to easily transport filmed scenes or “dailies,” such as the first rough cut of “Apocalypse Now.” In 1976, the high-band BVU (Broadcast Video U-matic) version of 3/4″ videotape, with better color reproduction and lower noise levels, replaced the previous “lo-band” version.
The U-matic format remained popular at TV stations throughout the 1980’s, but was soon replaced by Sony’s 1/2″ Betacam SP format. The BVU-900 series was the last U-matic product line made by Sony, and Duke Libraries’ Digital Production Center uses two BVU-950s for NTSC tapes, as well as a VO-9800P for tapes in PAL format. A U-matic videotape player in good working order is now an obsolete collector’s item, so they can be hard to find, and expensive to purchase.
Unfortunately, most U-matic tapes have not aged well. After decades in storage, many of the videotapes in our collection now have sticky-shed syndrome, a condition in which the oxide that holds the visual content is literally flaking off the polyester tape base, and is moist and gummy in texture. When a videotape has sticky-shed, not only will it not play correctly, the residue can also clog up the tape heads in the U-matic playback deck, then transfer the contaminant to other tapes played afterwards in the same deck.
To combat this, we always bake (dehumidify) our U-matic videotapes in a scientific oven at 52 celsius (125 fahrenheit) for at least 10 hours. Then we run each tape through a specialized tape-cleaning machine, which fast-forwards and rewinds each tape, while using a burnishing blade to wipe off any built-up residue. We also clean the video heads inside our U-matic decks before each playback, using denatured alcohol.
Most of the time, these procedures make the U-matic tape playable, and we are able to digitize them, which rescues the content from the videotapes, before the magnetic tape ages and degrades any further. While the U-matic tapes are nearing the end of their life-span, the digital surrogates will potentially last for centuries to come, and will be accessible online through our Duke Digital Repository, from anywhere in the world.
In the audio world, we take our tools seriously, sometimes to an unhealthy and obsessive degree. We give them pet names, endow them with human qualities, and imbue them with magical powers. In this context, it’s not really strange that a manufacturer of professional audio interfaces would call themselves “Mark of the Unicorn.”
Here at the Digital Production Center, we recently upgraded our audio interface to a MOTU 896 mk3 from an ancient (in tech years) Edirol UA-101. The audio interface, which converts analog signals to digital and vice-versa, is the heart of any computer-based audio system. It controls all of the routing from the analog sources (mostly cassette and open reel tape decks in our case) to the computer workstation and the audio recording/editing software. If the audio interface isn’t seamlessly performing analog to digital conversion at archival standards, we have no hope of fulfilling our mission of creating high-quality digital surrogates of library A/V materials.
While the Edirol served us well from the very beginning of the Library’s forays into audio digitization, it had recently begun to cause issues resulting in crashes, restarts, and lost work. Given that the Edirol is over 10 years old and has been discontinued, it is expected that it would eventually fail to keep up with continued OS and software updates. After re-assessing our needs and doing a bit of research, we settled on the MOTU 896 mk3 as its replacement. The 896 had the input, output, and sync options we needed along with plenty of other bells and whistles.
I’ve been using the MOTU for several weeks now, and here are some things that I’m liking about it:
Easy installation of drivers
Designed to fit into standard audio rack
Choice of USB or Firewire connection to PC workstation
Good visual feedback on audio levels, sample rate, etc. via LED meters on front panel
Clarity and definition of sound
I haven’t had a chance to explore all of the additional features of the MOTU yet, but so far it has lived up to expectations and improved our digitization workflow. However, in a production environment such as ours, each piece of equipment needs to be a workhorse that can perform its function day in and day out as we work our way through the vaults. Only time can tell if the Mark of the Unicorn will be elevated to the pantheon of gear that its whimsical name suggests!
About four and a half years ago I wrote a blog post here on Bitstreams titled: “Digitization Details: Before We Push the “Scan” Button” in which I wrote about how we use color calibration, device profiling and modified viewing environments to produce “consistent results of a measurable quality” in our digital images. About two and a half years ago, I wrote a blog post adjacent to that subject titled “The FADGI Still Image standard: It isn’t just about file specs” about the details of the FADGI standard and how its guidelines go beyond ppi and bit depth to include information about UV light, vacuum tables, translucent material, oversized material and more. I’m surprised that I have never shared the actual process of digitizing a collection because that is what we do in the Digital Production Center.
Building digital collections is a complex endeavor that requires a cross-departmental team that analyzes project proposals, performs feasibility assessments, gathers project requirements, develops project plans, and documents workflows and guidelines in order to produce a consistent and scalable outcome in an efficient manner. We call our cross-departmental team the Digital Collections Implementation Team (DCIT) which includes representatives from the Conservation staff, Technical Services, Digital Production, Metadata Architects and Digital Collections UI developers, among others. By having representatives from each department participate, we are able to consider all perspectives including the sticking points, technical limitations and time constraints of each department. Over time, our understanding of each other’s workflows and sticking points has enabled us to refine our approach to efficiently hand off a project between departments.
I will not be going into the details of all the work other departments contribute to building digital collections (you can read just about any post on the blog for that). I will just dip my toe into what goes on in the Digital Production Center to digitize a collection.
Once the specifics of a project are nailed down, the scope of the project has been finalized, the material has been organized by Technical Services, Conservation has prepared the material for digitization, the material has been transferred to the Digital Production Center and an Assessment Checklist is filled out describing the type, condition, size and number of items in a collection, we are ready to begin the digitization process.
A starter digitization guide is created using output from ArchivesSpace and the DPC adds 16-20 fields to capture technical metadata during the digitization process. The digitization guide is an itemized list representing each item in a collection and is centrally stored for ease of access.
Cameras and monitors are calibrated with a spectrometer. A color profile is built for each capture device along with job settings in the capture software. This will produce consistent results from each capture device and produce an accurate representation of any items captured which in turn removes subjective evaluation from the scanning process.
Instructions are developed describing the scanning, quality control, and handling procedures for the project and students are trained.
Following instructions developed for each collection, the scanner operator will use the appropriate equipment, settings and digitization guide to digitize the collection. Benchmark tests are performed and evaluated periodically during the project. During the capture process the images are monitored for color fidelity and file naming errors. The images are saved in a structured way on the local drive and the digitization guide is updated to reflect the completion of an item. At the end of each shift the files are moved to a production server.
Quality Control 1
The Quality Control process is different depending on the device with which an item was captured and the nature of the material. All images are inspected for: correct file name, skew, clipping, banding, blocking, color fidelity, uniform crop, and color profile. The digitization guide is updated to reflect the completion of an item.
Quality Control 2
Images are cropped (leaving no background) and saved as JPEGs for online display. During the second pass of quality control each image is inspected for: image consistency from operator to operator and image to image, skew and other anomalies.
During this phase we compare the digitization guide against the item and file counts of the archival and derivative images on our production server. Discrepancies such as missing files, misnamed files and missing line items in the digitization guide and are resolved.
Create Checksums and dark storage
We then create a SHA1 checksum for each image file in the collection and push the collection into a staging area for ingest into the repository.
Sometimes this process is referred to simply as “scanning”.
Not only is this process in active motion for multiple projects at the same time, the Digital Production Center also participates in remediation of legacy projects for ingest into the Duke Digital Repository, multispectral imaging, audio digitization and video digitization for, preservation, patron and staff requests… it is quite a juggling act with lots of little details but we love our work!
Time to get back to it so I can get to a comfortable stopping point before the Thanksgiving break!
Duke University Libraries is recruiting a Digital Production Services Manager to direct the operations of our Digital Production Center, its staff (3 FTE plus student assistants), and associated digitization services. We are seeking someone experienced in leading digitization projects who is excited to partner with colleagues around the library to reformat and preserve unique library collections and provide access to them online. This is an excellent opportunity for someone who likes working with people, projects, and primary sources!
This newly created position combines people and project management responsibilities with hands-on digitization duties. Previous supervisory experience is not required; however, the ability to direct the work of others is essential to this position, as is a service oriented attitude. Strong organizational and project management skills are also a must. Some form of digitization experience in a library or other cultural heritage setting is required for this role as well. The successful candidate will join the highly collaborative Digital Collections and Curation Services department and work under the direct supervision of the department head.
Duke is a diverse community committed to the principles of excellence, fairness, and respect for all people. As part of this commitment, we actively value diversity in our workplace and learning environments as we seek to take advantage of the rich backgrounds and abilities of everyone. We believe that when we understand, celebrate, and tap into our uniqueness to creatively solve problems and address shared goals, our possibilities are limitless. Duke University Libraries value diversity of thought, perspective, experience, and background and are actively committed to a culture of inclusion and respect.
Duke offers a comprehensive benefit package, which includes both traditional benefits such as health insurance, leave time and retirement, as well as wide ranging work/life and cultural benefits. Details can be found at: http://www.hr.duke.edu/benefits/index.php.
If you are a regular Bitstreams reader, you know we just love talking about Multispectral Imaging. Seriously, we can go on and on about it, and we are not the only ones. This week however we are keeping it short and sweet and sharing a couple before and after images from one of our most recent imaging sessions.
Below are two stacked images of Ashkar MS 16 (from the Rubenstein Library). The top half of each image is the manuscript under natural light, and the bottom are the results of Multispectral imaging and processing. We tend to post black and white MSI images most often as they are generally the most legible, however our MSI software can produce a lot of wild color variations! The orange one below seemed the most appropriate for a hot NC July afternoon like today. More processing details are included in the image captions below – enjoy!
As a Digital Production Specialist at Duke Libraries, I work with a variety of obsolete videotape formats, digitizing them for long-term preservation and access. Videotape is a form of magnetic tape, consisting of a magnetized coating on one side of a strip of plastic film. The film is there to support the magnetized coating, which usually consists of iron oxide. Magnetic tape was first invented in 1928, for recording sound, but it would be several decades before it could be used for moving images, due to the increased bandwidth that is required to capture the visual content.
Television was live in the beginning, because there was no way to pre-record the broadcast other than with traditional film, which was expensive and time-consuming. In 1951, Bing Crosby Enterprises (BCE), owned by actor and singer Bing Crosby, demonstrated the first videotape recording. Crosby had previously incorporated audiotape recording into the production of his radio broadcasts, so that he would have more time for other commitments, like golf! Instead of having to do a live radio broadcast once a week for a month, he could record four broadcasts in one week, then have the next three weeks off. The 1951 demonstration ran quarter-inch audiotape at 360 inches per second, using a modified Ampex 200 tape recorder, but the images were reportedly blurry and not broadcast quality.
More companies experimented with the emerging technology in the early 1950’s, until Ampex introduced 2” black and white quadruplex videotape at the National Association of Broadcasters convention in 1956. This was the first videotape that was broadcast quality. Soon, television networks were broadcasting pre-recorded shows on quadruplex, and were able to present them at different times in all four U.S. time zones. Some of the earliest videotape broadcasts were CBS’s “The Edsel Show,” CBS’s “Douglas Edwards with the News,” and NBC’s “Truth or Consequences.” In 1958, Ampex debuted a color quadruplex videotape recorder. NBC’s “An Evening with Fred Astaire” was the first major TV show to be videotaped in color, also in 1958.
One of the downsides to quadruplex, is that the videotapes could only be played back using the same tape heads which originally recorded the content. Those tape-heads wore out very quickly, which mean’t that many tapes could not be reliably played back using the new tape-heads that replaced the exhausted ones. Quadruplex videotapes were also expensive, about $300 per hour of tape. So, many TV stations maximized the expense, by continually erasing tapes, and then recording the next broadcast on the same tape. Unfortunately, due to this, many classic TV shows are lost forever, like the vast majority of the first ten years (1962-1972) of “The Tonight Show with Johnny Carson,” and Super Bowl II (1968).
Quadruplex was the industry standard until the introduction of 1” Type C, in 1976. Type C video recorders required less maintenance, were more compact and enabled new functions, like still frame, shuttle and slow motion, and 1” Type C did not require time base correction, like 2” Quadruplex did. Type C is a composite videotape format, with quality that matches later component formats like Betacam. Composite video merges the color channels so that it’s consistent with a broadcast signal. Type C remained popular for several decades, until the use of videocassettes gained in popularity. We will explore that in a future blog post.
In past posts, I’ve paid homage to the audio ancestors with riffs on such endangered–some might say extinct–formats as DAT and Minidisc. This week we turn our attention to the smallest (and perhaps the cutest) tape format of them all: the Microcassette.
Introduced by the Olympus Corporation in 1969, the Microcassette used the same width tape (3.81 mm) as the more common Philips Compact Cassette but housed it in a much smaller and less robust plastic shell. The Microcassette also spooled from right to left (opposite from the compact cassette) as well as using slower recording speeds of 2.4 and 1.2 cm/s. The speed adjustment, allowing for longer uninterrupted recording times, could be toggled on the recorder itself. For instance, the original MC60 Microcassette allowed for 30 minutes of recorded content per “side” at standard speed and 60 minutes per side at low speed.
The microcassette was mostly used for recording voice–e.g. lectures, interviews, and memos. The thin tape (prone to stretching) and slow recording speeds made for a low-fidelity result that was perfectly adequate for the aforementioned applications, but not up to the task of capturing the wide dynamic and frequency range of music. As a result, the microcassette was the go-to format for cheap, portable, hand-held recording in the days before the smartphone and digital recording. It was standard to see a cluster of these around the lectern in a college classroom as late as the mid-1990s. Many of the recorders featured voice-activated recording (to prevent capturing “dead air”) and continuously variable playback speed to make transcription easier.
The tiny tapes were also commonly used in telephone answering machines and dictation machines.
As you may have guessed, the rise of digital recording, handheld devices, and cheap data storage quickly relegated the microcassette to a museum piece by the early 21st century. While the compact cassette has enjoyed a resurgence as a hip medium for underground music, the poor audio quality and durability of the microcassette have largely doomed it to oblivion except among the most willful obscurantists. Still, many Rubenstein Library collections contain these little guys as carriers of valuable primary source material. That means we’re holding onto our Microcassette player for the long haul in all of its atavistic glory.
“There is nothing wrong with your television set. Do not attempt to adjust the picture. We are controlling transmission. We will control the horizontal. We will control the vertical. We repeat: there is nothing wrong with your television set.”
That was part of the cold open of one of the best science fiction shows of the 1960’s, “The Outer Limits.” The implication being that by controlling everything you see and hear in the next hour, the show’s producers were about to blow your mind and take you to the outer limits of human thought and fantasy, which the show often did.
In regards to controlling the horizontal and the vertical, one of the more mysterious parts of my job is dealing with aspect ratios when it comes to digitizing videotape. The aspect ratio of any shape is the proportion of it’s dimensions. For example, the aspect ratio of a square is always 1 : 1 (width : height). That means, in any square, the width is always equal to the height, regardless of whether a square is 1-inch wide or 10-feet wide. Traditionally, television sets displayed images in a 4 : 3 ratio. So, if you owned a 20” CRT (cathode ray tube) TV back in the olden days, like say 1980, the broadcast image on the screen was 16” wide by 12” high. So, the height was 3/4 the size of the width, or 4 : 3. The 20” dimension was determined by measuring the rectangle diagonally, and was mainly used to categorize and advertise the TV.
Almost all standard-definition analog videotapes, like U-matic, Beta and VHS, have a 4 : 3 aspect ratio. But when digitizing the content, things get more complicated. Analog video monitors display pixels that are tall and thin in shape. The height of these pixels is greater than their width, whereas modern computer displays use pixels that are square in shape. On an analog video monitor, NTSC video displays at roughly 720 (tall and skinny) pixels per horizontal line, and there are 486 visible horizontal lines. If you do the math on that, 720 x 486 is not 4 : 3. But because the analog pixels display tall and thin, you need more of them aligned vertically to fill up a 4 : 3 video monitor frame.
When Duke Libraries digitizes analog video, we create a master file that is 720 x 486 pixels, so that if someone from the broadcast television world later wants to use the file, it will be native to that traditional standard-definition broadcast specification. However, in order to display the digitized video on Duke’s website, we make a new file, called a derivative, with the dimensions changed to 640 x 480 pixels, because it will ultimately be viewed on computer monitors, laptops and smart phones, which use square pixels. Because the pixels are square, 640 x 480 is mathematically a 4 : 3 aspect ratio, and the video will display properly. The derivative video file is also compressed, so that it will stream smoothly regardless of internet bandwidth limits.
“We now return control of your television set to you. Until next week at the same time, when the control voice will take you to – The Outer Limits.”
Notes from the Duke University Libraries Digital Projects Team