All posts by Alex Marsh

Behind the Scenes, Digital Collections, Digitization Expertise, Duke Digital Repository, Equipment, Technology

FFV1: The Gains of Lossless

4 Comments

One of the greatest challenges to digitizing analog moving-image sources such as videotape and film reels isn’t the actual digitization. It’s the enormous file sizes that result, and the high costs associated with storing and maintaining those files for long-term preservation. For many years, Duke Libraries has generated 10-bit uncompressed preservation master files when digitizing our vast inventory of analog videotapes.

Unfortunately, one hour of uncompressed video can produce a 100 gigabyte file. That’s at least 50 times larger than an audio preservation file of the same duration, and about 1000 times larger than most still image preservation files. That’s a lot of data, and as we digitize more and more moving-image material over time, the long-term storage costs for these files can grow exponentially.

To help offset this challenge, Duke Libraries has recently implemented the FFV1 video codec as its primary format for moving image preservation. FFV1 was first created as part of the open-source FFmpeg software project, and has been developed, updated and improved by various contributors in the Association of Moving Image Archivists (AMIA) community.

FFV1 enables lossless compression of moving-image content. Just like uncompressed video, FFV1 delivers the highest possible image resolution, color quality and sharpness, while avoiding the motion compensation and compression artifacts that can occur with “lossy” compression. Yet, FFV1 produces a file that is, on average, 1/3 the size of its uncompressed counterpart.

sleeping bag
FFV1 produces a file that is, on average, 1/3 the size of its uncompressed counterpart. Yet, the audio & video content is identical, thanks to lossless compression.

The algorithms used in lossless compression are complex, but if you’ve ever prepared for a fall backpacking trip, and tightly rolled your fluffy goose-down sleeping bag into one of those nifty little stuff-sacks, essentially squeezing all the air out of it, you just employed (a simplified version of) lossless compression. After you set up your tent, and unpack your sleeping bag, it decompresses, and the sleeping bag is now physically identical to the way it was before you packed.

Yet, during the trek to the campsite, it took up a lot less room in your backpack, just like FFV1 files take up a lot less room in our digital repository. Like that sleeping bag, FFV1 lossless compression ensures that the compressed video file is mathematically identical to it’s pre-compressed state. No data is “lost” or irreversibly altered in the process.

Duke Libraries’ Digital Production Center utilizes a pair of 6-foot-tall video racks, which house a current total of eight videotape decks, comprised of a variety of obsolete formats such as U-matic (NTSC), U-matic (PAL), Betacam, DigiBeta, VHS (NTSC) and VHS (PAL, Secam). Each deck is converted from analog to digital (SDI) using Blackmagic Design Mini Converters.

The SDI signals are sent to a Blackmagic Design Smart Videohub, which is the central routing center for the entire system. Audio mixers and video transcoders allow the Digitization Specialist to tweak the analog signals so the waveform, vectorscope and decibel levels meet broadcast standards and the digitized video is faithful to its analog source. The output is then routed to one of two Retina 5K iMacs via Blackmagic UltraStudio devices, which convert the SDI signal to Thunderbolt 3.

FFV1 video digitization in progress in the Digital Production Center.

Because no major company (Apple, Microsoft, Adobe, Blackmagic, etc.) has yet adopted the FFV1 codec, multiple foundational layers of mostly open-source systems software had to be installed, tested and tweaked on our iMacs to make FFV1 work: Apple’s Xcode, Homebrew, AMIA’s vrecord, FFmpeg, Hex Fiend, AMIA’s ffmprovisr, GitHub Desktop, MediaInfo, and QCTools.

FFV1 operates via terminal command line prompts, so some understanding of programming language is helpful to enter the correct prompts, and be able to decipher the terminal logs.

The FFV1 files are “wrapped” in the open source Matroska (.mkv) media container. Our FFV1 scripts employ several degrees of quality-control checks, input logs and checksums, which ensure file integrity. The files can then be viewed using VLC media player, for Mac and Windows. Finally, we make an H.264 (.mp4) access derivative from the FFV1 preservation master, which can be sent to patrons, or published via Duke’s Digital Collections Repository.

An added bonus is that, not only can Duke Libraries digitize analog videotapes and film reels in FFV1, we can also utilize the codec (via scripting) to target a large batch of uncompressed video files (that were digitized from analog sources years ago) and make much smaller FFV1 copies, that are mathematically lossless. The script runs checksums on both the original uncompressed video file, and its new FFV1 counterpart, and verifies the content inside each container is identical.

Now, a digital collection of uncompressed masters that took up 9 terabytes can be deleted, and the newly-generated batch of FFV1 files, which only takes up 3 terabytes, are the new preservation masters for that collection. But no data has been lost, and the content is identical. Just like that goose-down sleeping bag, this helps the Duke University budget managers sleep better at night.

Equipment, Technology

Videotelephony, Better Late than Never

1 Comment

A technology allowing most of us to keep working effectively during the COVID-19 pandemic is called “videotelephony,” which is real-time, simultaneous audio-visual communication between two or more users. Right now, millions of workers and families are using Zoom, FaceTime, WhatsApp, WebEx, Skype and other software to see and hear each other live, using the built-in microphones and video cameras on our computers, tablets and mobile phones.

We take this capability for granted now, but it’s actually been over a century in the making. Generations of trial and error, billions in spent capital, technical brick walls and failed business models have paved the way to this morning’s Zoom meeting with your work team. You might want to change out of your pajamas, by the way.

AT&T’s Picturephone (Model 1) was introduced at the 1964 World’s Fair.

Alexander Graham Bell famously patented the telephone in 1876. Shortly after, the concept of not only hearing the person you are talking to, but also seeing them simultaneously, stirred the imagination of inventors, writers and artists. It seemed like a reasonably-attainable next step. Early terms for a hypothetical device that could accomplish this included the “Telephonoscope” and the “Telectroscope”

Mr. Bell himself conceived of a device called an “electrical radiophone,” and predicted “the day would come when the man at the telephone would be able to see the distant person to whom he was speaking.” But that day would not come until long after Bell’s death in 1922.

The problem was, the transmission of moving images was a lot more complicated than transmitting audio. Motion picture film, also introduced in the late 1800s, was brought to life by chemicals reacting to silver-halide crystals in a darkroom, but unlike the telephone, electricity played no part in film’s construction or dissemination.

The telephone converted sound waves to electrical signals, as did radio station towers. Neither could transmit without electricity. And a telephone is “full-duplex,” meaning the data is transmitted in both directions, simultaneously, on a single carrier. The next challenge was to somehow electrify moving images, make them full-duplex, and accommodate their exponentially larger bandwidth.

The Picturephone (Model 2). Only a few hundred were sold in the 1970s.

It wasn’t until the late 1930s that cathode-ray-tube television sets were introduced to the world, and the concept of analog video began to gain traction. Unlike motion picture film, video is an electronic medium. Now that moving images were utilizing electricity, they could be transmitted to others, using antennas.

After World War II ended, and Americans had more spending money, black & white television sets became popular household items in the 1950s. But unlike the telephone, communication was still one way. It wasn’t full-duplex. You could see “The Honeymooners,” but they couldn’t see you, and it wasn’t live.  Live television broadcasts were rare, and still in the experimental phase.

In 1964, AT&T’s Bell Labs (originally founded by Alexander Graham Bell), introduced the “Picturephone” at the New York World’s Fair and at Disneyland, demonstrating a video call between the two locales. Later, AT&T introduced public videophone booths in New York City, Chicago and Washington, DC. If you were in the New York videophone booth, you could see and hear someone in the Chicago videophone booth, in real time, and it was two-way communication.

The problem was, it was outrageously expensive. A three-minute call cost $225 in today’s money. The technology was finally here, but who could afford it? AT&T poured billions into this concept for years, manufacturing “PicturePhones” and “VideoPhones” for home and office, all the way through 1995, but they were always hampered by the limitations of low-bandwidth telephone lines and very high prices, making them not worth it for the consumer, and never widely adopted.

AT&T’s VideoPhone 2500, released in 1992, priced at $1599.99.

It wasn’t until broadband internet, and high-compression video codecs became widespread in the new millennium, that videotelephony finally became practical, affordable and thus marketable. In recent years, electronics manufacturers began to include video cameras and microphones as a standard feature in CPUs, tablets and mobile phones, making external webcams obsolete. Services like Skype, FaceTime and WebEx were introduced, and later WhatsApp, Zoom and numerous others.

Now it’s simple, and basically free, to have a high-quality, full-color video chat with your friend, partner or co-worker, and a company like Zoom has a net worth of 40 billion. It’s amazing that it took more than 100 years since the invention of the telephone to get here. And just in time for a global pandemic requiring strict physical distancing. Don’t forget to update your clever background image!

Behind the Scenes, Digital Collections, Equipment

All About that Time Base

The video digitization system in Duke Libraries’ Digital Production Center utilizes many different pieces of equipment: power distributors, waveform and vectorscope monitors, analog & digital routers, audio splitters & decibel meters, proc-amps, analog (BNC, XLR and RCA) to digital (SDI) converters, CRT & LCD video monitors, and of course an array of analog video playback decks of varying flavors (U-matic-NTSC, U-matic-PAL, Betacam SP, DigiBeta, VHS-NTSC and VHS-PAL/SECAM). We also transfer content directly from born-digital DV and MiniDV tapes.

A grandfather clock is a time base.

One additional component that is crucial to videotape digitization is the Time Base Corrector (TBC). Each of our analog video playback decks must have either an internal or external TBC, in order to generate an image of acceptable quality. At the recent Association of Moving Image Archivist’s Conference in Baltimore, George Blood (of George Blood Audio/Video/Film/Data) gave a great presentation on exactly what a Time Base Corrector is, appropriately entitled “WTF is a TBC?” Thanks to George for letting me relay some of his presentation points here.

A time base is a consistent reference point that one can utilize to stay in sync. For example, The Earth rotating around the Sun is a time base that the entire human race relies on, to stay on schedule. A grandfather clock is also a time base. And so is a metronome, which a musical ensemble might use to all stay “in time.”

Frequency is defined as the number of occurrences of a repeating event per unit of time. So, the frequency of the Earth rotating around the Sun is once per 24 hrs. The frequency of a grandfather clock is one pendulum swing per second. The clock example can also be defined as one “cycle per second” or one hertz (Hz), named after Heinrich Hertz, who first conclusively proved the existence of electromagnetic waves in the late 1800’s.

One of the DPC’s external Time Base Correctors

But anything mechanical, like grandfather clocks and videotape decks, can be inconsistent. The age and condition of gears and rods and springs, as well as temperature and humidity, can significantly affect a grandfather clock’s ability to display the time correctly.

Videotape decks are similar, full of numerous mechanical and electrical parts that produce infinite variables in performance, affecting the deck’s ability to play the videotape’s frames-per-second (frequency) in correct time.

NTSC video is supposed to play at 29.97 frames-per-second, but due to mechanical and electro-magnetic variables, some frames may be delayed, or some may come too fast. One second of video might not have enough frames, another second may have too many. Even the videotape itself can stretch, expand and contract during playback, throwing off the timing, and making the image wobbly, jittery, too bright or dark, too blue, red or green.

A Time Base Corrector does something awesome. As the videotape plays, the TBC stores the unstable video content briefly, fixes the timing errors, and then outputs the corrected analog video signal to the DPC’s analog-to-digital converters. Some of our videotape decks have internal TBCs, which look like a computer circuit board (shown below). Others need an external TBC, which is a smaller box that attaches to the output cables coming from the videotape deck (shown above, right). Either way, the TBC can delay or advance the video frames to lock them into correct time, which fixes all the errors.

An internal Time Base Corrector card from a Sony U-matic BVU-950 deck

An internal TBC is actually able to “talk” to the videotape deck, and give it instructions, like this…

“Could you slow down a little? You’re starting to catch up with me.”

“Hey, the frames are arriving at a strange time. Please adjust the timing between the capstan and the head drum.”

“There’s a wobble in the rate the frames are arriving. Can you counter-wobble the capstan speed to smooth that out?”

“Looks like this tape was recorded with bad heads. Please increase gain on the horizontal sync pulse so I can get a clearer lock.”

Without the mighty TBC, video digitization would not be possible, because all those errors would be permanently embedded in the digitized file. Thanks to the TBC, we can capture a nice, clean, stable image to share with generations to come, long after the magnetic videotape, and playback decks, have reached the end of their shelf life.

Digitization Expertise, Equipment

U-matic for the People

Duke Libraries has a large collection of analog videotapes, in several different formats. One of the most common in our archives is 3/4″ videotape, also called “U-matic” (shown above). Invented by Sony in 1969, U-matic was the first videotape to be housed inside a plastic cassette for portability. Before U-matic, videotape was recorded on very large reels in the 2″ format known as Quadruplex which required heavy recording and playback machines the size of household refrigerators. U-matic got its name from the shape of the tape path as it wraps around the video head drum, which looks like the letter U.

The VO-3800 enabled TV news crews to record directly to U-matic videotape at breaking news events.

The format was officially released in 1971, and soon became popular with television stations, when the portable Sony VO-3800 video deck was released in 1974. The VO-3800 enabled TV crews to record directly to U-matic videotape at breaking news events, which previously had to be shot with 16mm film. The news content was now immediately available for broadcast, as opposed to film, which had to wait for processing in a darkroom. And the compact videocassettes could easily and quickly be transported to the TV station.

In the 1970’s, movie studios also used U-matic tapes to easily transport filmed scenes or “dailies,” such as the first rough cut of “Apocalypse Now.” In 1976, the high-band BVU (Broadcast Video U-matic) version of 3/4″ videotape, with better color reproduction and lower noise levels, replaced the previous “lo-band” version.

The Digital Production Center’s Sony VO-9800P for PAL videotapes (top), and a Sony BVU-950 for NTSC tapes (bottom).

The U-matic format remained popular at TV stations throughout the 1980’s, but was soon replaced by Sony’s 1/2″ Betacam SP format. The BVU-900 series was the last U-matic product line made by Sony, and Duke Libraries’ Digital Production Center uses two BVU-950s for NTSC tapes, as well as a VO-9800P for tapes in PAL format. A U-matic videotape player in good working order is now an obsolete collector’s item, so they can be hard to find, and expensive to purchase.

Unfortunately, most U-matic tapes have not aged well. After decades in storage, many of the videotapes in our collection now have sticky-shed syndrome, a condition in which the oxide that holds the visual content is literally flaking off the polyester tape base, and is moist and gummy in texture. When a videotape has sticky-shed, not only will it not play correctly, the residue can also clog up the tape heads in the U-matic playback deck, then transfer the contaminant to other tapes played afterwards in the same deck.

The DPC’s RTI VT3100 U-matic tape cleaner.

To combat this, we always bake (dehumidify) our U-matic videotapes in a scientific oven at 52 celsius (125 fahrenheit) for at least 10 hours. Then we run each tape through a specialized tape-cleaning machine, which fast-forwards and rewinds each tape, while using a burnishing blade to wipe off any built-up residue. We also clean the video heads inside our U-matic decks before each playback, using denatured alcohol.

Most of the time, these procedures make the U-matic tape playable, and we are able to digitize them, which rescues the content from the videotapes, before the magnetic tape ages and degrades any further. While the U-matic tapes are nearing the end of their life-span, the digital surrogates will potentially last for centuries to come, and will be accessible online through our Duke Digital Repository, from anywhere in the world.

Behind the Scenes, Conferences

Something Good

One of the highlights of the Association of Moving Image Archivists’ annual conference is “Archival Screening Night,” where members of the AMIA community showcase recently-discovered and newly-restored film and video footage. The event usually takes place in a historic movie theater, with skilled projectionists that are able to present the film materials in their original format, on the big screen. At the most recent AMIA conference, in Portland, Oregon, there was a wide array of impressive material presented, but one film in particular left the audience speechless, and is a wonderful example of how archivists can unearth treasures that can alter our perspective on human history, and humanity itself.

The film, “Something Good – Negro Kiss” was made in 1898. It’s silent, black & white, and is less than a minute long. But it’s groundbreaking, in that it shows the earliest known depiction of an African-American couple kissing, and stands in opposition to the racist, minstrel-show portrayals of black people so common in the early days of American filmmaking. The couple embrace, kiss, and sway back and forth in a playful, spontaneous dance that comes across as genuine and heartwarming. Although it may not have been intentional, the short film seems to be free of negative racial stereotypes. You can watch it here:

Dino Everett, an archivist at the University of Southern California’s Hugh M. Hefner Moving Image Archive, recently discovered the 35mm nitrate film within a batch of silent films once owned by a Louisiana collector. Unique perforation marks helped him to identify the film’s age, and Allyson Nadia Field, of the University of Chicago, was able to help track down the history: where it was shot (Chicago), who the filmmaker was (William Selig), and what the original title of the the film was (Something Good). The actors have been identified as Saint Suttle and Gertie Brown. The film has now been added to the Library of Congress’ National Film Registry.

The film is likely an homage to “The Kiss” (also known as the May Irwin Kiss), a film made in 1896, with a white couple kissing. It was one of the first films ever shown commercially, and is the very first kiss on film. Even though the couple was white, and the kissing is remarkably tame by today’s standards,  it created a lot of controversy at the time, because kissing in public was prohibited by law.  The Catholic church and newspaper editorials denounced “The Kiss” and called for censorship and prosecution.  Although there is no documented history yet about the public reaction to “Something Good – Negro Kiss,” one can only imagine the shock and scandal it must have caused, showing an African-American couple kissing each other, only two years later.

Technology

Enter the Vortex

When Duke students tour the Digital Production Center, I always show them our video digitization system, and point out that Duke Libraries’ collection of U-matic, VHS and Betacam analog videotapes are ancient relics from the last century. This fall’s first-year students were born in the new millennium. They have little use for physical media, except for perhaps an occasional external thumb-drive. Their concept of video is something you capture on your iPhone or stream online, not play using a crude plastic rectangular-shell. And rewinding videotape? “Like… what is that?, it’s so… weird!”

So, imagine my surprise when I recently walked into Raleigh’s brand new Alamo Drafthouse Cinema, and entered their Video Vortex, a massive library of over 75,000 video titles on VHS and DVD, that are free for customers to check out and watch at home. Video Vortex even rents out VHS players, another historical artifact. This may seem odd at a time when everyone is streaming movie content online. But, Video Vortex specializes in movies you can’t get from Netflix, Amazon and other streaming services. Many of their titles are out of print, and some of these films were never released on DVD, or in any digital format, so the only way you can see them is on VHS.

Blood Orgy of the Leather Girls (1988). Directed by Meredith Lucas

Walking through the VHS collection is like going to a run-down grindhouse movie theater in 1975, or tuning into an obscure cable-TV channel at 3 A.M. in 1987. Many of the films would be classified as “exploitation:” cheaply-made horror, cult or action titles that never had a chance at the Oscars, and are “so bad, they’re good,” like “Blood Orgy of the Leather Girls.” But there’s also critically acclaimed films like Frank Perry’s “Last Summer,” which earned an academy award nomination in 1969. Due to copyright issues or lack of funds, these two films have never made it into the digital realm, and can only be seen on VHS.

Last Summer (1969). Directed by Frank Perry

Josh Schafer is co-manager and “VHS Culture Captain” of Video Vortex. He moved here from New Jersey to work in the vortex, because he’s a longtime connoisseur and expert on the VHS format, and even publishes a VHS fanzine called “Lunchmeat.”

“The whole goal here is to not just reimagine the video store, and give people that feeling and experience again, but also give people this library, this community asset where both film-lovers and the casual movie-goer alike, can come in and explore all kinds of cinema, for free,” says Schafer. The Alamo’s lobby, where Video Vortex lives, is decorated with rare movie posters, giant VHS facsimiles, and has tables where film-nerds can congregate, order from the Alamo’s full kitchen and bar, and discuss their favorite obscure animation titles.

Video Vortex has every film by every director in the known universe.

Skip Elsheimer of AV Geeks has taken on the job of helping to maintain the Video Vortex collection, which involves cleaning off mold, splicing tape, fixing cases and repairing DVDs. “A lot of these videotapes and DVDs were boxed up for years in storage spaces that were not ideal,” says Skip. “We do TLC on these titles, many of which don’t exist in any other format.”

DVDs are actually harder to fix and reclaim than videotape. Skip says the rescue rate of VHS is about 90%, because he can swap out tape, put it in a new cassette case, splice it, etc. But once the lamination separates on a DVD, or if there’s a significant scratch, it’s toast, because the laser can no longer read the data, and there’s no way to retrieve it. So much for the idea of digital = permanent. Or as the VHS Culture Captain says, “only analog is real.”

In addition to Video Vortex, Alamo Drafthouse Cinema offers a mix of first-run, independent and vintage films on 11 screens. The comfy theater seats recline, and customers can order an eclectic mix of foods, cocktails, craft-beers and wine, right from their seat. Most everyone who works at Alamo is a movie fan, and it shows in everything from the vintage movie posters that line the walls, to the enthusiasm of the employees. The only way to dampen that enthusiasm is if you talk or text in the theater, because, after one warning, you will be asked to leave, as explained in this colorful public service announcement.

Equipment, Technology

Adventures in 4K

2 Comments

When it comes to moving image digitization, Duke Libraries’ Digital Production Center primarily deals with obsolete videotape formats like U-matic, Betacam, VHS and DV, which are in standard-definition (SD). We typically don’t work with high-definition (HD) or ultra-high-definition (UHD) video because that’s “born digital,” and doesn’t need any kind of conversion from analog, or real-time migration from magnetic tape. It’s already in the form of a digital file.

However, when I’m not at Duke, I do like to watch TV at home, in high-definition. This past Christmas, the television in my living room decided to kick the bucket, so I set out to get a new one. I went to my local Best Buy and a few other stores, to check out all the latest and greatest TVs. The first thing I noticed is that just about every TV on the market now features 4K ultra-high-definition (UHD), and many have high dynamic range (HDR).

Before we dive into 4K, some history is in order. Traditional, standard-definition televisions offered 480 lines of vertical resolution, with a 4:3 aspect ratio, meaning the height of the image display is 3/4 the dimension of the width. This is how television was broadcast for most of the 20th century. Full HD television, which gained popularity at the turn of the millennium, has 1080 pixels of vertical resolution (over twice as much as SD), and an aspect ratio of 16:9, which makes the height barely more than 1/2 the size of the width.

16:9 more closely resembles the proportions of a movie theater screen, and this change in TV specification helped to usher in the “home theater” era. Once 16:9 HD TVs became popular, the emergence of Blu-ray discs and players allowed consumers to rent or purchase movies, watch them in full HD and hear them in theater-like high fidelity, by adding 5.1 surround sound speakers and subwoofers. Those who could afford it started converting their basements and spare rooms into small movie theaters.

4K UHD has 3840 horizontal pixels and 2160 vertical pixels, twice as much resolution as HD, and almost five times more resolution than SD.

The next step in the television evolution was 4K ultra-high-definition (UHD) TVs, which have flooded big box stores in recent years. 4K UHD has an astounding resolution of 3840 horizontal pixels and 2160 vertical pixels, twice as much resolution as HD, and almost five times more resolution than SD. Gazing at the images on these 4K TVs in that Best Buy was pretty disorienting. The image is so sharp and finely-detailed, that it’s almost too much for your eyes and brain to process.

For example, looking at footage of a mountain range in 4K UHD feels like you’re seeing more detail than you would if you were actually looking at the same mountain range in person, with your naked eye. And high dynamic range (HDR) increases this effect, by offering a much larger palette of colors and more levels of subtle gradation from light to dark. The latter allows for more detail in the highlight and shadow areas of the image. The 4K experience is a textbook example of hyperreality, which is rapidly encroaching into every aspect of our modern lives, from entertainment to politics.

The next thing that dawned on me was: If I get a 4K TV, where am I going to get the 4K content? No television stations or cable channels are broadcasting in 4K and my old Blu-ray player doesn’t play 4K. Fortunately, all 4K TVs will also display 1080p HD content beautifully, so that warmed me up to the purchase. It meant I didn’t have to immediately replace my Blu-ray player, or just stare at a black screen night after night, waiting for my favorite TV stations to catch up with the new technology.

The salesperson that was helping me alerted me to the fact that Best Buy also sells 4K UHD Blu-ray discs and 4K-ready Blu-ray players, and that some content providers, like Netflix, are streaming many shows in 4K and in HDR, like “Stranger Things,” “Daredevil” and “The Punisher,” to name a few. So I went ahead with the purchase and brought home my new 4K TV. I also picked up a 4K-enabled Roku, which allows anyone with a fast internet connection and subscription to stream content from Netflix, Amazon and Hulu, as well as accessing most cable-TV channels via services like DirecTV Now, YouTube TV, Sling and Hulu.

I connected the new TV (a 55” Sony X800E) to my 4K Roku, ethernet, HD antenna and stereo system and sat down to watch. The 1080p broadcasts from the local HD stations looked and sounded great, and so did my favorite 1080p shows streaming from Netflix. I went with a larger TV than I had previously, so that was also a big improvement.

To get the true 4K HDR experience, I upgraded my Netflix account to the 4K-capable version, and started watching the new Marvel series, “The Punisher.” It didn’t look quite as razor sharp as the 4K images did in Best Buy, but that’s likely due to the fact that the 4K Netflix content is more compressed for streaming, whereas the TVs on the sales floor are playing 4K video in-house, that has very little, if any, compression.

As a test, I went back and forth between watching The Punisher in 4K UHD, and watching the same Punisher episodes in HD, using an additional, older Roku though a separate HDMI port. The 4K version did have a lot more detail than its HD counterpart, but it was also more grainy, with horizons of clear skies showing additional noise, as if the 4K technology is trying too hard to bring detail out of something that is inherently a flat plane of the same color.

Also, because of the high dynamic range, the image loses a bit of overall contrast when displaying so many subtle gradations between dark and light. 4K streaming also requires a fast internet connection and it downloads a lot of data, so if you want to go 4K, you may need to upgrade your ISP plan, and make sure there are no data caps. I have a 300 Mbps fiber connection, with ethernet cable routed to my TV, and that works perfectly when I’m streaming 4K content.

I have yet to buy a 4K Blu-ray player and try out a 4K Blu-ray disc, so I don’t know how that will look on my new TV, but from what I’ve read, it more fully takes advantage of the 4K data than streaming 4K does. One reason I’m reluctant to buy a 4K Blu-ray player gets back to content. Almost all the 4K Blu-ray discs for sale or rent now are recently-made Hollywood movies. If I’m going to buy a 4K Blu-ray player, I want to watch classics like “2001: A Space Odyssey,” “The Godfather,” “Apocalypse Now” and “Vertigo” in 4K, but those aren’t currently available because the studios have yet to release them in 4K. This requires going back to the original film stock and painstakingly digitizing and restoring them in 4K.

Some older films may not have enough inherent resolution to take full advantage of 4K, but it seems like films such as “2001: A Space Odyssey,” which was originally shot in 65 mm, would really be enhanced by a 4K restoration. Filmmakers and the entertainment industry are already experimenting with 8K and 16K technology, so I guess my 4K TV will be obsolete in a few years, and we’ll all be having seizures while watching TV, because our brains will no longer be able to handle the amount of data flooding our senses.

Prepare yourself for 8K and 16K video.

 

Behind the Scenes, Collections

The Letter Compels You!

1 Comment

Every Halloween at Duke Libraries, we have our annual “Screamfest.” This is when the David M. Rubenstein Rare Book & Manuscript Library shows off unique holdings related to extrasensory perception, premature burial, 16th century witches, devils (not just blue ones), creepy advertisements, eerie pulp fiction, scary zines and more. Attendees sometimes show up in costumes, and there is of course, lots of candy. I always eat too much.

When I look through the various materials on display, there is one item in particular that always seems to draw me in. In fact, you could say I am compelled to read it, almost as if I am not in control of my actions! It’s a simple one-page letter, written in 1949 by Luther M. Schulze, a Lutheran pastor in Washington, D.C., addressed to J.B Rhine, the scientist who founded parapsychology as a branch of psychology, and started the Duke Parapsychology Laboratory, which operated at Duke University from 1935 until the 1960’s. Parapsychology is the study of phenomena such as telepathy, clairvoyance, hypnosis, psychokinesis and other paranormal mysteries.

The 1949 letter from the Rev. Luther Schulze to J.B. Rhine. (click to enlarge)

The letter begins: “We have in our congregation a family who are being disturbed by poltergeist phenomena. It first appeared about January 15, 1949. The family consists of the maternal grandmother, a fourteen (year) old boy who is an only child, and his parents. The phenomena is present only in the boy’s presence. I had him in my home on the night of February 17-18 to observe for myself. Chairs moved with him and one threw him out. His bed shook whenever he was in it.” The letter also states that his family says that “words appeared on the boy’s body” and he “has visions of the devil and goes into a trance and speaks in a strange language”

This letter immediately reminded me of “The Exorcist,” one of the best horror movies of all time. I was too young to see the film when it was originally released in 1973, but got the chance to see the director’s cut on the big screen in 2000. It’s a very unsettling and frightening film, but not because of gratuitous gore like you see in today’s monotonously-sadistic slasher films. The Exorcist is one of the scariest movies ever because it expertly taps into a central fear within our Judeao-Christian collective subconscious: That evil isn’t just something we battle outside of ourselves. The most frightening evil of all is that which can take root within us.

It turns out there’s a direct link between this mysterious letter to J.B Rhine and “The Exorcist.” William Peter Blatty, who wrote the 1971 novel and adapted it for the film, based his book on a real-life 1949 exorcism performed by Jesuit priests in St. Louis. The exorcism was performed on a 14-yr-old boy under the pseudonym of “Roland Doe” and that is the same boy that Rev. Schulze is referring to in his letter to J.B. Rhine at Duke. When Rhine received the letter, Roland’s family had taken him to St. Louis for the exorcism, having given up on conventional psychiatry. Blatty changed the gender and age of the child for his novel and screenplay, but many of the occurrences described in the letter are recognizable to anyone familiar with the book or movie.

The reply from J.B. Rhine to the Rev. Luther Schulze. (click to enlarge)

Unfortunately for this blog post, poltergeists or demons or psychosomatic illnesses (depending on your point of view) often vanish as unexpectedly as they show up, and that’s what happened in this case. After an initial reply to the letter from L.E. Rhine, his wife and lab partner, J.B. Rhine responded to Rev. Schulze that he was “deeply interested in this case,” and that “the most likely normal explanation is that the boy is, himself led to create the effect of being the victim of mysterious agencies or forces and might be sincerely convinced of it. Such movements as those of the chair and bed might, from your very brief account of them, have originated within himself.” Part of the reason Rhine was successful in his field is that he was an empirical skeptic. Rhine later visited Schulze in person, but by then, the exorcism had ended, and Roland’s condition had returned to normal.

According to subsequent research, Roland married, had children and leads a quiet, ordinary life near Washington, D.C. He refuses to talk about the events of 1949, other than saying he doesn’t remember. In the mid-1960’s, Duke and J.B. Rhine parted ways, and the Duke Parapsychology Lab closed. This was likely due in part to the fact that, despite Rhine’s extensive research and empirical testing, parapsychology was, and still is, considered a dubious pseudoscience. Duke probably realized the association wasn’t helping their reputation as a stellar academic institution. The Rhines continued their research, setting up the “Foundation for Research on the Nature of Man,” independently of Duke. But the records of the Duke Parapsychology Laboratory are available for study at Duke Libraries. I wonder what other dark secrets might be discovered, brought to light and exorcized?

Digitization Expertise, Equipment, Technology

A History of Videotape, Part 1

3 Comments

As a Digital Production Specialist at Duke Libraries, I work with a variety of obsolete videotape formats, digitizing them for long-term preservation and access. Videotape is a form of magnetic tape, consisting of a magnetized coating on one side of a strip of plastic film. The film is there to support the magnetized coating, which usually consists of iron oxide. Magnetic tape was first invented in 1928, for recording sound, but it would be several decades before it could be used for moving images, due to the increased bandwidth that is required to capture the visual content.

Bing Crosby was the first major entertainer who pushed for audiotape recordings of his radio broadcasts. in 1951, his company, Bing Crosby Enterprises (BCE) debuted the first videotape technology to the public.

Television was live in the beginning, because there was no way to pre-record the broadcast other than with traditional film, which was expensive and time-consuming. In 1951, Bing Crosby Enterprises (BCE), owned by actor and singer Bing Crosby, demonstrated the first videotape recording. Crosby had previously incorporated audiotape recording into the production of his radio broadcasts, so that he would have more time for other commitments, like golf! Instead of having to do a live radio broadcast once a week for a month, he could record four broadcasts in one week, then have the next three weeks off. The 1951 demonstration ran quarter-inch audiotape at 360 inches per second, using a modified Ampex 200 tape recorder, but the images were reportedly blurry and not broadcast quality.

Ampex introduced 2” quadruplex videotape at the National Association of Broadcasters convention in 1956. Shown here is a Bosch 2″ Zoll Quadruplex Machine.

More companies experimented with the emerging technology in the early 1950’s, until Ampex introduced 2” black and white quadruplex videotape at the National Association of Broadcasters convention in 1956. This was the first videotape that was broadcast quality. Soon, television networks were broadcasting pre-recorded shows on quadruplex, and were able to present them at different times in all four U.S. time zones. Some of the earliest videotape broadcasts were CBS’s “The Edsel Show,” CBS’s “Douglas Edwards with the News,” and NBC’s “Truth or Consequences.” In 1958, Ampex debuted a color quadruplex videotape recorder. NBC’s “An Evening with Fred Astaire” was the first major TV show to be videotaped in color, also in 1958.

Virtually all the videotapes of the first ten years (1962-1972) of “The Tonight Show with Johnny Carson” were taped over by NBC to save money, so no one has seen these episodes since broadcast, nor will they… ever.

 

One of the downsides to quadruplex, is that the videotapes could only be played back using the same tape heads which originally recorded the content. Those tape-heads wore out very quickly, which mean’t that many tapes could not be reliably played back using the new tape-heads that replaced the exhausted ones. Quadruplex videotapes were also expensive, about $300 per hour of tape. So, many TV stations maximized the expense, by continually erasing tapes, and then recording the next broadcast on the same tape. Unfortunately, due to this, many classic TV shows are lost forever, like the vast majority of the first ten years (1962-1972) of “The Tonight Show with Johnny Carson,” and Super Bowl II (1968).

Quadruplex was the industry standard until the introduction of 1” Type C, in 1976. Type C video recorders required less maintenance, were more compact and enabled new functions, like still frame, shuttle and slow motion, and 1” Type C did not require time base correction, like 2” Quadruplex did. Type C is a composite videotape format, with quality that matches later component formats like Betacam. Composite video merges the color channels so that it’s consistent with a broadcast signal. Type C remained popular for several decades, until the use of videocassettes gained in popularity. We will explore that in a future blog post.

Behind the Scenes, Digitization Expertise, Technology

The Outer Limits of Aspect Ratios

“There is nothing wrong with your television set. Do not attempt to adjust the picture. We are controlling transmission. We will control the horizontal. We will control the vertical. We repeat: there is nothing wrong with your television set.”

That was part of the cold open of one of the best science fiction shows of the 1960’s, “The Outer Limits.” The implication being that by controlling everything you see and hear in the next hour, the show’s producers were about to blow your mind and take you to the outer limits of human thought and fantasy, which the show often did.

In regards to controlling the horizontal and the vertical, one of the more mysterious parts of my job is dealing with aspect ratios when it comes to digitizing videotape. The aspect ratio of any shape is the proportion of it’s dimensions. For example, the aspect ratio of a square is always 1 : 1 (width : height). That means, in any square, the width is always equal to the height, regardless of whether a square is 1-inch wide or 10-feet wide. Traditionally, television sets displayed images in a 4 : 3 ratio. So, if you owned a 20” CRT (cathode ray tube) TV back in the olden days, like say 1980, the broadcast image on the screen was 16” wide by 12” high. So, the height was 3/4 the size of the width, or 4 : 3. The 20” dimension was determined by measuring the rectangle diagonally, and was mainly used to categorize and advertise the TV.

 

 

Almost all standard-definition analog videotapes, like U-matic, Beta and VHS, have a 4 : 3 aspect ratio. But when digitizing the content, things get more complicated. Analog video monitors display pixels that are tall and thin in shape. The height of these pixels is greater than their width, whereas modern computer displays use pixels that are square in shape. On an analog video monitor, NTSC video displays at roughly 720 (tall and skinny) pixels per horizontal line, and there are 486 visible horizontal lines. If you do the math on that, 720 x 486 is not 4 : 3. But because the analog pixels display tall and thin, you need more of them aligned vertically to fill up a 4 : 3 video monitor frame.


When Duke Libraries digitizes analog video, we create a master file that is 720 x 486 pixels, so that if someone from the broadcast television world later wants to use the file, it will be native to that traditional standard-definition broadcast specification. However, in order to display the digitized video on Duke’s website, we make a new file, called a derivative, with the dimensions changed to 640 x 480 pixels, because it will ultimately be viewed on computer monitors, laptops and smart phones, which use square pixels. Because the pixels are square, 640 x 480 is mathematically a 4 : 3 aspect ratio, and the video will display properly. The derivative video file is also compressed, so that it will stream smoothly regardless of internet bandwidth limits.

“We now return control of your television set to you. Until next week at the same time, when the control voice will take you to – The Outer Limits.”