Type keyword(s) to search

Quick Hits

The Robots Are Coming For Your Closed Captions — And That’s No Baloney, Patsy

Laugh at the botched efforts of captioning software all you want. AI always laughs last.
  • AI doesn't know the difference between "Patsy Baloney" and "Pat Cipillone"... until it does..
    AI doesn't know the difference between "Patsy Baloney" and "Pat Cipillone"... until it does..

    During a recent episode of the January 6 Commission hearings — this summer's most-talked-about new show — comedy goddess Merrill Markoe took to Twitter to call attention to what she was seeing at the bottom of her screen:

    "Patsy Baloney" was soon trending on Twitter, but that's not the really interesting part (though it was hilarious, which was Markoe's intent). The interesting part is that the software started fixing the error by midway through the same hearing:

    This kind of on-screen mishap is common in the world of real-time closed captioning. A weather service in 2014 was reported to the FCC for a caption that declared that a looming storm "will come right assholes." People with nothing better to do might make fun of a poor caption, even if it's corrected right away.

    But word soup, like Talk Soup, is soon going to be a thing of the past. To understand why, let's rewind 50 years. Television captions might be the greatest thing in electronics today and they have never gotten their due.

    The first captions (1972-79)

    You've probably heard captions referred to as "closed captions." That means they're hidden from the regular video signal sent out by TV stations. But when advocates for the deaf began developing a system to provide captions to shows in the early 1970s, the signal didn't have that capability.

    Most of the pioneering work on making media accessible was done by PBS stations, notably WGBH-TV in Boston, where both captioning and its younger sibling Audio Description were developed. WGBH introduced captioning by burning the letters into the previously-recorded program. The French Chef aired a version with "open captions" starting in 1972.

    Right away, captioned shows began to attract fans who did not have hearing difficulties. Every weeknight PBS stations would rebroadcast the ABC network newscast from earlier that evening, open captioned. Since humans had to transcribe every word of the program, and then painstakingly burn the captions into the video, the show didn't air until late night. Which was perfect for many viewers who found they could watch it in bed with the volume turned down and not wake their sleeping partner.

    The free-market era (1980-89)

    The government, in the form of the Federal Communications Commission, or FCC, ordered Line 21 of the TV broadcast signal to be set aside for closed captions beginning in 1976. The National Captioning Institute, with mostly federal funding, created a system for real-time captioning in 1982. But all of these developments occured mostly under the radar, except within the small community of the hearing impaired. Nobody at the time imagined that captioning might some day have mass appeal.

    Thus the expansion of closed captioning was driven chiefly by market players. TV makers balked at adding Line 21 transcoders, which they said would add $200 to the cost of a set. Instead Sears and Sanyo sold converter boxes. Many popular programs were closed captioned, but the overwhelming number of shows were not.

    Slowly, though, pockets of television users — bar owners, stock traders, gym members — began to discover the joys of reading TV shows rather than watching them. Their enthusiasm for the technology would be a big reason that captioning became an early beneficiary of the disability revolution that started in 1990.

    The game-changer: ADA (1990-2010)

    Thirty-two years ago this month, President George H.W. Bush signed into law the Americans with Disabilities Act. It was the civil-rights watershed that advocates for the disabled had planned decades for. And one of the first things they took aim at following passage of the ADA was closed captioning.

    With the passage of the Television Decoder Circuitry Act of 1990, the makers of TV sets with screens 13 inches or larger (don't laugh, screens were small back then) were required to include Line 21 captioning decoders. Prices of TV sets did not go through the roof. With the arrival of digital TV signals in the mid-1990s, embedding captions in multiple languages became seamless. Real-time captions for emergency broadcasts became the law. A station that aired stupid captions during a tornado was not a laughing matter at the FCC, which responded with hefty fines.

    The future of captions

    Around the world today, captions are ubiquitous. And they are only getting more popular. Millennials who like monitoring more than one screen at a time find that captions (and audio descriptions) help them follow along with a show. Streaming services typically offer at least a dozen different captions (aka subtitles) for their shows, and a certain portion of viewers actually prefer them when watching shows that are dubbed — poorly, in their opinion — into English.

    Since 2010 TV manufacturers have been required to have an on/off button for captions on their remotes. And in 2014, the FCC passed legislation requiring TV providers to improve the accuracy of their captions. The BBC, which began closed captioning its shows in 1979, began using a "respeaker" on live sportscasts in 2003. This is an announcer who listens to the on-air commentary in their headphones, then quickly repeats it with clear e-nun-ci-ation into a microphone that goes to the captioning system. Many stations continue to employ stenographers to caption their newscasts.

    But these are human solutions, and the future of captioning is non-human. In recent years artificial intelligence has been implemented by many of the leading captioning services. Right now, the best-in-class employ a human backup who monitors the caption feed and jumps in to educate the computer when it guesses wrong on certain words or phrases — say, "Pat Cipollone." Industry forecasters believe that AI technologies like IBM's Watson will make captioning fully automated in a few years, and soon you'll be able to watch any video from any source with accurate, real-time captions.

    So laugh and screengrab those word salads while you can. The AI robots are here to clean up TV captions. Then they'll clean up your kid's tweets, so you'll be the one getting mocked, "Weird Al" style, for your sloppy old-school texts. Some day your grandkids will wonder why you insist on watching captions on "the TV" instead of having them put directly on your corneas. Just tell them it reminds you of your favorite old show, Patsy Baloney.

    Aaron Barnhart has written about television since 1994, including 15 years as TV critic for the Kansas City Star.

    TOPICS: closed-captioning, PBS, Merrill Markoe, Audio description, Disabilities and TV, FCC, January 6 Committee hearings