AI-Mimi Creates Inclusive TV Experiences for Deaf and Hard of Hearing Users in Japan

All over the world there is a growing demand for subtitles. In the UK, for example, the BBC reports that subtitles are primarily aimed at hearing-impaired viewers, but are used by a wide range of people:about 10% of viewers regularly use subtitles, with this percentage increasing to 35% for some online content. The majority of these viewers are not hard of hearing.

Similar trends are recorded around the world for television, social media and other channels that provide video content.

It is estimated that more than 360,000 people in Japan are deaf or hard of hearing – 70,000 of them use sign language as their main form of communication, while the rest prefer written Japanese as their main means of access. to content. Moreover, with nearly 30 percent of people in Japan aged 65 or older, the Japan Hearing Aid Industry Association estimates that 14.2 million people have hearing loss.

Major Japanese broadcasters have captions for the majority of their programming, requiring a process that includes dedicated staff and the use of specialist equipment valued at tens of millions of Japanese yen. “More than 100 local TV stations in Japan face obstacles in providing captions for live programs due to high equipment cost and staff limitations,” said Muneya Ichise of SI- com. Local stations are of great importance to the communities they serve, with local news programs carrying important updates about the area and its people.

To meet this need for accessibility, from 2018, SIcom and its parent company, ISCEC Japan, have been testing with local TV stations innovative and cost-effective ways to introduce captions into live streaming. Their technical solution to deliver captions for live streaming, AI Mimi, is an innovative pairing of human input and the power of Microsoft Azure Cognitive Service, creating a more accurate and faster solution through the hybrid format. In addition, ISCEC is able to compensate for the shortage of people responsible for entering subtitles locally by relying on its own specialized staff. AI-Mimi was also introduced at Okinawa University and innovation has been recognized and awarded Microsoft AI Grant for Accessibility.

Based on extensive testing and user feedback, focusing on the need for larger fonts and better on-screen display of subtitles, SI-com is able to create a template with more than 10 lines of subtitles on the right side of the TV screen, moving away from the more commonly used version with only two lines displayed at the bottom. In December 2021, they demonstrated the technology for the first time, during a live broadcast, in partnership with a local television station in Nagasaki.

TV screenshot of a demo with a local TV channel in Nagasaki

About Wendy Hall

Check Also

Flags will fly at half mast in honor of Shinzo Abe, former Prime Minister of Japan

Media contacts: Alex Reuss, 402-471-1970 Justin Pinkerman, 402-471-1967 Press release : Flags will fly at …