I think we can all agree that technology has the ability to improve lives. This is especially true for people with disabilities and special needs, for whom technology can make the world more accessible. Smart glasses have a lot of potential in this area.
We have already talked about how smart glasses can help those with low vision, but they can also make a significant difference for people with hearing loss. In this article, I will be telling you about the innovative applications of smart glasses for the deaf and hard of hearing.
Benefits of smart glasses for the deaf and people with impaired hearing
But how would the deaf or hard of hearing even benefit from smart glasses, you might ask? The reason lies in the fact that smart glasses are able to take audible information and turn them into visual information. Thanks to such smart glasses, the deaf and hard of hearing are now able to acquire the information they aren’t able to hear through visual means.
In most cases, the information they need to receive is, as one might have guessed, human speech. Smart glasses can be very beneficial in this, especially in situations where the other speaker doesn’t know sign language.
Lip reading is an option, though it might not always be accurate and can be difficult, especially in cases where the speaker has a thick foreign accent or mumbles a lot. Hearing aids might also not be accessible to everyone.
Here is where smart glasses can be incredibly useful. The content of the other person’s speech can be delivered to the user through the smart glasses by using either speech to text technologies or through displaying a sign language interpreter.
In using speech to text technologies, the speaker’s speech is converted into text and the text is then displayed inside the smart glasses. The use of speech to text technologies is already a familiar phenomenon, often used in smartphone apps, for example.
However, including smart glasses in the equation is a game-changer exactly for the reason that all the information is displayed in one field of view. With the smart glasses on, the user doesn’t need to direct their glance at another gadget – they can still be looking at the person, while also understanding what they’re saying.
The same applies for a sign language interpreter being displayed inside the glasses – it is much more convenient to be able to look at both the speaker and the displayed sign language interpreter at the same time.
The deaf and hard of hearing could definitely benefit from smart glasses in many situations. From what I could find, however, there isn’t currently one widely used model of smart glasses for the hearing impaired out on the market. Several different developers have given their effort but none have yet taken off as a wide-spread product.
Glasses notifying the user of danger
Before moving on the solutions of converting speech into text, let’s first talk about the important topic of safety. Someone deaf or hard of hearing could have trouble in a situation where there is a danger that is only indicated by sound. To tackle this problem, Sangjin Joo has designed ‘alarm glasses’ – smart glasses that warn the user with hearing loss of danger.
The smart glasses notify the user when there is a loud sound, such as a car horn, shouting sound or alarm. The glasses notify the user either by displaying a visual animation, which differentiates between the color and the wave movement, or by giving a vibration.
Such glasses could definitely be beneficial, especially for those with hearing loss who do not have access to hearing aids. But now, let’s move onto the neat world of smart glasses that convert speech to text.
3 clever solutions of smart glasses for speech recognition
1) Speech to Text Glasses
In a study, researchers at Nagoya University in Japan tested out smart glasses, which used automatic speech recognition (ASR) to translate spoken speech into text and displayed this text inside the glasses.
They paid special attention to the glasses’ ability to properly recognize speech, as it is critical for the user to receive text that correctly corresponds with the spoken speech.
They found that their smart glasses produced similarly positive results to an existing smartphone application that also uses ASR. However, their smart glasses were lacking a little in terms of ASR accuracy and the processing speed. These aspects would have to be improved for the glasses to be completely effective for the deaf and hard of hearing.
2) Smart Caption on Google Glass
Another app, developed at Georgia Tech in Atlanta, USA, works on a similar principle. However, instead of using the mics in the smart glasses, a smartphone is spoken into. The phone then runs voice recognition algorithms, after which the text is displayed in the deaf or hard of hearing user’s Google Glass.
If there are ambiguities in terms of the speech, the app provides different possibilities for the text and the Glass wearer can quickly select the correct option to help define the context of the conversation. The size of the text can also be adjusted.
3) Cheap “Transcribe” Smart Glasses
A young man from India called Madhav Lavakare has taken cost into account and is set on a mission to develop low-cost smart glasses for the deaf and hard of hearing.
He was inspired by the possibilities that Google Glass provides, but found Glass to be a bit too expensive. He wanted to find a way to develop smart glasses that could help people with hearing loss, but ones that would be more affordable. He did this by using cheap electronic parts and named his product ‘Transcribe’.
Madhav built the glasses for Rs 2500 (~36$) and is selling them for Rs 3500 (~45$). While the look of the glasses is nowhere near discrete, as they are rather bulky, the incredibly affordable price makes up for it.
In India, 60 million people have hearing loss and many of them cannot afford a Google Glass. Therefore, the Transcribe could help many people with hearing loss better interact with society at a low cost. Madhav is currently working on bettering and perfecting his prototypes and is looking for potential investors and programmers to work with.
Places where smart glasses can improve accessibility
Smart glasses have the potential to help those with hearing loss better understand others’ speech in everyday situations. However, there are some specific places and events where measures need to be taken so the deaf and hard of hearing could have as enjoyable of an experience as every other person there.
Many of these examples come from the entertainment industry. I will give an overview of 4 areas that can improve the accessibility of their services by using smart glasses.
Smart Captioning Glasses for Theatre
The Royal National Theatre took a big leap toward inclusivity when they enabled smart glasses for all their deaf and hard of hearing audience members. The smart glasses display captions of what the actors are saying, so the theatre-goer with hearing loss can comfortably follow the show.
While there already were captions shown on screens on either side of the stage, displaying them inside the smart glasses means that the glasses wearer doesn’t need to avert his or her eyes between two visual fields.
The glasses were developed to be as comfortable as possible – though they are slightly heavier than regular reading glasses, they aren’t big or bulky at all. After a few minutes, you forget that you’re even wearing them.
There is a light keypad, which allows the person to alter the way the text is displayed – for example, it is possible to change the size and color of the text, as well as the position on the screen.
The headset was manufactured by Epson in partnership with the theatre. The glasses took two years to develop. They could have opted to offer the scripts on a phone, but according to Jonathan Suffolk, the theater’s technical director, they “wanted a technology that was much more discreet and immersive and wouldn’t disturb anyone”.
A struggle in developing the idea was getting the text to completely correspond with the actors’ speech. They couldn’t just load the entire script into the system and expect it to run smoothly, as problems would arise when actors speak quicker or slower than expected.
The software actually follows live speech and can recognize certain stage directions, such as lighting changes, so that the subtitles will appear in the right place.
The smart glasses are completely free of charge for the theatre-goer. Such glasses are available at every show at the National Theatre. I say, kudos to the National Theatre for making their performances equally enjoyable for everyone.
Planetariums
Other places are following the National Theatre’s example. The Cite des Sciences et de L’industrie in Paris had the idea of using smart glasses to improve the experiences of deaf or hard of hearing visitors in their planetarium.
The glasses would be very beneficial in the planetarium, as they enable the deaf or hard of hearing to look at the 360° projection onto the room’s vault, while simultaneously seeing the translation into sign language or subtitles. Without smart glasses, the person would have to avert their attention from the show to a sign language interpreter or captions.
Although the planetarium was planning to have the glasses available to the general public by the second half of 2018, at the moment there is unfortunately no information about the use of smart glasses in the planetarium on their website. Hopefully, they’ll revisit the idea in the future, as it would considerably improve accessibility.
Church
There has also been talk of including smart glasses for the hard of hearing in churches. It would be especially useful for churches that don’t have access to a sign language interpreter.
The difficulty lies, again, in making the captions accurate to the speech. However, as we’ve learned from previous examples, there are technologies that make this possible. There is definitely potential in using smart glasses to make church accessible to everyone.
Lectures
Researchers also see potential in using smart glasses to make lectures more convenient for the deaf and hard of hearing. The smart glasses would be beneficial as they could reduce the effects of visual field switches by having a single display in front of the students’ eyes.
Usually, the students would have to switch their attention between the captions/sign language interpreter and the lecturer/slides. The researchers in this study displayed ASL (American Sign Language) interpreters in the glasses.
The researchers found that even though the students’ comprehension of the lecture topic didn’t improve with the smart glasses, the glasses did help make the lecture a lot more comfortable to listen to. With the smart glasses on, the students no longer had to shift their attention between two viewpoints. They also reported being able to take notes more easily.
The researchers noted that the comfortable fit and look of the eyewear are very important. All in all, the potential to use smart glasses to make lectures easier to follow for the deaf and hard of hearing is definitely there.
Conclusion
As we can see, there is a lot of potential in using smart glasses to improve the experiences of the deaf and hard of hearing. Whether that be in terms of alarm glasses, understanding speech in a conversation setting or making shows and lectures more comfortable to follow. There are already excellent solutions out there, but there is also great potential for the future.
