The need to constantly improve conditions and increase opportunities for the Deaf* has been on the table for a long time. SignAll Technologies, a Hungarian-American startup, took notice of the need and developed its own software and tool to translate sign language. The product, called SignAll Lab, is a workstation where ASL is captured by the device and communicated into written American English, which then appears in an app.
SignAll: an inclusive device
SignAll has been actively addressing the issue of increasing the use of sign language. The d/Deaf communicate predominantly in written form when they work alongside those who hear. Overall, they have very limited options to engage in oral conversations with others. SignAll aims to bring more inclusiveness so that both the hearing can understand the d/Deaf and vice versa, as CEO Zsolt Robotka confessed in a personal article last year.
Although founded in Hungary, SignAll operates predominantly in the United States, where it found support among institutions for the d/Deaf. Indeed, the only language that it currently translates, American Sign Language (ASL), is used predominantly in North America but is also found in other parts of the world.
The key facility to help SignAll was Gallaudet University in Washington, D.C., which is the only university for the d/Deaf in the world. The very first user was Derek Frank, a staff member of the institution, who was able to communicate with those who did not know ASL immediately. Frank acknowledged and appreciated the inclusiveness this tool had managed to bring in an article released shortly after the products’ completion.
Currently, SignAll operates two offices and has a revenue of over EUR 10 million. It emerged as a spinoff of the investment company Dophio Technologies when both founders and CEOs, Zsolt Robotka and János Rovnyai, observed that their parent enterprises had problems communicating with and hiring d/Deaf employees. The SignAll team, which included both hearing and d/Deaf members, worked for seven years on the product, finally launching in 2018.
How does the product work?
The automated sign language device consists of two screens and a device that holds cameras. There are four cameras in total. Three depth cameras capture body and hand movements and facial expressions. The other one detects color. For this reason, users wear colored gloves during conversations. Images are subsequently merged. The data is processed and compared to the software database, where its similarity is evaluated.
The software, for example, recognizes and distinguishes between nouns, verbs, and other language formations. The AI and machine learning then narrows down the possible meaning of the captured movements and colors to three options from which the most likely one is displayed. Other accessories include side lights and a table.
The device is installed by the SignAll team and is offered for education and business purposes, as well as to individual users. The difference is not in the product itself but in the software license and pricing. Even though the invention significantly reduces the barrier between the hearing and d/Deaf, the price is rather high, ranging from EUR 1300 to over 5000, which limits some potential users.
Future challenges and potential
Nonetheless, the device is constantly improving, and the app with translations can already be accessed on Android. The company hopes to see the app work on Siri and Alexa in the future and is enthusiastic about translating other sign languages into English. Additionally, the company hopes to make ASL more ‘broadly known and interpretation universally available’ to revolutionize the way people communicate with people and machines, Robotka confirmed.
(*Note: The uppercase ‘Deaf’ is used when referring to a particular group of deaf people who share a language – American Sign Language (ASL) – and culture, as opposed to the condition itself, which uses the lowercase ‘d’.)
