Huawei recently announced a new mobile app that uses artificial intelligence (AI) to translate children books into sign language to help deaf kids read at a younger age.
Through an app called StorySign, the technology is capable of reading texts in children’s book via a phone camera and convert it into sing language, thanks to an onscreen character. The app, in partnership with the European Union of the Deaf and the British Deaf Association, was born as Huawei believes in using the power of technology to good use and to push the boundaries of what is humanly possible.
There are about 32 million deaf children globally and many struggle to learn to read because they can’t hear their parents read them a bedtime story or a teacher repeating sentences – all key milestones in helping a child learn at the crucial early learning stages.
“StorySign is a world’s first – nothing like this exists yet, however its concept is remarkably simple, but the tech that makes it work is very complex…it has the ability to change the life of deaf children,” said Huawei UK Global Senior Product Marketing Manager, Peter Gauden, to the Mirror UK.
In terms of how the app works, it is very simple. Once StorySign is downloaded, the parent of the child can open the app and choose a book from the app’s library. All she or he has to do is simply point the smartphone’s camera at the first page of the book and it will highlight each word of the story. Then, a friendly avatar called Star will read and understand the words, and translates them into sign language. This allows parents and children to learn to sign and read together, at their own pace.
However, it’s worth mentioning that the app doesn’t work with all kid’s book. The books that StorySign works will have to be pre-programmed with the app first, but the app library is expanding and will become more extensive in the near future, Huawei said in an article by the Mirror UK.
Currently, StorySign is only available in the UK and Europe with plans to expand to Asia next year. The app is available for free on Android via the Google Play Store and the Huawei AppGalley in 10 languages.
