Project Title: Express
Introduction:
AAC or Augmentative and Alternative Communication can be defined as a non-speech form of communication. This form of communication usually uses images and/or speech generating mechanism. The need for AAC is due to various reasons. Some frequent AAC users are people who are treated with the following –
- Autism – About 1% of world’s population has autism spectrum disorder. In India, about 13 million people suffer from Autism spectrum Disorder and are socially isolated because they can’t communicate.
- Parkinson’s disease – About 1 million people are affected by it alone in the U.S.
- Cerebral Palsy – About 17 million people are affected by it globally.
- Dementia – Around 50 million people have dementia worldwide with around 10 million new cases every year.
- Traumatic Brain Injury – Results in loss of speech or not able to form words or loss of memory.
- People who can’t speak (mute) – It effects around 40 million people globally.
Speech is the most important form of communication. In this age of connectivity, people with disability should not be left behind irrespective of their age, gender, etc. The best form of communication for them is via pictures or images or a mix of both.
Conceptual framework:
Express is a multilingual application which uses text to speech and using images to make statements and/or ask questions so that its users can interact with each other as well as their surroundings. This app is mostly aimed at differently abled people who have lost their speech or can’t communicate properly or lie in the autism spectrum. Keeping that in mind, we kept the interface as simple and understandable as possible.
It also has multiple speech rates and pitch so that user can decide themselves that how slow or fast they want to hear the words. The questions and/or statements makes it even easier to interact. , For example,: With the image of water, there are statements like I am thirsty which is completely relevant and important yet simple to understand.
Being multilingual, it has huge potential and can impact many lives irrelevant of the borders. Since it is made using react-native, it can be compiled to run as an is app or an android app thus covering most of the latest operating systems available on mobile devices.
Diagrams:
Flow Diagram
In App Flow Diagram
Relevant questions and statements along with text to speech
Day to day conversations
Places
Tools Used:
- React js
- React-native
- Expo.io
- Google translate
- Watchman
- Node.js
- Genymotion
- Amazon cloudfront
Application:
It can be used to decrease the gap between differently abled people and the community. It will help them in communicating easily and express their feelings, ideas, and thoughts. It can be easily used by children as well as old people. It is multilingual so that people can express themselves language they know, and other languages will also be added soon.
Also, children who lie on the autism spectrum can be taught using this application on what our feelings and how to express them like anger, sadness, happiness, etc. They can also ask questions or make statements using this app which will expand their method of communication.
People who are learning sign language can use it to communicate in the early stages of learning and can also communicate with people who don’t know sign language. The images used in this app are easily understood in our society and will help in smoother interactions.
Link to GitHub repository: https://github.com/sayak119/Express
Link to deployed app: https://expo.io/@sayak119/express
Link to youtube video: https://youtu.be/lnoBehuz3sk
Research work done:
- https://www.asha.org/public/speech/disorders/AAC/
- https://www.aacandautism.com/why-aac
- https://www.nidcd.nih.gov/health/autism-spectrum-disorder-communication-problems-children
Team Members:
- Sayak Kundu
- Kushagra Nagori
- Akshat Maheshwari
- Kalpit Pokra
Note: This project idea is contributed for ProGeek Cup 2.0- A project competition by GeeksforGeeks.