The idea for this project came from a Kaggle competition. The goal for the competition was to help the deaf and hard-of-hearing better communicate using computer vision applications. The National Institute on Deafness and other Communications Disorders (NIDCD) indicates that the 200-year-old American Sign Language is a complete, complex language (of which letter gestures are only part) but is the primary language for many deaf North Americans. ASL is the leading minority language in the U.S. after the "big four": Spanish, Italian, German, and French. One could implement computer vision and some Text-to-Speech to enabling improved and automated translation applications. The original project was created over a year ago using Android App as the edge device, this is the updated version of it using the Vision AI Dev Kit camera as the edge device.
Implementation
The project uses the Custom Vision service to create and export the model and integrates it with the Vision AI DevKit.
Software and Services used
Hardware
Custom Vision
Azure IoT Hub
Azure Storage Account
VLC Media Player
Vision AI DevKit camera
Repository
Find all relevant information, including code, pictures used for model training, and the Custom Vision model file for full implementation of this product here.
Users are always encouraged to innovate and continue to improve the functionality of current projects.
Future Improvements and Project Suggestions
One potential additional function could be to enable audio output using the built-in speakers of the camera.
Feel free to fork the project and contribute back any improvements or suggestions. Contributors and maintainers are encouraged.
About the Creator
Jomit Vaghela is a Technology Strategist, Architect and Developer working on fulfilling the promises of Digital Transformation using Machine Learning, IoT, Blockchain and Cloud.
You can learn more about what Jomit is working on here.
Leave a comment