AI-Powered Gesture Recognition for Sign Language Translation
AI-Powered Gesture Recognition for Sign Language Translation
Authors:
DR. AJAY M. POL*1, VAIBHAV PANDHARINATH GADHE*2, RITESH VINAYAK CHAVAN*3, MUKESH NATHARAM CHOUDHARY*4, ABHISHEK PRAVIN JAIN*5
*1 Assistant Professor, Department of Electronics & Telecommunication Engineering, KIT’S College of Engineering, Kolhapur (Empowered Autonomous), Maharashtra, India
*2,3,4,5 B.Tech Students, Department of Electronics & Telecommunication Engineering, KIT’S College of Engineering, Kolhapur (Empowered Autonomous), Maharashtra, India
Abstract - Sign language recognition is a key to better communication between the deaf and non-deaf. This study presents a light weighted and real time Sign Language Recognition (SLR) system which includes hand joint detection along with deep learning for which we use the former to identify the signs. We have put together a system which recognizes over 40 hand gestures which include numbers (0-9), letters (A-Z), also we included common signs like Hii, yes, Help, and Space. Also, we present a set of over 22,000 annotated hand gesture samples which we collected from web and mobile cameras in various lighting and environment settings.
The system uses Media Pipe Hands to determine 21 key points of the hand which in turn create a 63 dimensional feature vector that which shows the location of the finger joints and palm. We have put in a method called centroid alignment which helps the model do well with new data and also reduce positional differences. We use a one dimensional Convolutional Neural Network (1D CNN) for gesture classification which we did for its speed and low computer resource use. For real time accuracy the system includes what we do in temporal stability filtering and also we use confidence thresholds to reduce errors and also to prevent repeated gesture outputs.
A display which features dynamic output with automatic text wrap and a blurred background setting helps to present recognized gestures better. We also have an SQLite database which logs each gesture session along with the time of recording, so it may be tracked and studied at a later date. We report that the system does very well in terms of accuracy, response time, and performance in real time which in turn does not require expensive hardware. This makes it a great asset for communication tools, in education, and for applications which support people with disabilities.
Keywords: Sign Language Recognition, Computer Vision, Convolutional Neural Network (CNN), MediaPipe Hands, Real-Time Gesture Detection, Assistive Technology, Hand Gesture Recognition.