Gesture Based Two-Way Communication System  for Deaf and Non-Verbal Users Using Machine Learning and Python

Erusu Kata Raju Reddy¹, Venne Narayana Rao²

¹Assistant Professor, ² MCA Final Semester,

Master of Computer Applications, Sanketika Vidya Parishad Engineering College, Vishakhapatnam,

Andhra Pradesh, India

rajureddy800746@gmail.com, narayanaraovenne39@gmail.com

ABSTRACT:

Communication between deaf, non-verbal, and hearing individuals is often difficult because they use different methods of communication. Deaf users mainly depend on sign language and hand gestures, which are not commonly understood by everyone. To address this issue, the proposed Gesture-Based Two-Way Communication System enables seamless interaction through real-time gesture and speech translation. The system uses a webcam to capture hand gestures and detects hand landmarks with the help of MediaPipe technology. These landmarks are processed and classified using the K-Nearest Neighbors (KNN) machine learning algorithm to identify predefined gestures accurately. Once recognized, the gestures are converted into readable text and audible speech, making communication easier for hearing individuals. The system also supports communication in the opposite direction by capturing spoken input through a microphone and converting it into text using speech recognition techniques. The corresponding gesture image is then displayed for deaf users to understand the message. This two-way communication approach improves accessibility, reduces communication barriers, and provides an efficient assistive solution. The system is simple, user-friendly, cost-effective, and helps create better interaction between deaf and hearing communities.