Abstract
Sign languages are essential to bridge the communication gap between regular people and deaf people. However, several factors, including varied perspectives, sizes, and lighting, as well as the difficulty of capturing facial emotions concurrently, make real-time interpretation of these languages very difficult. Due to these issues, it is challenging to communicate clearly and smoothly in the real world. To overcome problems like these, we propose a revolutionary Real-time Sign Language [1] and Emotion Detection (RTSLED) system that combines MediaPipe’s hand-tracking capabilities with Convolutional Neural Networks (CNN) to identify Indian Sign Language (ISL) gestures and emotions from live video input. Based on a dataset of around 43,000 ISL images, our approach uses two CNN models to efficiently categorize signs and seven emotional expressions after Preprocessing hand landmarks and face signals. A text-to-speech feature that uses Python Script 3 and a threading system to ensure smooth operation is used to increase accessibility. Our approach offers reliable real-time performance without any specialized equipment. Our main goal is to promote inclusive communication and empower the hearing-impaired community.