Real-Time ASL Interpreter

Computer vision pipeline bridging the communication gap.

Overview

I engineered a Real-Time ASL Interpreter focused on translating static alphabet letters accurately using a live camera feed. This platform provides continuous tracking of gestures and converts recognized signs into synthesized text-to-speech output in real-time.

Pipeline & Technologies

Instead of just sending entire images into a bulky Convolutional Neural Network, the pipeline utilizes Google's MediaPipe framework. MediaPipe extracts precise structural geometry (21 3D landmarks) from the hand skeleton.

These coordinate vectors are serialized and fed into a custom scikit-learn classifier pipeline. This significantly increases frame rates and inference speeds since the mathematical payload is drastically reduced compared to pixel-inference arrays.

• Real-Time Tracking: Leverages OpenCV to capture streaming data and layer visual UI elements denoting the skeletal tracking grid.
• High Accuracy Classifier: Achieved a sustained 95% classification accuracy across the dataset matrix on gestures captured from variable lighting conditions.
• Text-to-Speech Engine: Actively converts new and distinct sequential predictions into audible speech automatically, mimicking natural spoken word translation.
• UX Optimization: Integrated a "None" class and a mute toggle to prevent audio spamming when the user's gestures are transient or unclear between distinct letters.