Graduate Student Seminar Series – Maryam Dehghani

Graduate Student Seminar Series
Please ensure you invite your Principal Investigator by adding their email via the ‘Add Guest’ button and they will also be notified of your presentation.
Location: MS2158 – 1 King’s College Circle
Presentation Title: Advancing deep learning for sign language to spoken language translation
Abstract:
Communication barriers between Deaf and hearing individuals continue to limit access to education, healthcare, and everyday social interactions. This research aims to reduce those barriers by developing an AI-driven framework for real-time, two-way sign language translation. The long-term goal is to combine Sign Language Recognition (SLR) and Sign Language Production (SLP) into a single system that enables more natural and accessible communication between Deaf and hearing communities.
The current stage of this research focuses on the SLR component using the American Sign Language Lexicon Video Dataset (ASLLVD). A complete preprocessing pipeline was developed to clean and organize the dataset, extract body and hand landmarks using MediaPipe, and convert sign language videos into machine-learning-ready representations. Three deep learning architectures, LSTM-only, CNN+LSTM, and CNN+Transformer, were then implemented and compared to identify the most effective approach for recognizing sign language from video.
Among the evaluated models, the CNN+Transformer model achieved the strongest performance, demonstrating a greater ability to capture both the spatial and temporal patterns that characterize sign language. These findings provide a strong foundation for the next phase of the project, which will extend the model to generate natural, expressive sign language from spoken or written language.
This work demonstrates how artificial intelligence can make communication more accessible across different languages and modalities. By integrating computer vision, deep learning, and sign language technologies, the proposed framework supports more natural interactions between Deaf and hearing individuals, helping improve access to essential services while promoting inclusion, accessibility, and equitable participation in society.
Supervisor Name: Dr. Tom Chau
Year of Study: 3
Program of Study: PhD
Powered by Calendly.com