Exploiting phonological constraints for handshape inference in ASL video
Date
2011-05-15
DOI
Authors
Thangali, Ashwin
Nash, Joan
Sclaroff, Stan
Neidle, Carol
Version
OA Version
Citation
Thangali, Ashwin; Nash, Joan; Sclaroff, Stan; Neidle, Carol. "Exploiting Phonological Constraints for Handshape Inference in ASL Video", Technical Report BUCS-TR-2011-013, Computer Science Department, Boston University, May 15, 2011. [Available from: http://hdl.handle.net/2144/11370]
Abstract
Handshape is a key linguistic component of signs, and thus, handshape recognition is essential to algorithms for sign language recognition and retrieval. In this work, linguistic constraints on the relationship between start and end handshapes are leveraged to improve handshape recognition accuracy. A Bayesian network formulation is proposed for learning and exploiting these constraints, while taking into consideration inter-signer variations in the production of particular handshapes. A Variational Bayes formulation is employed for supervised learning of the model parameters. A non-rigid image alignment algorithm, which yields improved robustness to variability in handshape appearance, is proposed for computing image observation likelihoods in the model. The resulting handshape inference algorithm is evaluated using a dataset of 1500 lexical signs in American Sign Language (ASL), where each lexical sign is produced by three native ASL signers.