Eye-hand coordination: factors influencing gaze patterns during fingerspelling perception in American Sign Language
OA Version
Citation
Abstract
Sign languages such as American Sign Language (ASL) unfold dynamically over time, requiring perceivers to make real-time decisions about where to direct their gaze to capture linguistic information as it occurs and before it disappears. While skilled deaf signers look mostly at the face during sign perception, gaze is sometimes directed to the hands when fingerspelling occurs. I conducted three experiments to explore how different features of the input (familiarity, predictability) and cues from the sign producer (mouthing and indexing to the fingerspelling hand) impact looks to the hands during fingerspelling perception. In Study 1, I compare the role of sign type (lexical signs vs. fingerspelled words) and familiarity (novel vs. familiar) on deaf signers’ gaze patterns while perceiving single sign video clips, finding that sign perceivers look more at the hands when viewing fingerspelled words and novel items. In Study 2, I explore how cues from the sign producer (mouthing and indexing, when the sign producer looks at and points to their own hand during fingerspelling) impact gaze patterns, finding that, while mouthing results in more looks to the face, indexing does not increase looks to the hands. In Study 3, I explore the role of prediction on gaze patterns within a sentence context, finding that signers make anticipatory looks when they expect fingerspelling to occur but are unable to predict the specific word that will be produced, such as during the introduction of a name. This dissertation provides evidence that sign perceivers flexibly adapt their gaze based on both the perceptual and linguistic demands of the input. Pairing eye-tracking methodology with the visual modality of ASL offers insight into language processing and prediction, showcasing how language users dynamically attend to certain features of the linguistic input in real-time.
Description
2026
License
Attribution 4.0 International