CAS: Linguistics: Scholarly Papers
Permanent URI for this collection
Browse
Recent Submissions
Item Intoxication and pitch control in tonal and non-tonal language speakers(Acoustical Society of America (ASA), 2022-06) Tang, Kevin; Chang, Charles B.; Green, Sam; Bao, Kai Xin; Hindley, Michael; Kim, Young Shin; Nevins, AndrewAlcohol intoxication is known to affect pitch variability in non-tonal languages. In this study, intoxication's effects on pitch were examined in tonal and non-tonal language speakers, in both their native language (L1; German, Korean, Mandarin) and nonnative language (L2; English). Intoxication significantly increased pitch variability in the German group (in L1 and L2), but not in the Korean or Mandarin groups (in L1 or L2), although there were individual differences. These results support the view that pitch control is related to the functional load of pitch and is an aspect of speech production that can be advantageously transferred across languages, overriding the expected effects of alcohol.Item Bidirectional skeleton-based isolated sign recognition using graph convolution networks(2022-06-20) Dafnis, Konstantinos M.; Chroni, Evgenia; Neidle, Carol; Metaxas, DimitrisItem Isolated sign recognition using ASL datasets with consistent text-based gloss labeling and curriculum learning(2022-06-04) Dafnis, Konstantinos M.; Chroni, Evgenia; Neidle, Carol; Metaxas, DimitrisWe present a new approach for isolated sign recognition, which combines a spatial-temporal Graph Convolution Network (GCN) architecture for modeling human skeleton keypoints with late fusion of both the forward and backward video streams, and we explore the use of curriculum learning. We employ a type of curriculum learning that dynamically estimates, during training, the order of difficulty of each input video for sign recognition; this involves learning a new family of data parameters that are dynamically updated during training. The research makes use of a large combined video dataset for American Sign Language (ASL), including data from both the American Sign Language Lexicon Video Dataset (ASLLVD) and the Word-Level American Sign Language (WLASL) dataset, with modified gloss labeling of the latter—to ensure 1-1 correspondence between gloss labels and distinct sign productions, as well as consistency in gloss labeling across the two datasets. This is the first time that these two datasets have been used in combination for isolated sign recognition research. We also compare the sign recognition performance on several different subsets of the combined dataset, varying in, e.g., the minimum number of samples per sign (and therefore also in the total number of sign classes and video examples).Item Sign language video anonymization(2022-06-25) Xia, Zhaoyang; Chen, Yuxiao; Zhangli, Qilong; Huenerfauth, Matt; Neidle, Carol; Metaxas, DimitrisItem Resources for computer-based sign recognition from video, and the criticality of consistency of gloss labeling across multiple large ASL video Corpora(2022-06-25) Neidle, Carol; Opoku, Augustine; Ballard, Carey; Dafnis, Konstantinos M.; Chroni, Evgenia; Metaxas, DimitrisItem Discovery of informative unlabeled data for improved learning(2005) He, Weijun; Huang, Xiaolei; Tsechpenakis, Gabriel; Metaxas, Dimitris; Neidle, CarolItem Benchmark databases for video-based automatic sign language recognition(EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA, 2008) Dreuw, P.; Neidle, Carol; Athitsos, V.; Sclaroff, Stanley; Ney, H.Item SignStream™: a database tool for research on visual-gestural language(American Sign Language Linguistic Research Project, Boston University, 2000-08) Neidle, CarolItem SignStream™ annotation: conventions used for the American Sign Language Linguistic Research Project(American Sign Language Linguistic Research Project, Boston University, 2002-08) Neidle, CarolItem SignStream annotation: addendum to conventions used for the American Sign Language Linguistic Research Project(American Sign Language Linguistic Research Project, Boston University, 2007-08) Neidle, CarolItem A User's guide to SignStream® 3(American Sign Language Linguistic Research Project, Boston University (111 pages), 2017-08) Neidle, CarolItem What's new in SignStream® 3.3 ?(American Sign Language Linguistic Research Project, Boston University, 2020-08-01) Neidle, CarolItem Update on linguistically annotated ASL video data available through the American Sign Language Linguistic Research Project (ASLLRP)(American Sign Language Linguistic Research Project, 2021-06) Neidle, Carol; Opoku, AugustineThe American Sign Language Linguistic Research Project (ASLLRP) provides Internet access to high-quality ASL video data, generally including front and side views and a close-up of the face. The manual and non-manual components of the signing have been linguistically annotated using SignStream®. The recently expanded video corpora can be browsed and searched through the Data Access Interface (DAI 2) we have designed; it is possible to carry out complex searches. The data from our corpora can also be downloaded; annotations are available in an XML export format. We have also developed the ASLLRP Sign Bank, which contains almost 6,000 sign entries with distinct English-based glosses, with a total of 41,830 examples (in addition to about 300 gestures, over 1,000 fingerspelled signs, and 475 classifier examples). The Sign Bank is likewise accessible and searchable on the Internet; it can also be accessed from within SignStream® to make annotations more accurate and efficient. Here we describe the available resources. These data have been used for many types of research into computerbased sign language recognition from video.Item A user's guide to the American Sign Language Linguistic Research Project (ASLLRP) data access interface (DAI) 2 — Version 2(American Sign Language Linguistic Research Project, Boston University, 2020-05) Neidle, CarolItem What's new in SignStream® 3.4.0 ?(Boston University American Sign Language Linguistic Research Project, 2022-05) Neidle, CarolItem Documentation for download of ASLLRP sign bank citation-form sign datasets(Boston University American Sign Language Linguistic Research Project, 2022-03) Neidle, Carol; Opoku, AugustineItem Why alternative gloss labels will increase the value of the WLASL dataset(Boston University American Sign Language Linguistic Research Project, 2022-03) Neidle, Carol; Ballard, CareyItem Understanding ASL learners’ preferences for a sign language recording and automatic feedback system to support self-study(ACM, 2022-10-22) Hassan, Saad; Lee, Sooyeon; Metaxas, Dimitris; Neidle, Carol; Huenerfauth, MattAdvancements in AI will soon enable tools for providing automatic feedback to American Sign Language (ASL) learners on some aspects of their signing, but there is a need to understand their preferences for submitting videos and receiving feedback. Ten participants in our study were asked to record a few sentences in ASL using software we designed, and we provided manually curated feedback on one sentence in a manner that simulates the output of a future automatic feedback system. Participants responded to interview questions and a questionnaire eliciting their impressions of the prototype. Our initial findings provide guidance to future designers of automatic feedback systems for ASL learners.Item Bidirectional skeleton-based isolated sign recognition using graph convolution networks(European Language Resources Association (ELRA), 2022-06-25) Dafnis, Konstantinos M.; Chroni, Evgenia; Neidle, Carol; Metaxas, Dimitris N.To improve computer-based recognition from video of isolated signs from American Sign Language (ASL), we propose a new skeleton-based method that involves explicit detection of the start and end frames of signs, trained on the ASLLVD dataset; it uses linguistically relevant parameters based on the skeleton input. Our method employs a bidirectional learning approach within a Graph Convolutional Network (GCN) framework. We apply this method to the WLASL dataset, but with corrections to the gloss labeling to ensure consistency in the labels assigned to different signs; it is important to have a 1-1 correspondence between signs and text-based gloss labels. We achieve a success rate of 77.43% for top-1 and 94.54% for top-5 using this modified WLASL dataset. Our method, which does not require multi-modal data input, outperforms other state-of-the-art approaches on the same modified WLASL dataset, demonstrating the importance of both attention to the start and end frames of signs and the use of bidirectional data streams in the GCNs for isolated sign recognition.Item Isolated sign recognition using ASL datasets with consistent text-based gloss labeling and curriculum learning(European Language Resources Association (ELRA), 2022-06-24) Dafnis, Konstantinos M.; Chroni, Evgenia; Neidle, Carol; Metaxas, Dimitris N.We present a new approach for isolated sign recognition, which combines a spatial-temporal Graph Convolution Network (GCN) architecture for modeling human skeleton keypoints with late fusion of both the forward and backward video streams, and we explore the use of curriculum learning. We employ a type of curriculum learning that dynamically estimates, during training, the order of difficulty of each input video for sign recognition; this involves learning a new family of data parameters that are dynamically updated during training. The research makes use of a large combined video dataset for American Sign Language (ASL), including data from both the American Sign Language Lexicon Video Dataset (ASLLVD) and the Word-Level American Sign Language (WLASL) dataset, with modified gloss labeling of the latter—to ensure 1-1 correspondence between gloss labels and distinct sign productions, as well as consistency in gloss labeling across the two datasets. This is the first time that these two datasets have been used in combination for isolated sign recognition research. We also compare the sign recognition performance on several different subsets of the combined dataset, varying in, e.g., the minimum number of samples per sign (and therefore also in the total number of sign classes and video examples).