Audio-visual speech perception and word-learning in monolingual and bilingual infants

As speech signal is highly multimodal, the thesis aims to explore to what extent seeing and hearing spoken words influence lexical learning in infancy and beyond. This will be done by investigating the use of auditory and visible speech information during word-learning in 14-to-36-month-old infants with behavioural and neurophysiological methods. The use of visible speech information during word-learning is susceptible to developmental changes. Previous work indicates an auditory dominance in 18-month-old infants that is not found anymore in adulthood (Havy et al., 2012, submitted). The purpose of the current research is to specify when the shift in sensory dominancy occurs and how this relates to developmental changes in lexical capacities.

It examines whether infants can learn new words in either auditory (acoustic form of the word with no accompanying face) or visual (talking facewith no sound) modality and whether lexical representations have a format that is sensori-specified or multimodal at different ages (study 1). This project also questions the role of language experience on the development of these mechanisms, focusing on bilingualism (study 2) and foreign language acquisition (study 3). Word form processing will be examined at a broad level but also at a fine-grained phonetic level, to determine the detail with which visible speech information is encoded in lexical representations (study 4). Another part of this research will also investigate neurophysiological correlates of audiovisual integration of speech processing (study 5) in infancy. This would provide temporal information concerning multisensory integration in infancy.

In essence, this research intends to illuminate multisensory mechanisms by which infants learn new words, bringing together theoretical and empirical advances from different disciplines, including linguistics, psychology and neuroscience.