The Human Speechome Project
Deb Roy of the MIT Media Lab
The Human Speechome Project is an effort to observe and computationally model the longitudinal course of language development of one child at an unprecedented scale. We are collecting audio and video recordings for the first two to three years of one child’s life, in its near entirety, as it unfolds in the child’s home. To analyze the resulting massive audio-visual corpus, we are developing new data mining technologies to help human analysts rapidly annotate and transcribe recordings using semi-automatic methods, and to detect and visualize salient patterns of behavior and interaction. To make sense of large-scale patterns that span across months or even years of observations, we will develop computational models of language acquisition that are able to learn from the child’s experiential record. I will describe the methodology for ultra-dense in vivo data collection (including privacy management), preliminary data analysis tools, and sketch our plans for expanding the effort beyond N=1 for the purpose of understanding and treating developmental disorders such as autism.
About Deb
Deb Roy directs the Media Lab’s Cognitive Machines group, and is Chair of the Academic Program in Media Arts and Sciences. His research focuses on the interaction of language with physical and social context, which he explores through the construction of robots, video games, and the study of child language acquisition. Roy has published over 60 peer-reviewed papers in the areas of Artificial Intelligence, cognitive modeling, human-machine interaction, data mining and visualization, and machine learning. He has served as guest editor for the journal Artificial Intelligence and is an Associate of Behavioral and Brain Sciences. His work has been featured in mainstream media including NPR, BBC News, Wired Magazine, Science Magazine, New Scientist, Technology Review, PC Magazine, and Popular Science. Roy holds a Bachelor of Applied Science in computer engineering from the University of Waterloo, Canada, and both an MS and PhD in media arts and sciences from MIT.