Natural human verbal communication

Description

This project proposes experimental approaches to investigate the neural bases of natural human verbal communication with fMRI. It is a unique occasion to bring together the two main aspects of his past research, namely the investigation of social cognition using artificial agents in control conditions (e.g. Chaminade & al., 2012, 2015, 2016) and investigating the evolutionary origins of language (e.g. Stout et al., 2014,2015). The research program will investigate the behavioural and neural correlates associated with having a live discussion with a real or an artificial partner.
This project is currently being developed thanks to collaboration with linguists (Laboratoire Parole et Language) and computer scientists (Laboratoire des Sciences de l’Information et des Systèmes). The core of this project is the development of an experimental paradigm derived from social cognitive neuroscience in which a participant is recorded in trials in which he discusses in the exact same set-up with another human or with an Embodied Conversational Agent (ECA), allowing the direct comparisons of responses to the natural and the artificial agent. This procedure is adapted to acquire an fMRI corpus, for which the participants are scanned during the discussions with human and artificial interlocutor.
Behavioral features, both verbal and non-verbal are automatically extracted from audio, video and gaze tracking. Analysis of physiological uses a deconvolution of skin conductance and of response in localized regions of the brain involved in different mechanisms of social cognition (such as motor resonance, social motivation or mentalization) to identify the timing of unidimensional (skin conductance) or multidimensional (fMRI) cognitive events during the conversations with human or ECA interlocutor. Comparing responses to the two interlocutors in these corpora will provide psychophysiological assessment of social competence. A number of questions pertinent for “second-person social neuroscience”, which puts forward the importance of studying real-time social cognition in truly interactive paradigms, but also for the development of artificial social agents, will be addressed once the paradigm is firmly established.