Title
Dissociating facial appearance and dynamics during videoconference conversations
Speaker
Steven Boker, University of Virginia
Abstract
When we speak to one another, we adapt and coordinate our facial expressions in response to our inner states and in response to our perceptions of the person with whom we are speaking. We say we are speaking "with" someone, since this is a shared experience; one that creates an interpersonal coupled system with feedback between the dynamics of the two speakers' perceptions and actions. There is evidence that structural appearance of faces as represented in static photographs and the dynamics of faces may involve separate visual processing.
Dissociating the contribution of facial appearance and facial dynamics to the adaptive coupled system involved in conversation requires being able to assign appearance to dynamics in real time. This talk presents a method for assignment of appearance charateristics such as gender or race while preserving facial dynamics in a way that is sufficiently convincing that naive videoconference participants do not realize that they are speaking with a computer generated model face. This means that we can randomly assign characteristics such as gender and race in a naturalistic conversational setting.
The method can reconstruct near-photorealistic video from between 8 principal components to account for the appearance and 8 for the shape of each person's face. In judgment studies we find that such a low dimensional representation of facial appearance and shape is sufficient for ratings of personality characteristics with high inter-rater agreement. The talk presents evidence that there are differential contributions of gender and dynamics to perceptions of personality constructs.
These data present several interesting statistical challenges in multivariate time series. Exploratory techniques such as windowed cross-correlation suggest high nonstationarity: short periods of high symmetry between conversants punctuated by regions of symmetry breaking. Multifractal structure of head movements varies with the context of the conversation, suggesting that self-symmetry is adapted at least in part to the conversational partner. Low dimensional representation of movements covary with perceptions of many personality constructs. It appears that there is reliable structure to be found, but this structure exhibits nonstationarity that may be one of the characteristics of interest.
Meet the speaker in Room 212 Cockins Hall at 4:30 p.m. Refreshments will be served.
