"Facial Expression Analysis by Computer Processing."
National Institute of Mental Health
5/1/06 to 4/30/11

Joint work with Takeo Kanade, Simon Lucey, Fernando De la Torre, and Zara Ambadar.

Description: Facial expression provides cues about emotional response, regulates interpersonal behavior, and communicates aspects of psychopathology. Human-observer based methods for measuring facial expression are labor intensive, qualitative, and difficult to standardize. Supported by the NIMH (NIMH #1R01MH51435), our interdisciplinary team of computer and behavioral scientists has developed the CMU/Pitt Automated Facial Image Analysis (AFA) system that is capable of automatically recognizing facial action units and analyzing their timing in facial behavior. The quantitative measurement achieved by AFA represents a major advance over manual and subjective measurement without requiring the use of invasive sensors.

We envision to use AFA’s reliable, valid, and efficient measurement of emotion expression and related nonverbal behavior for assessment of symptom severity in depression. AFA is capable of extracting both the type and timing of nonverbal indicators of depression. We hypothesize that quantitative measures of the configuration and timing of facial expression, head motion, and gaze obtainable by AFA will improve clinical assessment of symptom severity and evaluation of treatment outcomes when combined with information from interviews and self-report questionnaires.