Soutenance de thèse Mohammad Soleymani

M. Mohammad Soleymani soutiendra, en vue de l'obtention du grade de docteur ès sciences, mention informatique, sa thèse intitulée

Implicit and Automated Emotional Tagging of Videos

Abstract :

Emotions play an important role in viewers' content selection and consumption. The main aim of the study which was carried out and reported in this thesis is to detect and estimate affective characteristics of videos based on automated content analysis as well as to recognize the felt emotions in viewers in response to videos. These emotional characterizations can be used to tag the content. Implicit or automated tagging of videos using affective information help recommendation and retrieval systems to improve their performance.

The analysis and evaluations directions in this thesis are twofold: first, methodology and results of emotion recognition methods employed to detect emotion in response to videos are presented. Second, methodology and results of emotional understanding of multimedia using content analysis are provided. In the first direction, a regression based method to detect emotion in continuous space is presented and the correlates of emotional self assessments and physiological responses are shown. Moreover, a multi-modal participant independent emotion recognition study is presented. The second study shows the performance of an inter-participant emotion recognition tagging approach using participants' EEG signals, gaze distance and pupillary response as affective feedbacks. The feasibility of an approach to recognize emotion in response to videos using such a system is shown. The best classification accuracy of 68.5% for three labels of valence and 76.4% for three labels of arousal are obtained using a modality fusion strategy and a support vector machine. After studying the responses to movie scenes, the results and the methods for emotion assessment in response to music clips are given.

Moreover, content analysis methods to detect emotions that are more likely to be elicited by a given multimedia content are presented. Low level content features which are used for affective understanding are introduced. Again the regression method is used for affective understanding of videos and the correlation between content features, physiological responses and emotional self reports have been studied. Content based multimedia features' correlations with both physiological features and users' self-assessment of valence-arousal are shown to be significant. This implies the usefulness of physiological responses and content features for emotional tagging of videos.

Next, an affective representation system for estimating felt emotions at the scene level has been proposed using a Bayesian classification framework. The classification accuracy of 56% that was obtained on three emotional classes with a naive Bayesian classifier was improved to 64% after utilizing temporal and genre priors.

In conclusion, promising results have been obtained in emotional tagging of videos. However, emotional understanding of multimedia is a challenging task and with the current state of the art methods a universal solution to detect and tag all different content which suits all the users is not possible. Systems based on affective computing can only work if they are able to take context and personal profiles into account.

Date: Vendredi 4 novembre 2011 à 14h00

Lieu: Battelle bât.A - Auditoire rez-de-chaussée

31 octobre 2011

À la Une