Real-Time Emotion Measurement in Consumption Decisions

Emotions play a decisive role in almost all areas of the customer journey. They can trigger impulse purchases, foster product or brand relations, or determine the reaction to advertising and its perception. In general, emotions are relevance detectors, which is why the automatic detection of emotional reactions to stimuli like TV commercials has become a focus topic in Affective Computing. But which emotional reactions do consumers show, e.g., in response to TV commercials, and how can they be measured?

Scientists agree that emotional processes happen at different levels: the subjective experience, the physiological level, and the behavioral-expressive level. However, while measurement approaches exist at each of these levels, there is a sparsity of research trying to identify response pattern across different processing levels.

This project tries to address this issue by using physiological and behavioral indicators to identify the emotional states of valence and arousal in order to evaluate and compare measurement approaches.

For addressing the research gaps, we experimentally induced emotional states, namely positive and negative valence, as well as high and low arousal, and measured their interaction with physiological responses (specifically heart rate) and vocal, and facial expression.

Besides implications for academic research, this topic is also of importance for practical applications. Emotions are one of the main drivers in many different decision situations and can provoke, e.g., product or brand loyalty, or word-of-mouth. Thus, their measurement is relevant in many different areas. However, the measurement approaches differ with respect to their costs, applicability, reliability, etc. For practical applications, for example, there are instances where webcams can be used, such as in determining emotions of customers in service calls that are done via video chat. With webcams, not only voice and video, but also heart rates could be measured easily. However, the heart rate measurements may not be very accurate. Moreover, voice and videos could be used in situation where emotions of whole groups need to be analyzed, whereas measuring physiology in groups of people might be very effortful and hard to implement. Hence, there are trade-offs to be made for selecting the measurement approaches.

This project answers the question of which methods are most appropriate for different emotional reactions and how these methods can be applied in marketing contexts.