Artificial Voices in Human Choices

More and more often we meet artificial language in everyday life: When driving a car with navigation, when doing a search on the Internet via voice or when speaking with Alexa, to name just a few examples. Thanks to the latest developments in the field of artificial intelligence, artificial voices do sound more and more natural. The voice is a powerful information carrier. Besides the content, the sound reveals a lot about personal and social characteristics such as gender, personality, identity and emotion. The interpretation of these characteristics is essential in interpersonal interaction and has a great influence on behavior and decisions. But what about the linguistic interaction between man and machine? The aim of our research project is to analyze the influence of the tone of artificial voices on consumer decisions. Using the latest deep learning methods from the field of artificial intelligence, we first generate artificial voices with a specific tone of voice and then test its influence on consumer decisions in various experiments.

An important social characteristic of the human voice is emotion. Emotions are often contagious in human-human interaction. By imitating facial expressions, gestures and voice, people empathize with the mood of their dialog partner. But does the phenomenon of emotional contagion also work in human-machine communication? And what effect does this possibly have on human behavior? For example, a question from a marketing point of view: Can an enthusiastic-sounding language assistant make consumers excited about a product and encourage them to make an impulse purchase? For this purpose, we have developed a deep learning model, which is able to generate artificial language with different emotions. This enables us to test the influence of synthetic voices on buying behavior in laboratory experiments.

New deep learning methods make it possible to create artificial speech in a tone of voice so that it sounds like the voices of specific persons. Amazon, for example, offers its Alexa customers in the U.S. Samuel L. Jackson’s voice as output voice – and more celebrity voices are planned. For marketing, this technology opens up new opportunities (e.g., presentation of personalized content in the voice of a favorite star) but also new risks (e.g., publication of fake news in the voice of an important decision-maker).

In our research, we therefore analyze the question of how artificial voices in the voice of well-known people affect consumers. Moreover, we examine to what extent education about the possibilities of modern speech synthesis can influence the effect and may counteract Fake-Voice news, for example. Existing research in the field of human-computer interaction has shown that people mentally assign a personality to artificial voices and find several artificial voices more credible than one artificial voice – even if they are told that they hear artificially generated voices. The research question for us is therefore whether this is also the case with well-known artificially generated voices.

Speech assistants are booming: Intelligent speakers from Amazon, Google and other providers are becoming more and more frequently part of our households, and many of their owners are already using the intelligent speakers for everyday tasks: They ask the speakers to suggest music or products, for example. Users can also imagine having some of their regular purchases organized by a voice assistant in the near future. However, the owners would prefer more emotionality, happiness or variety in the voice of their voice assistant – as another study by NIM shows.

 It is therefore essential for marketing to deal with the growing importance of digital voice assistants as a new sales channel. To what extent will the devices shift the balance of power between consumers, dealers and manufacturers? This question is still open. But it is clear:  With emotionality in their voice, speech assistants have arrived at the core of purchase decisions. The fact that emotions in human-human interaction – for example, triggered by voice - are contagious and influence shopping behavior has been extensively documented for years. For example, emotionally excited customers tend to make more impulse purchases.

Since one of the aims of the Nuremberg Institute for Market Decisions is to analyze the influence of trends and new technologies on the decision-making behavior of consumers, it investigated in November 2019 whether and how the emotional tone of voice assistants affects the buying behavior of consumers. In our study "Artificial Voices in Human Choices – The Impact of Synthesized Speech with Emotional Tone on Consumer Behavior", 213 students from the Friedrich-Alexander University of Erlangen-Nuremberg took part.

In order to obtain reliable data for the still little-researched area, a multi-step procedure model was necessary, orienting on the scientifically established stimulus-organism reaction model. Emotional voices of speech assistants, which had to be learned by them first, served as the stimulus. Subsequently, the emotions of the test participants were measured in the dimensions valence (positive vs. negative) and arousal (calm vs. emotionally excited). Afterwards, the concrete reaction to the stimulus was determined in a shopping experiment – by observing the buying behavior and querying buying attitudes.

Here you can hear synthetic voices developed by our researchers Dr. Carolin Kaiser and René Schallner. The voices are part of their research project Artificial Voices In Human Choices.

Excited voice  

Happy voice 

Uninvolved voice