An Auditory-Visual Conflict of Emotions: Evidence from McGurk Effect

Abstract

Experiments such as the McGurk effect have supported the inter-dependence of the auditory and visual modalities. This perceptive reasoning is in line with the assumption that emotional expressions in the face and voice are processed by the same perceptual/cognitive mechanism. This research aimed to study the accuracy of the identification of intended emotions in the Kannada language using the auditory and/or visual modality; and also to study its perception using the McGurk paradigm. An emotionally neutral word “namaskara” was uttered by a native Kannada speaker in four basic emotions. Subjects were asked to identify the intended emotional expressions under the different experimental conditions (unimodal, bimodal-I, bimodal-II). The scores were recorded and analysed respectively. The results were in line with studies who also stated the over-reliance of visual over auditory modality when the subjects were presented with the McGurk stimuli thereby also perceiving a new different emotion.