Skip to main content

How Do We Experience Crossmodal Correspondent Mulsemedia Content?

Covaci, Alexandra, Bissoli Saleme, Estêvão, Mesfin, Gebremariam, Nadia, Hussain, Kani-Zabihi, Elahe, Ghinea, George (2019) How Do We Experience Crossmodal Correspondent Mulsemedia Content? IEEE Transactions on Multimedia, 22 (5). pp. 1249-1258. ISSN 1520-9210. E-ISSN 1941-0077. (doi:10.1109/TMM.2019.2941274) (KAR id:77596)

Abstract

Sensory studies emerged as a significant influence upon Human Computer Interaction and traditional multimedia. Mulsemedia is an area that extends multimedia addressing issues of multisensorial response through the combination of at least three media, typically a non-traditional media with traditional audio-visual content. In this paper, we explore the concepts of Quality of Experience and crossmodal correspondences through a case study of different types of mulsemedia setups. The content is designed following principles of crossmodal correspondence between different sensory dimensions and delivered through olfactory, auditory and vibrotactile displays. The Quality of Experience is evaluated through both subjective (questionnaire) and objective means (eye gaze and heart rate). Results show that the auditory experience has an influence on the olfactory sensorial responses and lessens the perception of lingering odor. Heat maps of the eye gazes suggest that the crossmodality between olfactory and visual content leads to an increased visual attention on the factors of the employed crossmodal correspondence (e.g., color, brightness, shape).

Item Type: Article
DOI/Identification number: 10.1109/TMM.2019.2941274
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Engineering and Digital Arts
Depositing User: Alexandra Covaci
Date Deposited: 18 Oct 2019 16:06 UTC
Last Modified: 09 Dec 2022 05:51 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/77596 (The current URI for this page, for reference purposes)

University of Kent Author Information

  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.