Skip to main content

How Do We Experience Crossmodal Correspondent Mulsemedia Content?

Covaci, Alexandra, Bissoli Saleme, Estêvão, Mesfin, Gebremariam, Nadia, Hussain, Kani-Zabihi, Elahe, Ghinea, George (2019) How Do We Experience Crossmodal Correspondent Mulsemedia Content? IEEE Transactions on Multimedia, 22 (5). pp. 1249-1258. ISSN 1520-9210. E-ISSN 1941-0077. (doi:10.1109/TMM.2019.2941274) (KAR id:77596)

PDF Author's Accepted Manuscript
Language: English
Download (10MB) Preview
[thumbnail of 08836637.pdf]
This file may not be suitable for users of assistive technology.
Request an accessible format
Official URL


Sensory studies emerged as a significant influence upon Human Computer Interaction and traditional multimedia. Mulsemedia is an area that extends multimedia addressing issues of multisensorial response through the combination of at least three media, typically a non-traditional media with traditional audio-visual content. In this paper, we explore the concepts of Quality of Experience and crossmodal correspondences through a case study of different types of mulsemedia setups. The content is designed following principles of crossmodal correspondence between different sensory dimensions and delivered through olfactory, auditory and vibrotactile displays. The Quality of Experience is evaluated through both subjective (questionnaire) and objective means (eye gaze and heart rate). Results show that the auditory experience has an influence on the olfactory sensorial responses and lessens the perception of lingering odor. Heat maps of the eye gazes suggest that the crossmodality between olfactory and visual content leads to an increased visual attention on the factors of the employed crossmodal correspondence (e.g., color, brightness, shape).

Item Type: Article
DOI/Identification number: 10.1109/TMM.2019.2941274
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Engineering and Digital Arts
Depositing User: Alexandra Covaci
Date Deposited: 18 Oct 2019 16:06 UTC
Last Modified: 16 Feb 2021 14:08 UTC
Resource URI: (The current URI for this page, for reference purposes)
Covaci, Alexandra:
  • Depositors only (login required):


Downloads per month over past year