Skip to main content

An Attention Pooling based Representation Learning Method for Speech Emotion Recognition

Li, Pengcheng, Song, Yan, McLoughlin, Ian Vince, Guo, Wu, Dai, Li-Rong (2018) An Attention Pooling based Representation Learning Method for Speech Emotion Recognition. In: ISCA Conference. . International Speech Communication Association (doi:10.21437/Interspeech.2018-1242) (KAR id:67453)

PDF Author's Accepted Manuscript
Language: English
Download (551kB) Preview
[img]
Preview
Official URL
http://dx.doi.org/10.21437/Interspeech.2018-1242

Abstract

This paper proposes an attention pooling based representation learning method for speech emotion recognition (SER). The emotional representation is learned in an end-to-end fashion by applying a deep convolutional neural network (CNN) directly to spectrograms extracted from speech utterances. Motivated by the success of GoogleNet, two groups of filters with different shapes are designed to capture both temporal and frequency domain context information from the input spectrogram. The learned features are concatenated and fed into the subsequent convolutional layers. To learn the final emotional representation, a novel attention pooling method is further proposed.

Compared with the existing pooling methods, such as max-pooling and average-pooling, the proposed attention pooling can effectively incorporate class-agnostic bottom-up, and class-specific top-down, attention maps. We conduct extensive evaluations on benchmark IEMOCAP data to assess the effectiveness of the proposed representation. Results demonstrate a recognition performance of 71.8% weighted accuracy (WA) and 68% unweighted accuracy (UA) over four emotions, which outperforms the state-of-the-art method by about 3% absolute for WA and 4% for UA.

Item Type: Conference or workshop item (Paper)
DOI/Identification number: 10.21437/Interspeech.2018-1242
Uncontrolled keywords: Speech emotion recognition, high-level feature learning, convolutional neural network, second-order pooling
Subjects: T Technology
Divisions: Faculties > Sciences > School of Computing > Data Science
Depositing User: Ian McLoughlin
Date Deposited: 29 Jun 2018 09:28 UTC
Last Modified: 29 May 2019 20:40 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/67453 (The current URI for this page, for reference purposes)
McLoughlin, Ian Vince: https://orcid.org/0000-0001-7111-2008
  • Depositors only (login required):

Downloads

Downloads per month over past year