Skip to main content

An Attention Pooling based Representation Learning Method for Speech Emotion Recognition

Li, Pengcheng, Song, Yan, McLoughlin, Ian Vince, Guo, Wu, Dai, Li-Rong (2018) An Attention Pooling based Representation Learning Method for Speech Emotion Recognition. In: ISCA Conference. . International Speech Communication Association (doi:10.21437/Interspeech.2018-1242) (KAR id:67453)


This paper proposes an attention pooling based representation learning method for speech emotion recognition (SER). The emotional representation is learned in an end-to-end fashion by applying a deep convolutional neural network (CNN) directly to spectrograms extracted from speech utterances. Motivated by the success of GoogleNet, two groups of filters with different shapes are designed to capture both temporal and frequency domain context information from the input spectrogram. The learned features are concatenated and fed into the subsequent convolutional layers. To learn the final emotional representation, a novel attention pooling method is further proposed.

Compared with the existing pooling methods, such as max-pooling and average-pooling, the proposed attention pooling can effectively incorporate class-agnostic bottom-up, and class-specific top-down, attention maps. We conduct extensive evaluations on benchmark IEMOCAP data to assess the effectiveness of the proposed representation. Results demonstrate a recognition performance of 71.8% weighted accuracy (WA) and 68% unweighted accuracy (UA) over four emotions, which outperforms the state-of-the-art method by about 3% absolute for WA and 4% for UA.

Item Type: Conference or workshop item (Paper)
DOI/Identification number: 10.21437/Interspeech.2018-1242
Uncontrolled keywords: Speech emotion recognition, high-level feature learning, convolutional neural network, second-order pooling
Subjects: T Technology
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Depositing User: Ian McLoughlin
Date Deposited: 29 Jun 2018 09:28 UTC
Last Modified: 09 Dec 2022 01:16 UTC
Resource URI: (The current URI for this page, for reference purposes)

University of Kent Author Information

McLoughlin, Ian Vince.

Creator's ORCID:
CReDIT Contributor Roles:
  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.