Skip to main content

Context-aware Mouse Behaviour Recognition using Hidden Markov Models

Jiang, Zheheng, Crookes, Danny, Green, Brian D, Zhao, Yunfeng, Ma, Haiping, Li, Ling, Zhang, Shengping, Tao, Dacheng, Zhou, Huiyu (2018) Context-aware Mouse Behaviour Recognition using Hidden Markov Models. IEEE Transactions on Image Processing, 28 (3). pp. 1133-1148. ISSN 1057-7149. E-ISSN 1941-0042. (doi:10.1109/TIP.2018.2875335) (KAR id:69761)

PDF Author's Accepted Manuscript
Language: English
Download (2MB) Preview
[thumbnail of bare_jrnl.pdf]
This file may not be suitable for users of assistive technology.
Request an accessible format
Official URL


Automated recognition of mouse behaviours is crucial in studying psychiatric and neurologic diseases. To achieve this objective, it is very important to analyse temporal dynamics of mouse behaviours. In particular, the change between mouse neighbouring actions is swift in a short period. In this paper, we develop and implement a novel Hidden Markov Model (HMM) algorithm to describe the temporal characteristics of mouse behaviours. In particular, we here propose a hybrid deep learning architecture, where the first unsupervised layer relies on an advanced spatial-temporal segment Fisher Vector (SFV) encoding both visual and contextual features. Subsequent supervised layers based on our segment aggregate network (SAN) are trained to estimate the state dependent observation probabilities of the HMM. The proposed architecture shows the ability to discriminate between visually similar behaviours and results in high recognition rates with the strength of processing imbalanced mouse behaviour datasets. Finally, we evaluate our approach using JHuang’s and our own datasets, and the results show that our method outperforms other state-of-the-art approaches.

Item Type: Article
DOI/Identification number: 10.1109/TIP.2018.2875335
Uncontrolled keywords: Mouse behaviours, Hidden Markov Model, spatial-temporal segment, Fisher Vector, segment aggregate network
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming,
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Depositing User: Caroline Li
Date Deposited: 23 Oct 2018 18:48 UTC
Last Modified: 16 Feb 2021 13:58 UTC
Resource URI: (The current URI for this page, for reference purposes)
Li, Ling:
  • Depositors only (login required):


Downloads per month over past year