Skip to main content

Word Embedding as Maximum A Posteriori Estimation

Jameel, Shoaib, Fu, Zihao, Shi, Bei, Lam, Wai, Schockart, Steven (2019) Word Embedding as Maximum A Posteriori Estimation. In: Proceedings of the AAAI Conference on Artificial Intelligence. 33 (1). pp. 6562-6569. Association for the Advancement of Artificial Intelligence (doi:10.1609/aaai.v33i01.33016562)

PDF (updated version) - Author's Accepted Manuscript
Download (182kB) Preview
[img]
Preview
Official URL
https://doi.org/10.1609/aaai.v33i01.33016562

Abstract

The GloVe word embedding model relies on solving a global optimization problem, which can be reformulated as a maximum likelihood estimation problem. In this paper, we propose to generalize this approach to word embedding by considering parametrized variants of the GloVe model and incorporating priors on these parameters. To demonstrate the usefulness of this approach, we consider a word embedding model in which each context word is associated with a corresponding variance, intuitively encoding how informative it is. Using our framework, we can then learn these variances together with the resulting word vectors in a unified way. We experimentally show that the resulting word embedding models outperform GloVe, as well as many popular alternatives.

Item Type: Conference or workshop item (Proceeding)
DOI/Identification number: 10.1609/aaai.v33i01.33016562
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming,
Divisions: Faculties > Sciences > School of Computing
Faculties > Sciences > School of Computing > Data Science
Depositing User: Shoaib Jameel
Date Deposited: 09 Nov 2018 10:56 UTC
Last Modified: 23 Aug 2019 11:50 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/70009 (The current URI for this page, for reference purposes)
  • Depositors only (login required):

Downloads

Downloads per month over past year