Topic-Document Inference with the Gumbel-Softmax Distribution

Kumar, A; Esmaili, N; Piccardi, M

Topic-Document Inference with the Gumbel-Softmax Distribution

Kumar, A Esmaili, N Piccardi, M

Permalink

Publisher:: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Publication Type:: Journal Article
Citation:: IEEE Access, 2021, 9, pp. 1313-1320
Issue Date:: 2021-01-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

The embargo period expires on 31 Jan 2023

Adobe PDF

Download Accepted manuscriptAdobe PDF (621.42 kB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Kumar, A
dc.contributor.author	Esmaili, N
dc.contributor.author	Piccardi, M https://orcid.org/0000-0001-9250-6604
dc.date.accessioned	2021-03-04T07:11:46Z
dc.date.available	2021-03-04T07:11:46Z
dc.date.issued	2021-01-01
dc.identifier.citation	IEEE Access, 2021, 9, pp. 1313-1320
dc.identifier.issn	2169-3536
dc.identifier.issn	2169-3536
dc.identifier.uri	http://hdl.handle.net/10453/146779
dc.description.abstract	© 2013 IEEE. Topic modeling is an important application of natural language processing (NLP) that can automatically identify the set of main topics of a given, typically large, collection of documents. In addition to identifying the main topics in the given collection, topic modeling infers which combination of topics is addressed by each individual document (the so-called topic-document inference), which can be useful for their classification and organization. However, the distributional assumptions for this inference are typically restricted to the Dirichlet family which can limit the performance of the model. For this reason, in this paper we propose modeling the topic-document inference with the Gumbel-Softmax distribution, a distribution recently introduced to expand differentiability in deep networks. To set up a performing system, the proposed approach integrates Gumbel-Softmax topic-document inference in a state-of-the-art topic model based on a deep variational autoencoder. Experimental results over two probing datasets show that the proposed approach has been able to outperform the original deep variational autoencoder and other popular topic models in terms of test-set perplexity and two topic coherence measures.
dc.language	English
dc.publisher	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
dc.relation.ispartof	IEEE Access
dc.relation.isbasedon	10.1109/ACCESS.2020.3046607
dc.rights	info:eu-repo/semantics/embargoedAccess
dc.rights	© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.	en_US
dc.subject	08 Information and Computing Sciences, 09 Engineering, 10 Technology
dc.title	Topic-Document Inference with the Gumbel-Softmax Distribution
dc.type	Journal Article
utslib.citation.volume	9
utslib.for	08 Information and Computing Sciences
utslib.for	09 Engineering
utslib.for	10 Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - GBDTC - Global Big Data Technologies
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
pubs.organisational-group	/University of Technology Sydney
utslib.copyright.status	open_access	*
pubs.consider-herdc	true
utslib.copyright.embargo	2023-01-31T00:00:00+1000Z
dc.date.updated	2021-03-04T07:11:45Z
pubs.publication-status	Published
pubs.volume	9

Abstract:

© 2013 IEEE. Topic modeling is an important application of natural language processing (NLP) that can automatically identify the set of main topics of a given, typically large, collection of documents. In addition to identifying the main topics in the given collection, topic modeling infers which combination of topics is addressed by each individual document (the so-called topic-document inference), which can be useful for their classification and organization. However, the distributional assumptions for this inference are typically restricted to the Dirichlet family which can limit the performance of the model. For this reason, in this paper we propose modeling the topic-document inference with the Gumbel-Softmax distribution, a distribution recently introduced to expand differentiability in deep networks. To set up a performing system, the proposed approach integrates Gumbel-Softmax topic-document inference in a state-of-the-art topic model based on a deep variational autoencoder. Experimental results over two probing datasets show that the proposed approach has been able to outperform the original deep variational autoencoder and other popular topic models in terms of test-set perplexity and two topic coherence measures.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/146779