Neural Topic Model Training with the REBAR Gradient Estimator
- Publisher:
- Association for Computing Machinery (ACM)
- Publication Type:
- Journal Article
- Citation:
- ACM Transactions on Asian and Low-Resource Language Information Processing, 2022, 21, (5), pp. 1-18
- Issue Date:
- 2022-11-15
Closed Access
| Filename | Description | Size | |||
|---|---|---|---|---|---|
| 3517336.pdf | Published version | 450.63 kB |
Copyright Clearance Process
- Recently Added
- In Progress
- Closed Access
This item is closed access and not available.
Topic modelling is an important approach of unsupervised machine learning that allows automatically extracting the main "topics"from large collections of documents. In addition, topic modelling is able to identify the topic proportions of each individual document, which can be helpful for organizing the collections. Many topic modelling algorithms have been proposed to date, including several that leverage advanced techniques such as variational inference and deep autoencoders. However, to date topic modelling has made limited use of reinforcement learning, a framework that has obtained vast success in many other unsupervised learning tasks. For this reason, in this article we propose training a neural topic model using a reinforcement learning objective and minimizing the objective with the recently-proposed REBAR gradient estimator. Experiments performed over two probing datasets have shown that the proposed model has achieved improvements over all the compared models in terms of both model perplexity and topic coherence, and produced topics that appear qualitatively informative and consistent.
Please use this identifier to cite or link to this item: