Unsupervised keyword extraction from microblog posts via hashtags

Li, L; Liu, J; Sun, Y; Xu, G; Yuan, J; Zhong, L

Unsupervised keyword extraction from microblog posts via hashtags

Li, L Liu, J Sun, Y Xu, G

Yuan, J Zhong, L

Permalink

Publication Type:: Journal Article
Citation:: Journal of Web Engineering, 2018, 17 (1-2), pp. 93 - 120
Issue Date:: 2018-03-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published VersionAdobe PDF (1.24 MB)

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Li, L	en_US
dc.contributor.author	Liu, J	en_US
dc.contributor.author	Sun, Y	en_US
dc.contributor.author	Xu, G https://orcid.org/0000-0003-4493-6663	en_US
dc.contributor.author	Yuan, J	en_US
dc.contributor.author	Zhong, L	en_US
dc.date.issued	2018-03-01	en_US
dc.identifier.citation	Journal of Web Engineering, 2018, 17 (1-2), pp. 93 - 120	en_US
dc.identifier.issn	1540-9589	en_US
dc.identifier.uri	http://hdl.handle.net/10453/126043
dc.description.abstract	© River Publishers. Nowadays, huge amounts of texts are being generated for social networking purposes on Web. Keyword extraction from such texts like microblog posts benefits many applications such as advertising, search, and content filtering. Unlike traditional web pages, a microblog post usually has some special social feature like a hashtag that is topical in nature and generated by users. Extracting keywords related to hashtags can reflect the intents of users and thus provides us better understanding on post content. In this paper, we propose a novel unsupervised keyword extraction approach for microblog posts by treating hashtags as topical indicators. Our approach consists of two hashtag enhanced algorithms. One is a topic model algorithm that infers topic distributions biased to hashtags on a collection of microblog posts. The words are ranked by their average topic probabilities. Our topic model algorithm can not only find the topics of a collection, but also extract hashtag-related keywords. The other is a random walk based algorithm. It first builds a word-post weighted graph by taking into account posts themselves. Then, a hashtag biased random walk is applied on this graph, which guides the algorithm to extract keywords according to hashtag topics. Last, the final ranking score of a word is determined by the stationary probability after a number of iterations. We evaluate our proposed approach on a collection of real Chinese microblog posts. Experiments show that our approach is more effective in terms of precision than traditional approaches considering no hashtag. The result achieved by the combination of two algorithms performs even better than each individual algorithm.	en_US
dc.relation.ispartof	Journal of Web Engineering	en_US
dc.title	Unsupervised keyword extraction from microblog posts via hashtags	en_US
dc.type	Journal Article
utslib.citation.volume	1-2	en_US
utslib.citation.volume	17	en_US
utslib.for	0804 Data Format	en_US
utslib.for	0803 Computer Software	en_US
utslib.for	08 Information and Computing Sciences	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
pubs.organisational-group	/University of Technology Sydney/Strength - AAI - Advanced Analytics Institute Research Centre
utslib.copyright.status	open_access
pubs.issue	1-2	en_US
pubs.publication-status	Published	en_US
pubs.volume	17	en_US

Abstract:

© River Publishers. Nowadays, huge amounts of texts are being generated for social networking purposes on Web. Keyword extraction from such texts like microblog posts benefits many applications such as advertising, search, and content filtering. Unlike traditional web pages, a microblog post usually has some special social feature like a hashtag that is topical in nature and generated by users. Extracting keywords related to hashtags can reflect the intents of users and thus provides us better understanding on post content. In this paper, we propose a novel unsupervised keyword extraction approach for microblog posts by treating hashtags as topical indicators. Our approach consists of two hashtag enhanced algorithms. One is a topic model algorithm that infers topic distributions biased to hashtags on a collection of microblog posts. The words are ranked by their average topic probabilities. Our topic model algorithm can not only find the topics of a collection, but also extract hashtag-related keywords. The other is a random walk based algorithm. It first builds a word-post weighted graph by taking into account posts themselves. Then, a hashtag biased random walk is applied on this graph, which guides the algorithm to extract keywords according to hashtag topics. Last, the final ranking score of a word is determined by the stationary probability after a number of iterations. We evaluate our proposed approach on a collection of real Chinese microblog posts. Experiments show that our approach is more effective in terms of precision than traditional approaches considering no hashtag. The result achieved by the combination of two algorithms performs even better than each individual algorithm.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/126043