An attention-based word-level interaction model for knowledge base relation detection

Zhang, H; Xu, G; Liang, X; Li, F; Fu, K; Wang, L; Huang, T

An attention-based word-level interaction model for knowledge base relation detection

Zhang, H Xu, G

Liang, X Li, F Fu, K Wang, L Huang, T

Permalink

Publication Type:: Journal Article
Citation:: IEEE Access, 2018, 6 pp. 75429 - 75441
Issue Date:: 2018-01-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Accepted manuscriptAdobe PDF (1.43 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Zhang, H	en_US
dc.contributor.author	Xu, G https://orcid.org/0000-0003-4493-6663	en_US
dc.contributor.author	Liang, X	en_US
dc.contributor.author	Li, F	en_US
dc.contributor.author	Fu, K	en_US
dc.contributor.author	Wang, L	en_US
dc.contributor.author	Huang, T	en_US
dc.date.available	2020-11-30T18:07:10Z
dc.date.issued	2018-01-01	en_US
dc.identifier.citation	IEEE Access, 2018, 6 pp. 75429 - 75441	en_US
dc.identifier.uri	http://hdl.handle.net/10453/129500
dc.description.abstract	© 2018 IEEE. Relation detection plays a crucial role in knowledge base question answering, and it is challenging because of the high variance of relation expression in real-world questions. Traditional relation detection models based on deep learning follow an encoding-comparing paradigm, where the question and the candidate relation are represented as vectors to compare their semantic similarity. Max-or average-pooling operation, which is used to compress the sequence of words into fixed-dimensional vectors, becomes the bottleneck of information flow. In this paper, we propose an attention-based word-level interaction model (ABWIM) to alleviate the information loss issue caused by aggregating the sequence into a fixed-dimensional vector before the comparison. First, attention mechanism is adopted to learn the soft alignments between words from the question and the relation. Then, fine-grained comparisons are performed on the aligned words. Finally, the comparison results are merged with a simple recurrent layer to estimate the semantic similarity. Besides, a dynamic sample selection strategy is proposed to accelerate the training procedure without decreasing the performance. Experimental results of relation detection on both SimpleQuestions and WebQuestions datasets show that ABWIM achieves the state-of-the-art accuracy, demonstrating its effectiveness.	en_US
dc.relation.ispartof	IEEE Access	en_US
dc.relation.isbasedon	10.1109/ACCESS.2018.2883304	en_US
dc.title	An attention-based word-level interaction model for knowledge base relation detection	en_US
dc.type	Journal Article
utslib.citation.volume	6	en_US
utslib.for	0805 Distributed Computing	en_US
utslib.for	0102 Applied Mathematics	en_US
utslib.for	08 Information and Computing Sciences	en_US
utslib.for	09 Engineering	en_US
utslib.for	10 Technology	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
pubs.organisational-group	/University of Technology Sydney/Strength - AAI - Advanced Analytics Institute Research Centre
utslib.copyright.status	open_access	*
pubs.publication-status	Published	en_US
pubs.volume	6	en_US

Abstract:

© 2018 IEEE. Relation detection plays a crucial role in knowledge base question answering, and it is challenging because of the high variance of relation expression in real-world questions. Traditional relation detection models based on deep learning follow an encoding-comparing paradigm, where the question and the candidate relation are represented as vectors to compare their semantic similarity. Max-or average-pooling operation, which is used to compress the sequence of words into fixed-dimensional vectors, becomes the bottleneck of information flow. In this paper, we propose an attention-based word-level interaction model (ABWIM) to alleviate the information loss issue caused by aggregating the sequence into a fixed-dimensional vector before the comparison. First, attention mechanism is adopted to learn the soft alignments between words from the question and the relation. Then, fine-grained comparisons are performed on the aligned words. Finally, the comparison results are merged with a simple recurrent layer to estimate the semantic similarity. Besides, a dynamic sample selection strategy is proposed to accelerate the training procedure without decreasing the performance. Experimental results of relation detection on both SimpleQuestions and WebQuestions datasets show that ABWIM achieves the state-of-the-art accuracy, demonstrating its effectiveness.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/129500