Correlation Encoder-Decoder Model for Text Generation

Zhang, X; Li, Y; Peng, X; Qiao, X; Zhang, H; Lu, W

Correlation Encoder-Decoder Model for Text Generation

Zhang, X Li, Y Peng, X

Qiao, X Zhang, H Lu, W

Permalink

Publisher:: IEEE
Publication Type:: Conference Proceeding
Citation:: Proceedings of the International Joint Conference on Neural Networks, 2022, 2022-July, pp. 1-7
Issue Date:: 2022-01-01

Embargoed

	Filename	Description	Size
	23e53726-1220-40c1-b98f-855e69fc490d.pdf	Accepted version	525.63 kB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Embargoed
Open Access

This item is currently unavailable due to the publisher's embargo.

The embargo period expires on 30 Sep 2024

Full metadata record

Field	Value	Language
dc.contributor.author	Zhang, X
dc.contributor.author	Li, Y
dc.contributor.author	Peng, X https://orcid.org/0000-0002-8901-1472
dc.contributor.author	Qiao, X
dc.contributor.author	Zhang, H
dc.contributor.author	Lu, W
dc.date	2022-07-18
dc.date.accessioned	2023-02-10T04:47:35Z
dc.date.available	2023-02-10T04:47:35Z
dc.date.issued	2022-01-01
dc.identifier.citation	Proceedings of the International Joint Conference on Neural Networks, 2022, 2022-July, pp. 1-7
dc.identifier.isbn	9781728186719
dc.identifier.uri	http://hdl.handle.net/10453/166045
dc.description.abstract	Text generation is crucial for many applications in natural language processing. With the prevalence of deep learning, the encoder-decoder architecture is dominantly adopted for this task. Accurately encoding the source information is of key importance to text generation, because the target text can be generated only when accurate and complete source information is captured by the encoder and fed into the decoder. However, most existing approaches fail to effectively encode and learn the entire source information, as some features are easy to be missed along with the encoding procedures of the encoder. Similar problems also confuse the implementation of the decoder. How to reduce the problem of information loss in the encoder-decoder model is critical for text generation. To address this issue, we propose a novel correlation encoder-decoder model, which optimizes both the encoder and the decoder to reduce the problem of information loss by enforcing them to minimize the differences between hierarchical layers by maximizing the mutual information. Experimental results on two benchmark datasets demonstrate that the proposed model substantially outperforms the existing state-of-the-art methods. Our source code is publicly available on GitHub1.
dc.language	en
dc.publisher	IEEE
dc.relation.ispartof	Proceedings of the International Joint Conference on Neural Networks
dc.relation.ispartof	2022 International Joint Conference on Neural Networks (IJCNN)
dc.relation.isbasedon	10.1109/IJCNN55064.2022.9891880
dc.rights	info:eu-repo/semantics/embargoedAccess
dc.title	Correlation Encoder-Decoder Model for Text Generation
dc.type	Conference Proceeding
utslib.citation.volume	2022-July
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Strength - AAII - Australian Artificial Intelligence Institute
utslib.copyright.status	embargoed	*
utslib.copyright.embargo	2024-09-30T00:00:00+1000Z
dc.date.updated	2023-02-10T04:47:33Z
pubs.finish-date	2022-07-23
pubs.publication-status	Published
pubs.start-date	2022-07-18
pubs.volume	2022-July

Abstract:

Text generation is crucial for many applications in natural language processing. With the prevalence of deep learning, the encoder-decoder architecture is dominantly adopted for this task. Accurately encoding the source information is of key importance to text generation, because the target text can be generated only when accurate and complete source information is captured by the encoder and fed into the decoder. However, most existing approaches fail to effectively encode and learn the entire source information, as some features are easy to be missed along with the encoding procedures of the encoder. Similar problems also confuse the implementation of the decoder. How to reduce the problem of information loss in the encoder-decoder model is critical for text generation. To address this issue, we propose a novel correlation encoder-decoder model, which optimizes both the encoder and the decoder to reduce the problem of information loss by enforcing them to minimize the differences between hierarchical layers by maximizing the mutual information. Experimental results on two benchmark datasets demonstrate that the proposed model substantially outperforms the existing state-of-the-art methods. Our source code is publicly available on GitHub1.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/166045