A graph-theoretic summary evaluation for ROUGE

ShafieiBavani, E; Ebrahimi, M; Wong, R; Chen, F

A graph-theoretic summary evaluation for ROUGE

ShafieiBavani, E Ebrahimi, M Wong, R Chen, F

Permalink

Publisher:: The Association for Computational Linguistics
Publication Type:: Conference Proceeding
Citation:: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, 2018, pp. 762-767
Issue Date:: 2018

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published VersionAdobe PDF (319.23 kB)

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	ShafieiBavani, E
dc.contributor.author	Ebrahimi, M
dc.contributor.author	Wong, R
dc.contributor.author	Chen, F https://orcid.org/0000-0003-4971-8729
dc.date	2018-10-31
dc.date.accessioned	2021-04-08T06:25:01Z
dc.date.available	2021-04-08T06:25:01Z
dc.date.issued	2018
dc.identifier.citation	Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, 2018, pp. 762-767
dc.identifier.isbn	9781948087841
dc.identifier.uri	http://hdl.handle.net/10453/147900
dc.description.abstract	ROUGE is one of the first and most widely used evaluation metrics for text summarization. However, its assessment merely relies on surface similarities between peer and model summaries. Consequently, ROUGE is unable to fairly evaluate summaries including lexical variations and paraphrasing. We propose a graph-based approach adopted into ROUGE to evaluate summaries based on both lexical and semantic similarities. Experiment results over TAC AESOP datasets show that exploiting the lexico-semantic similarity of the words used in summaries would significantly help ROUGE correlate better with human judgments.
dc.language	en
dc.publisher	The Association for Computational Linguistics
dc.relation.ispartof	Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018
dc.relation.ispartof	Conference on Empirical Methods in Natural Language Processing
dc.rights	info:eu-repo/semantics/openAccess
dc.title	A graph-theoretic summary evaluation for ROUGE
dc.type	Conference Proceeding
utslib.location.activity	Brussels, Belgium
utslib.for	0803 Computer Software
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/A/DRsch The Data Science Institute
utslib.copyright.status	open_access	*
pubs.consider-herdc	false
dc.date.updated	2021-04-08T06:25:00Z
pubs.finish-date	2018-11-04
pubs.place-of-publication	USA
pubs.publication-status	Published
pubs.start-date	2018-10-31
dc.location	USA

Abstract:

ROUGE is one of the first and most widely used evaluation metrics for text summarization. However, its assessment merely relies on surface similarities between peer and model summaries. Consequently, ROUGE is unable to fairly evaluate summaries including lexical variations and paraphrasing. We propose a graph-based approach adopted into ROUGE to evaluate summaries based on both lexical and semantic similarities. Experiment results over TAC AESOP datasets show that exploiting the lexico-semantic similarity of the words used in summaries would significantly help ROUGE correlate better with human judgments.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/147900