Comparing automatically detected reflective texts with human judgements

Ullmann, TD; Wild, F; Scott, P

Comparing automatically detected reflective texts with human judgements

Ullmann, TD Wild, F Scott, P

Permalink

Publication Type:: Conference Proceeding
Citation:: CEUR Workshop Proceedings, 2012, 931 pp. 101 - 116
Issue Date:: 2012-01-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published versionAdobe PDF (371.3 kB)

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Ullmann, TD	en_US
dc.contributor.author	Wild, F	en_US
dc.contributor.author	Scott, P https://orcid.org/0000-0001-5944-9361	en_US
dc.date.issued	2012-01-01	en_US
dc.identifier.citation	CEUR Workshop Proceedings, 2012, 931 pp. 101 - 116	en_US
dc.identifier.issn	1613-0073	en_US
dc.identifier.uri	http://hdl.handle.net/10453/121314
dc.description.abstract	This paper reports on the descriptive results of an experiment comparing automatically detected reflective and not-reflective texts against human judgements. Based on the theory of reflective writing assessment and their operationalisation five elements of reflection were defined. For each element of reflection a set of indicators was developed, which automatically annotate texts regarding reflection based on the parameterisation with authoritative texts. Using a large blog corpus 149 texts were retrieved, which were either annotated as reflective or notreflective. An online survey was then used to gather human judgements for these texts. These two data sets were used to compare the quality of the reflection detection algorithm with human judgments. The analysis indicates the expected difference between reflective and not-reflective texts.	en_US
dc.relation.ispartof	CEUR Workshop Proceedings	en_US
dc.title	Comparing automatically detected reflective texts with human judgements	en_US
dc.type	Conference Proceeding
utslib.citation.volume	931	en_US
utslib.for	1303 Specialist Studies in Education	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/DVC (Teaching and Learning)
utslib.copyright.status	open_access
pubs.publication-status	Published	en_US
pubs.volume	931	en_US

Abstract:

This paper reports on the descriptive results of an experiment comparing automatically detected reflective and not-reflective texts against human judgements. Based on the theory of reflective writing assessment and their operationalisation five elements of reflection were defined. For each element of reflection a set of indicators was developed, which automatically annotate texts regarding reflection based on the parameterisation with authoritative texts. Using a large blog corpus 149 texts were retrieved, which were either annotated as reflective or notreflective. An online survey was then used to gather human judgements for these texts. These two data sets were used to compare the quality of the reflection detection algorithm with human judgments. The analysis indicates the expected difference between reflective and not-reflective texts.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/121314