Recognizing Reflection: Computer-Assisted Analysis of First Year Medical Students’ Reflective Writing

Hanlon, CD; Frosch, EM; Shochet, RB; Buckingham Shum, SJ; Gibson, A; Goldberg, HR

Recognizing Reflection: Computer-Assisted Analysis of First Year Medical Students’ Reflective Writing

Hanlon, CD Frosch, EM Shochet, RB Buckingham Shum, SJ Gibson, A Goldberg, HR

Permalink

Publisher:: Springer
Publication Type:: Journal Article
Citation:: Medical Science Educator, 2021, 31, (1), pp. 109-116
Issue Date:: 2021-01-01

Closed Access

	Filename	Description	Size
	Hanlon2021_Article_RecognizingReflectionComputer-.pdf	Published version	401.7 kB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Hanlon, CD
dc.contributor.author	Frosch, EM
dc.contributor.author	Shochet, RB
dc.contributor.author	Buckingham Shum, SJ
dc.contributor.author	Gibson, A
dc.contributor.author	Goldberg, HR
dc.date.accessioned	2022-05-27T07:58:19Z
dc.date.available	2020-10-15
dc.date.available	2022-05-27T07:58:19Z
dc.date.issued	2021-01-01
dc.identifier.citation	Medical Science Educator, 2021, 31, (1), pp. 109-116
dc.identifier.issn	2156-8650
dc.identifier.issn	2156-8650
dc.identifier.uri	http://hdl.handle.net/10453/157763
dc.description.abstract	© 2020, International Association of Medical Science Educators. Background: Reflective writing is used throughout medical education to help students navigate their transformation into medical professionals. Assessment of reflective writing, however, is challenging; each available methodology of assessment has distinct advantages and disadvantages. We tested if combining two independent assessment mechanisms—a faculty-designed rubric and Academic Writing Analytics (AWA), an automated technique—could be used together to form a more robust form of evaluation. Methods: We obtained reflective essays written by first year medical students as part of a clinical skills course. Faculty scored essays using a rubric designed to evaluate Integration, Depth, and Writing. The same essays were subjected to AWA analysis, which counted the number of reflective phrases indicative of Context, Challenge, or Change. Results: Faculty scored the essays uniformly high, indicating that most students met the standard for reflection as described by the rubric. AWA identified over 1400 instances of reflective behavior within the essays, and there was significant variability in how often different types of reflective phrases were used by individual students. Conclusions: While data from faculty assessment or AWA alone is sufficient to evaluate reflective essays, combining these methods offer a richer and more valuable understanding of the student’s reflection.
dc.format	Electronic-eCollection
dc.language	eng
dc.publisher	Springer
dc.relation.ispartof	Medical Science Educator
dc.relation.isbasedon	10.1007/s40670-020-01132-7
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Recognizing Reflection: Computer-Assisted Analysis of First Year Medical Students’ Reflective Writing
dc.type	Journal Article
utslib.citation.volume	31
utslib.location.activity	United States
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/DVC (Teaching and Learning)
pubs.organisational-group	/University of Technology Sydney/DVC (Teaching and Learning)/Connected Intelligence Centre
utslib.copyright.status	closed_access	*
pubs.consider-herdc	true
dc.date.updated	2022-05-27T07:58:16Z
pubs.issue	1
pubs.publication-status	Published
pubs.volume	31
utslib.citation.issue	1

Abstract:

© 2020, International Association of Medical Science Educators. Background: Reflective writing is used throughout medical education to help students navigate their transformation into medical professionals. Assessment of reflective writing, however, is challenging; each available methodology of assessment has distinct advantages and disadvantages. We tested if combining two independent assessment mechanisms—a faculty-designed rubric and Academic Writing Analytics (AWA), an automated technique—could be used together to form a more robust form of evaluation. Methods: We obtained reflective essays written by first year medical students as part of a clinical skills course. Faculty scored essays using a rubric designed to evaluate Integration, Depth, and Writing. The same essays were subjected to AWA analysis, which counted the number of reflective phrases indicative of Context, Challenge, or Change. Results: Faculty scored the essays uniformly high, indicating that most students met the standard for reflection as described by the rubric. AWA identified over 1400 instances of reflective behavior within the essays, and there was significant variability in how often different types of reflective phrases were used by individual students. Conclusions: While data from faculty assessment or AWA alone is sufficient to evaluate reflective essays, combining these methods offer a richer and more valuable understanding of the student’s reflection.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/157763