Can a reflective rubric be applied consistently with raters globally? A study across three countries.

Lucas, C; Smith, L; Lonie, JM; Hough, M; Rogers, K; Mantzourani, E

Can a reflective rubric be applied consistently with raters globally? A study across three countries.

Lucas, C

Smith, L Lonie, JM Hough, M

Rogers, K

Mantzourani, E

Permalink

Publisher:: Elsevier
Publication Type:: Journal Article
Citation:: Currents in Pharmacy Teaching and Learning, 2019, 11, (10), pp. 987-994
Issue Date:: 2019-10

Closed Access

	Filename	Description	Size
	1-s2.0-S1877129718302818-main.pdf	Published version	422.44 kB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Lucas, C https://orcid.org/0000-0002-7857-2338
dc.contributor.author	Smith, L
dc.contributor.author	Lonie, JM
dc.contributor.author	Hough, M https://orcid.org/0000-0001-5213-3751
dc.contributor.author	Rogers, K https://orcid.org/0000-0001-5497-4298
dc.contributor.author	Mantzourani, E
dc.date.accessioned	2020-04-29T22:05:15Z
dc.date.available	2019-06-20
dc.date.available	2020-04-29T22:05:15Z
dc.date.issued	2019-10
dc.identifier.citation	Currents in Pharmacy Teaching and Learning, 2019, 11, (10), pp. 987-994
dc.identifier.issn	1877-1297
dc.identifier.issn	1877-1300
dc.identifier.uri	http://hdl.handle.net/10453/140338
dc.description.abstract	INTRODUCTION:Reflection is a powerful tool for assisting students to develop the skills to make better informed decisions. As a pharmacy competency standard, reliable and fair assessment strategies are required to measure reflective skills and support students in developing their reflective capacity. The aim of this research was to explore whether we can extend the applicability of a previously tested rubric to a range of educational settings, to account for diversity of pharmacy educators and curricula internationally. METHODS:Four raters from three countries applied a reflective rubric to assess a sample (n = 43) of reflective accounts, representing 41% of a cohort of 105 second-year undergraduate pharmacy students. The interrater reliability (IRR) was measured utilizing the intra-class correlation coefficient (ICC), using a two-way random effects model with absolute agreement, to determine the level of agreement between the raters' absolute scores. Generalizability Theory analysis was used to estimate generalizability of raters and stages. RESULTS:Results indicated agreement of raters for (i) each of the seven stages of reflection and (ii) overall score for the reflective account, with moderate to substantial agreement (ICC = 0.55-0.69, p < 0.001); and high agreement for all raters for the overall score (ICC = 0.96, p < 0.001), respectively. The G-Study estimated a relative error coefficient of 0.78. CONCLUSION:This additional analysis further confirms the reliability and applicability of the rubric to a range of rater academic backgrounds.
dc.format	Print-Electronic
dc.language	eng
dc.publisher	Elsevier
dc.relation.ispartof	Currents in Pharmacy Teaching and Learning
dc.relation.isbasedon	10.1016/j.cptl.2019.06.004
dc.rights	info:eu-repo/semantics/restrictedAccess
dc.subject	1302 Curriculum and Pedagogy
dc.title	Can a reflective rubric be applied consistently with raters globally? A study across three countries.
dc.type	Journal Article
utslib.citation.volume	11
utslib.location.activity	United States
utslib.for	1302 Curriculum and Pedagogy
utslib.for	1302 Curriculum and Pedagogy
pubs.organisational-group	/University of Technology Sydney/Graduate School of Health
pubs.organisational-group	/University of Technology Sydney
utslib.copyright.status	closed_access	*
pubs.consider-herdc	true
dc.date.updated	2020-04-29T22:05:13Z
pubs.issue	10
pubs.publication-status	Published
pubs.volume	11
utslib.start-page	987
utslib.citation.issue	10

Abstract:

INTRODUCTION:Reflection is a powerful tool for assisting students to develop the skills to make better informed decisions. As a pharmacy competency standard, reliable and fair assessment strategies are required to measure reflective skills and support students in developing their reflective capacity. The aim of this research was to explore whether we can extend the applicability of a previously tested rubric to a range of educational settings, to account for diversity of pharmacy educators and curricula internationally. METHODS:Four raters from three countries applied a reflective rubric to assess a sample (n = 43) of reflective accounts, representing 41% of a cohort of 105 second-year undergraduate pharmacy students. The interrater reliability (IRR) was measured utilizing the intra-class correlation coefficient (ICC), using a two-way random effects model with absolute agreement, to determine the level of agreement between the raters' absolute scores. Generalizability Theory analysis was used to estimate generalizability of raters and stages. RESULTS:Results indicated agreement of raters for (i) each of the seven stages of reflection and (ii) overall score for the reflective account, with moderate to substantial agreement (ICC = 0.55-0.69, p < 0.001); and high agreement for all raters for the overall score (ICC = 0.96, p < 0.001), respectively. The G-Study estimated a relative error coefficient of 0.78. CONCLUSION:This additional analysis further confirms the reliability and applicability of the rubric to a range of rater academic backgrounds.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/140338