Comparison between 2000 and 2018 on the reporting of statistical significance and clinical relevance in physiotherapy clinical trials in six major physiotherapy journals: a meta-research design

Verhagen, A; Stubbs, PW; Mehta, P; Kennedy, D; Nasser, AM; Quel de Oliveira, C; Pate, JW; Skinner, IW; McCambridge, AB

Comparison between 2000 and 2018 on the reporting of statistical significance and clinical relevance in physiotherapy clinical trials in six major physiotherapy journals: a meta-research design

Verhagen, A

Stubbs, PW Mehta, P Kennedy, D Nasser, AM Quel de Oliveira, C Pate, JW Skinner, IW McCambridge, AB

Permalink

Publisher:: BMJ
Publication Type:: Journal Article
Citation:: BMJ Open, 2022, 12, (1), pp. e054875-e054875
Issue Date:: 2022-01

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download Published VersionAdobe PDF (378.79 kB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Verhagen, A https://orcid.org/0000-0002-6195-0128
dc.contributor.author	Stubbs, PW
dc.contributor.author	Mehta, P
dc.contributor.author	Kennedy, D
dc.contributor.author	Nasser, AM
dc.contributor.author	Quel de Oliveira, C
dc.contributor.author	Pate, JW
dc.contributor.author	Skinner, IW
dc.contributor.author	McCambridge, AB
dc.date.accessioned	2022-01-05T00:39:21Z
dc.date.available	2022-01-05T00:39:21Z
dc.date.issued	2022-01
dc.identifier.citation	BMJ Open, 2022, 12, (1), pp. e054875-e054875
dc.identifier.issn	2044-6055
dc.identifier.issn	2044-6055
dc.identifier.uri	http://hdl.handle.net/10453/152676
dc.description.abstract	<jats:sec><jats:title>Design</jats:title><jats:p>Meta-research.</jats:p></jats:sec><jats:sec><jats:title>Objective</jats:title><jats:p>To compare the prevalence of reporting p values, effect estimates and clinical relevance in physiotherapy randomised controlled trials (RCTs) published in the years 2000 and 2018.</jats:p></jats:sec><jats:sec><jats:title>Methods</jats:title><jats:p>We performed a meta-research study of physiotherapy RCTs obtained from six major physiotherapy peer-reviewed journals that were published in the years 2000 and 2018. We searched the databases Embase, Medline and PubMed in May 2019, and extracted data on the study characteristics and whether articles reported on statistical significance, effect estimates and confidence intervals for baseline, between-group, and within-group differences, and clinical relevance. Data were presented using descriptive statistics and inferences were made based on proportions. A 20% difference between 2000 and 2018 was regarded as a meaningful difference.</jats:p></jats:sec><jats:sec><jats:title>Results</jats:title><jats:p>We found 140 RCTs: 39 were published in 2000 and 101 in 2018. Overall, there was a high prevalence (>90%) of reporting p values for the main (between-group) analysis, with no difference between years. Statistical significance testing was frequently used for evaluating baseline differences, increasing from 28% in 2000 to 61.4% in 2018. The prevalence of reporting effect estimates, CIs and the mention of clinical relevance increased from 2000 to 2018 by 26.6%, 34% and 32.8% respectively. Despite an increase in use in 2018, over 40% of RCTs failed to report effect estimates, CIs and clinical relevance of results.</jats:p></jats:sec><jats:sec><jats:title>Conclusion</jats:title><jats:p>The prevalence of using p values remains high in physiotherapy research. Although the proportion of reporting effect estimates, CIs and clinical relevance is higher in 2018 compared to 2000, many publications still fail to report and interpret study findings in this way.</jats:p></jats:sec>
dc.language	en
dc.publisher	BMJ
dc.relation.ispartof	BMJ Open
dc.relation.isbasedon	10.1136/bmjopen-2021-054875
dc.rights	© Author(s) (or their employer(s)) 2021. Re- use permitted under CC BY- NC. No commercial re- use. See rights and permissions. Published by BMJ.
dc.rights	info:eu-repo/semantics/openAccess
dc.subject	1103 Clinical Sciences, 1117 Public Health and Health Services, 1199 Other Medical and Health Sciences
dc.title	Comparison between 2000 and 2018 on the reporting of statistical significance and clinical relevance in physiotherapy clinical trials in six major physiotherapy journals: a meta-research design
dc.type	Journal Article
utslib.citation.volume	12
utslib.for	1103 Clinical Sciences
utslib.for	1117 Public Health and Health Services
utslib.for	1199 Other Medical and Health Sciences
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Health
pubs.organisational-group	/University of Technology Sydney/Faculty of Health/Graduate School of Health
pubs.organisational-group	/University of Technology Sydney/Faculty of Health/Graduate School of Health/GSH.Physiotherapy
utslib.copyright.status	open_access	*
dc.date.updated	2022-01-05T00:39:20Z
pubs.issue	1
pubs.publication-status	Published
pubs.volume	12
utslib.citation.issue	1

Abstract:

DesignMeta-research.ObjectiveTo compare the prevalence of reporting p values, effect estimates and clinical relevance in physiotherapy randomised controlled trials (RCTs) published in the years 2000 and 2018.MethodsWe performed a meta-research study of physiotherapy RCTs obtained from six major physiotherapy peer-reviewed journals that were published in the years 2000 and 2018. We searched the databases Embase, Medline and PubMed in May 2019, and extracted data on the study characteristics and whether articles reported on statistical significance, effect estimates and confidence intervals for baseline, between-group, and within-group differences, and clinical relevance. Data were presented using descriptive statistics and inferences were made based on proportions. A 20% difference between 2000 and 2018 was regarded as a meaningful difference.ResultsWe found 140 RCTs: 39 were published in 2000 and 101 in 2018. Overall, there was a high prevalence (>90%) of reporting p values for the main (between-group) analysis, with no difference between years. Statistical significance testing was frequently used for evaluating baseline differences, increasing from 28% in 2000 to 61.4% in 2018. The prevalence of reporting effect estimates, CIs and the mention of clinical relevance increased from 2000 to 2018 by 26.6%, 34% and 32.8% respectively. Despite an increase in use in 2018, over 40% of RCTs failed to report effect estimates, CIs and clinical relevance of results.ConclusionThe prevalence of using p values remains high in physiotherapy research. Although the proportion of reporting effect estimates, CIs and clinical relevance is higher in 2018 compared to 2000, many publications still fail to report and interpret study findings in this way.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/152676