Bias, coverage, and asymptotic behaviour of random effects meta-analysis: a clinically driven simulation study.

Johnson, K; Hayen, A; Lassere, MND; Mengersen, K

Bias, coverage, and asymptotic behaviour of random effects meta-analysis: a clinically driven simulation study.

Johnson, K Hayen, A

Lassere, MND Mengersen, K

Permalink

Publication Type:: Journal Article
Citation:: JBI evidence implementation, 2020, 18, (4), pp. 355-367
Issue Date:: 2020-12

Closed Access

	Filename	Description	Size
	02205615-202012000-00002.pdf	Published Version	35.5 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Johnson, K
dc.contributor.author	Hayen, A https://orcid.org/0000-0003-4046-8030
dc.contributor.author	Lassere, MND
dc.contributor.author	Mengersen, K
dc.date.accessioned	2021-03-31T09:03:31Z
dc.date.available	2021-03-31T09:03:31Z
dc.date.issued	2020-12
dc.identifier.citation	JBI evidence implementation, 2020, 18, (4), pp. 355-367
dc.identifier.issn	2691-3321
dc.identifier.uri	http://hdl.handle.net/10453/147730
dc.description.abstract	<h4>Aims</h4>The study of foundational features of meta-analysis is incomplete and continues to remain important. Using simulations we study bias and coverage and the asymptotic behaviour of the DerSimonian and Laird (D&L) meta-analysis with varying trial numbers and sizes, levels of risk, and extent of treatment effects.<h4>Methods</h4>With simulated data we model risk of untoward events in randomized controlled trials in meta-analyses. Treatment effect is expressed as relative risk reduction, with effect size estimated by the odds ratio which is then compared with the known population odds ratio. Performance is measured as bias, standardized bias and coverage, with thresholds for desirable results being prespecified.<h4>Results</h4>Bias, standardized bias, and coverage varied substantially across meta-analyses of different trial size and number, risk mean and distribution, and relative risk reduction. Although improvements were observed with increasing trial size and number, there was widespread lack of satisfactory performance. Performance using normal risk distributions was worse compared with performance using constant or narrow uniform risk distributions. Asymptotic behaviour using very large trial numbers failed to show bias that appeared to approach zero for any distribution.<h4>Conclusion</h4>The D&L random effects meta-analysis method performed modestly at best. We were unable to demonstrate asymptotic normality. These results question the validity of the random effects method. The findings need replication and extension, which, if confirmed, would warn against generic use of the D&L method.
dc.format	Print
dc.language	eng
dc.relation.ispartof	JBI evidence implementation
dc.relation.isbasedon	10.1097/xeb.0000000000000227
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Bias, coverage, and asymptotic behaviour of random effects meta-analysis: a clinically driven simulation study.
dc.type	Journal Article
utslib.citation.volume	18
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Health
pubs.organisational-group	/University of Technology Sydney/Faculty of Health/Public Health
utslib.copyright.status	closed_access	*
dc.date.updated	2021-03-31T09:01:45Z
pubs.issue	4
pubs.publication-status	Published
pubs.volume	18
utslib.citation.issue	4

Abstract:

Aims

The study of foundational features of meta-analysis is incomplete and continues to remain important. Using simulations we study bias and coverage and the asymptotic behaviour of the DerSimonian and Laird (D&L) meta-analysis with varying trial numbers and sizes, levels of risk, and extent of treatment effects.

Methods

With simulated data we model risk of untoward events in randomized controlled trials in meta-analyses. Treatment effect is expressed as relative risk reduction, with effect size estimated by the odds ratio which is then compared with the known population odds ratio. Performance is measured as bias, standardized bias and coverage, with thresholds for desirable results being prespecified.

Results

Bias, standardized bias, and coverage varied substantially across meta-analyses of different trial size and number, risk mean and distribution, and relative risk reduction. Although improvements were observed with increasing trial size and number, there was widespread lack of satisfactory performance. Performance using normal risk distributions was worse compared with performance using constant or narrow uniform risk distributions. Asymptotic behaviour using very large trial numbers failed to show bias that appeared to approach zero for any distribution.

Conclusion

The D&L random effects meta-analysis method performed modestly at best. We were unable to demonstrate asymptotic normality. These results question the validity of the random effects method. The findings need replication and extension, which, if confirmed, would warn against generic use of the D&L method.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/147730