A comparison of four quasi-experimental methods: an analysis of the introduction of activity-based funding in Ireland
Background: Health services research often relies on quasi-experimental study designs in the estimation of treatment effects of a policy change or an intervention. The aim of this study is to compare some of the commonly used non-experimental methods in estimating intervention effects, and to highlight their relative strengths and weaknesses. We estimate the effects of Activity-Based Funding, a hospital financing reform of Irish public hospitals, introduced in 2016.
Methods: We estimate and compare four analytical methods: Interrupted time series analysis, Difference-in-Differences, Propensity Score Matching Difference-in-Differences and the Synthetic Control method. Specifically, we focus on the comparison between the control-treatment methods and the non-control-treatment approach, interrupted time series analysis. Our empirical example evaluated the length of stay impact post hip replacement surgery, following the introduction of Activity-Based Funding in Ireland. We also contribute to the very limited research reporting the impacts of Activity-Based-Funding within the Irish context.
Results: Interrupted time-series analysis produced statistically significant results different in interpretation, while the Difference-in-Differences, Propensity Score Matching Difference-in-Differences and Synthetic Control methods incorporating control groups, suggested no statistically significant intervention effect, on patient length of stay.
Conclusion: Our analysis confirms that different analytical methods for estimating intervention effects provide different assessments of the intervention effects. It is crucial that researchers employ appropriate designs which incorporate a counterfactual framework. Such methods tend to be more robust and provide a stronger basis for evidence-based policy-making.
Health Research Board SPHeRE-2018-1
CommentsThe original article is available at https://bmchealthservres.biomedcentral.com/
Published CitationValentelyte G, Keegan C, Sorensen J. A comparison of four quasi-experimental methods: an analysis of the introduction of activity-based funding in Ireland. BMC Health Serv Res. 2022;22(1):1311.
Publication Date3 November 2022
- Health Outcomes Research Centre
- Published Version (Version of Record)