Royal College of Surgeons in Ireland
An international modified Delphi process supported updating the web-based....pdf (448.82 kB)

An international modified Delphi process supported updating the web-based "right review" tool

Download (448.82 kB)
journal contribution
posted on 2024-05-30, 14:38 authored by Barbara ClyneBarbara Clyne, Melissa SharpMelissa Sharp, Michelle O’ Neill, Danielle Pollock, Rosarie Lynch, Krystle Amog, Mairin Ryan, Susan SmithSusan Smith, Kamal Mahtani, Andrew Booth, Christina Godfrey, Zachary Munn, Andrea C Tricco

Objectives: The proliferation of evidence synthesis methods makes it challenging for reviewers to select the ''right'' method. This study aimed to update the Right Review tool (a web-based decision support tool that guides users through a series of questions for recommending evidence synthesis methods) and establish a common set of questions for the synthesis of both quantitative and qualitative studies (

Study design and setting: A 2-round modified international electronic modified Delphi was conducted (2022) with researchers, health-care providers, patients, and policy makers. Panel members rated the importance/clarity of the Right Review tool's guiding questions, evidence synthesis type definitions and tool output. High agreement was defined as at least 70% agreement. Any items not reaching high agreement after round 2 were discussed by the international Project Steering Group.

Results: Twenty-four experts from 9 countries completed round 1, with 12 completing round 2. Of the 46 items presented in round 1, 21 reached high agreement. Twenty-seven items were presented in round 2, with 8 reaching high agreement. The Project Steering Group discussed items not reaching high agreement, including 8 guiding questions, 9 review definitions (predominantly related to qualitative synthesis), and 2 output items. Three items were removed entirely and the remaining 16 revised and edited and/or combined with existing items. The final tool comprises 42 items; 9 guiding questions, 25 evidence synthesis definitions and approaches, and 8 tool outputs.

Conclusion: The freely accessible Right Review tool supports choosing an appropriate review method. The design and clarity of this tool was enhanced by harnessing the Delphi technique to shape ongoing development. The updated tool is expected to be available in Quarter 1, 202


NHMRC investigator grant APP1195676

Tier 2 Canada Research Chair in Knowledge Synthesis

Evidence synthesis and translation of findings for national clinical guideline development: addressing the needs and preferences of guideline development groups

Health Research Board

Find out more...


Data Availability Statement

The authors do not have permission to share data


The original article is available at

Published Citation

Clyne B. et al. An international modified Delphi process supported updating the web-based "right review" tool. J Clin Epidemiol. 2024;170:111333.

Publication Date

22 March 2024

PubMed ID



  • Public Health and Epidemiology
  • General Practice
  • School of Population Health




  • Published Version (Version of Record)