The effectiveness of debriefing in simulation-based... : JBI Evidence Synthesis (2024)

Primary Reviewer

Prof. Tracy Levett-Jones, RN, PhD, MEd & Work, BN, DipAppSc (Nursing)¹

Secondary Reviewer

Samuel Lapkin (PhD Candidate), RN, BN (Hons)¹

Email for correspondence: [emailprotected]

¹The University of Newcastle, Faculty of Health, School of Nursing and Midwifery, Australia and University of Newcastle Evidence Synthesis Group: a collaborating centre of the Joanna Briggs Institute

Review question/objective

This objective of this review is to systematically examine the evidence to answer the following question:

What is the effectiveness of debriefing as it relates to simulation-based learning for health professionals?

Background

Simulation is defined as a technique used to “ replace or amplify real experiences with guided experiences that evoke or replace substantial aspects of the real world in a fully interactive manner ”. ^1(p126) The use of simulation for educational purposes began decades ago with the use of low-fidelity simulations ² and has evolved at an unprecedented pace. Debriefing is considered by many to be an integral and critical part of the simulation process. ^3-5 However, different debriefing approaches have developed with little objective evidence of their effectiveness.

Some suggest that a structured debriefing should occur immediately after simulation; ^6-8 other researchers advocate that debriefing should include reflection on and for practice. ^{9, 10} Opportunities for formative feedback and self-evaluation are claimed to be essential components of debriefing; ¹¹ and the use of video recordings of the simulation during are said to enhance debriefing sessions by stimulating learning and discussion based on an accurate account of events. ¹² Depending on the simulation objectives, opportunities for discussion of students' non-technical skills such as clinical reasoning, situation awareness, communication, leadership and teamwork skills are also considered important to debriefing.

While simulation-based learning and debriefing have been adopted and used extensively in health care the use of the term ‘debriefing’ originated in the army. ¹³ Colonel Marshall, a United States (US) Army historian from World War II and the Korean and Vietnam wars is attributed with developing debriefing methods. ¹⁴ He developed ‘group historical debriefing’ as a method of conducting interviews with surviving soldiers from warfare. These interviews were conducted on the battlefield soon after the combat had ceased and involved all ranks; the emphasis was on learning from the experience. ¹⁵ Other debriefing techniques currently used in the military such as after-action reviews are based on Marshall's approaches. After-action reviews are debriefings conducted as part of military training exercises and involve immediate feedback on training proficiency. ¹⁶ These examples of military debriefings are educational and do not usually involve counselling or therapy. ¹⁷ The debriefings are aimed at improving combat performance by reflective learning and developing new military tactics as a result of the experience. ¹⁸

Debriefing has also been adopted by the airline industry in response to aviation incidents. Analysis of 35,000 National Aeronautics and Space Administration (NASA) Aviation Safety Reporting System reports between 1976-1983 indicated that nontechnical and communications skills rather than technical flying abilities or aircraft mechanical malfunctions attributed to most aviation incidents. ¹⁹ For example, a report by the National Transportation Safety Board concluded that the captain's failure to accept input from junior crew members and the flight engineer's lack of assertiveness contributed to a United Airlines crash in 1978. ²⁰ In response to similar findings from other accidents the aviation industry developed training programs called Crew Resource Management (CRM) in the 1970s. ²¹ These programs typically involve a range of knowledge, skills and attitudes including communications, situational awareness, problem solving, decision making, and teamwork. ²² In addition, components of the CRM training include simulated flights scenarios. At the end of the simulated flights scenario the flight instructor facilitates a debriefing. The crew critically analyses performance during the simulated flight scenarios to reinforce newly improved skills. ²³

Debriefing has also been used in experimental psychology in research involving deception (the purposeful provision of ambiguous details about the research and procedures when it is thought that truthful disclosure to participants may influence the phenomena under investigation). ²⁴ Although commonly used in psychological and neuroscience research the use of deception remains ethically controversial ²⁵ and for this reason debriefing is used to reverse any adverse effects on participants from the experience. During the debriefing participants who have been ‘deceived’ as a part of the study are informed of the true nature of the experiment. Another common example of the use of debriefing in psychology is the critical incident debriefing developed as a structured therapeutic approach to mitigate acute post-crisis psychological symptoms. ²⁶

While the debriefing approaches outlined above vary in terms of process and terminology, they each include structured and purposive discussions about prior experiences. However, in health care this discussion is aimed at facilitating learning to enhance future performance and ultimately improving patient outcomes. This is achieved, in part, by providing an opportunity to clarify the learner's knowledge and rationale for actions during the simulation experience. ¹⁸ Debriefing is considered critical to experiential learning as it encourages the learner to reflect on their performance and construct meaning from that experience with a view to clinical improvement. ²⁷ In a recent systematic review of high-fidelity simulation literature Issenberg, McGaghie, Petrusa, Gordon, and Scalese ⁴ reported that 51 studies listed educational feedback during debriefing as the single most important feature of simulation-based medical education. However, the effectiveness of the debriefing process may potentially be compromised by behaviours such as the use of ridicule, focusing on errors and non-constructive criticism. ⁸

Although debriefing following the simulation experience (post-simulation debriefing) is common practice with effectiveness a taken for granted assumption, there is little empirical evidence to support this approach. Additionally, differences in learning outcomes and effectiveness in relation to other types of debriefing are unclear. Some studies have examined pre-briefing during which the facilitator explains the purpose of the simulation and any learning objectives before the simulation experience ²⁸ whilst others have investigated the use of debriefing during the simulation experience (in-simulation debriefing). ²⁹ Another common approach is the use of reflective journals as an alternative or supplementary method to oral debriefing. ³⁰

There are conflicting views regarding the ideal length of debriefing with some proposing it should typically be three times longer than the length of the scenario ³ and others limiting it to 10 minutes after a 45 minutes simulation. ⁷ There is also uncertainly about the ideal number of participants in debriefing and who should be involved ³¹ with one study claiming that four participants per debrief is appropriate. ³²

The issues highlighted here, along with the limited number of empirical studies, illustrate the gaps that currently exist in relation to the effectiveness of debriefing in simulation-based learning. A search Cochrane Database of Systematic Reviews and JBI did not identify any systematically review focusing on simulation debriefing. This is an important finding given the assumption that the purpose of debriefing is to facilitate learning. This presents an opportunity for systematically searching, synthesising and summarising the best available evidence on the effectiveness of debriefing in simulation-based learning.

Review question/objective

Objective

The aim of this review is to appraise and synthesise the best available evidence based on primary studies comparing debriefing to no debriefing or different types of debriefing as it relates to simulation-based learning for health professionals

Inclusion criteria

Types of participants

The review will consider studies that include any health professional participants involved in debriefing as a part of simulation

Types of intervention(s)/phenomena of interest

The review will consider studies that include all types of debriefing for the purpose of simulation based learning

Types of outcome

Any objectively measured outcomes related to debriefing conducted as part of simulation-based learning will be considered. Examples include clinical reasoning, situation awareness, communication skills, teamwork, knowledge acquisition, and performance of psychom*otor skills.

Type of studies

The systematic review will primarily consider randomised controlled trials (RCTs); however in the absence of RCTs, other research designs, such as non-randomised controlled trials and before and after studies, will be considered.

Search Strategy

The search strategy aims to find both published and unpublished studies, limited to the English language. All databases will be searched from inception to current date (2011) using a three-step search strategy. Initially a limited scoping search of MEDLINE and Proquest databases will be undertaken followed by an analysis of the text words contained in the title and abstract, and of the index terms used to describe each article retrieved. Initial terms to be used are:

• Debriefing
• Simulation
• Health professional

The second step will involve searching electronic databases using several combinations and permutations of key words and index terms identified by the initial literature scoping. Using a defined search and retrieval method, the databases to be searched are:

1. AMED
2. CINAHL
3. Cochrane Central Register of Controlled Trials (CENTRAL)
4. Dissertation and Theses
5. EMBASE
6. ERIC
7. Journals@Ovid
8. MEDLINE
9. ProQuest Nursing Journals
10. PsycINFO

The following will be hand searched to find any additional articles:

• Mednar
• Directory of open access journals
• Conference Proceedings

Lastly, reference lists of all included literature will be searched for any additional relevant studies. The bibliographical software package Endnote^TM will be utilised to manage all references as it facilitates the importation of references from electronic databases as well as the linkage of references into the Joanna Briggs Institute (JBI) Comprehensive Review Management System (CReMS^TM) for assessment of methodological quality using the JBI critical appraisal tools. These guidelines have been developed to minimise bias and establish validity of the findings.

Assessment of methodological quality

Quantitative papers selected for retrieval will be assessed by two independent reviewers for methodological validity prior to inclusion in the review using standardised critical appraisal instruments from the Joanna Briggs Institute Meta Analysis of Statistics Assessment and Review Instrument (JBI-MAStARI) (Appendix I). Any disagreements that arise between the reviewers will be resolved through discussion, or with a third reviewer.

Data collection

Quantitative data will be extracted from papers included in the review using the standardised data extraction tool from JBI-MAStARI (Appendix II). The data extracted will include specific details about the interventions, populations, study methods and outcomes of significance to the review question and specific objectives.

Data Synthesis

Quantitative papers will, where possible be pooled in statistical meta-analysis using JBI-MAStARI. All results will be subject to double data entry. Effect sizes expressed as odds ratio (for categorical data) and weighted mean differences (for continuous data) and their 95% confidence intervals will be calculated for analysis. Heterogeneity will be assessed statistically using the standard Chi-square and also explored using subgroup analyses based on the different quantitative study designs included in this review. Where statistical pooling is not possible the findings will be presented in narrative form including tables and figures to aid in data presentation where appropriate.

Conflicts of interest

Nil

References

Appendix I: Joanna Briggs Institute Critical Appraisal Tools

Appendix II: Joanna Briggs Institute Data Extraction Tool