The Delirium Objective structured clinical examination Education (DOE) Pilot Study: implementation of the delirium OSCE education package within hospital education for registered healthcare professionals.

INTRODUCTION

Up to 60% of older people admitted to an acute healthcare setting experience delirium.^1–3 Despite the significance of delirium now being well recognised with the implementation of international best practice guidelines internationally^3–6; delirium remains undetected in more than a third of cases.⁷ Barriers to delirium detection and care include low levels of knowledge, lack of education and negative attitudes about delirium.^8,9 Evidence shows inadequate education leads to low levels of confidence, and stress and anxiety when providing delirium care.¹⁰ These barriers must be addressed to prevent the negative and critical consequences of undetected delirium.^11,12

The most effective delirium educational interventions use multi-modal approaches and involve practical sessions, such as simulation.¹³ One form of simulation is objective structured clinical examinations (OSCEs). OSCEs are an accepted form of assessment and learning in undergraduate healthcare degrees. However, registered healthcare practitioners are not often exposed to OSCEs in regular professional development education (reference by authors). The aim of the Delirium OSCE Education (DOE) pilot study was to evaluate the effects of a delirium educational intervention on self-perceived confidence and competence in delirium assessment tools and delirium knowledge; delirium knowledge score; clinical practice; planned practice change; and perceived usefulness of the education for registered healthcare practitioners.

METHODS

Design

Th DOE pilot study adopted a quasi-experimental design with qualitative follow-up.

Settings and participants

We recruited two metropolitan hospital sites in Australia. Site 1 was a 60-bed acute aged care ward in a public hospital, and Site 2 was a 30-bed acute medical ward, under the direction of a geriatrician, in a private hospital. A convenience sample of eligible participants were recruited at each site. Participants were eligible to participate in the study if they were a registered healthcare practitioner (enrolled nurse, registered nurse, medical doctor, occupational therapist, or physiotherapist) working in the participating sites.

Intervention

The interactive learning activities within the delirium educational intervention, The Delirium OSCE Education Package (Figure 1), were based on evidence from the Delirium Clinical Care Standard¹ and OSCE best practice guidelines.^14–16 The OSCE scenarios were designed in consultation with an expert panel of academics, delirium clinical experts and senior medical, nursing, and allied health staff to ensure they were representative of a typical presentation of delirium at the participating site. The Delirium OSCE Education Package, was delivered during each site’s professional development education time and was delivered over approximately two months.

Figure 1.The Delirium OSCE Education Package Overview

Each site recruited a senior registered nurse to lead the intervention implementation. The educational facilitators comprised senior aged care clinicians (senior registered nurses and doctors) who participated in a two-hour training session. Assistants in nursing (AIN) role-played the standardised patient (SP) and attended a 30-minute training session.

The Delirium OSCE Education Package

The educational intervention had two face-to-face components under clinical supervision:

1. Part One: interactive group face-to-face education session delivered by a senior aged care clinician, including a ‘take away’ delirium care booklet entitled ‘Delirium Care-what you need to know’¹⁷ (60 minutes).

2. Part Two: a one-on-one objective structured clinical examination (OSCE) conducted by a senior aged care clinician using an SP scenario role played by an AIN (30 minutes). The OSCE was divided into a five-minute pre-brief, 15-minute scenario, and a 10-minute debrief consisting of formative feedback and a seven-item OSCE rubric.

There were also two unsupervised options to complete an online learning activity involving:

2. Watching five filmed delirium care vignettes. The vignettes followed a patient experiencing delirium from admission to the emergency department, handover to the ward, interview with family, delirium screening, and discharge planning (total 15-minutes); and

3. A paper-based individual self-reflection exercise consisting of four questions to prompt participants to reflect on an experience with caring for a patient with delirium (15 minutes).

Data Collection

Survey

Survey data were collected at each site before Part One of The Delirium OSCE Education Package (T0) and six weeks post-completion of Part Two of the intervention (T1). No sections of the survey were mandatory for participants to complete. The survey consisted of three components: (i) demographic information, (ii) self-efficacy and, (iii) delirium knowledge.

1. Demographic information: seven questions asked participants about their gender, age, clinical role and educational level.

2. Self-efficacy: measured with four five-point Likert scale questions asked participants to rate their confidence and competence on (1) their ability to use the delirium assessment tool and (2) overall knowledge about delirium from one (strongly disagree) to five (strongly agree). The internal consistency of the four Likert scale questions (Cronbach’s α 0.90) revealed excellent reliability.

3. Delirium knowledge: measured using a 16-item multiple choice question (MCQ) delirium knowledge quiz. The quiz was developed by the ADHERe team¹⁷ using the ‘Delirium Clinical Care Standards’ published in Australia.¹

The six-week post-intervention survey (T1) had five additional questions asking participants about their experience and impact of the educational intervention. Two three-point Likert scale questions asked participants to rate the impact of The Delirium OSCE Education Package on their confidence and competence on a scale from one (low) to three (high). Three were open-ended questions about the impact of education.

OSCE and observations of delirium care in practice

Scores were derived from the seven-item OSCE marking rubric (maximum score of 15), with higher scores demonstrating a higher level of competency in best practice delirium care. The OSCE rubric was self-designed by the research team and were based on the OSCE scenarios co-designed with the healthcare practitioners. At T1, a practitioner evaluation of delirium care was conducted at Site Two using an observation of delirium care in practice tool developed by the research team. The tool was developed from the ‘Delirium Clinical Care Standards’ published in Australia⁴ and consultations with delirium experts. The tool had 10 items (maximum score of 12), with higher scores indicating a higher level of competency in best practice delirium care and translation of knowledge into clinical practice. The observer of clinical practice (author 4) had expertise in delirium care.

Ethical Considerations

We obtained Human Research Ethics approval before commencing recruitment and data collection (Approval No 2014/339). Individual implied consent was used for completion of surveys. As the education was conducted as part of usual professional development, implied consent was used; those who chose not to participate in the research could still choose to receive the education. Written informed consent to observe practice was obtained from participants.

Data Analysis

We used SPSS Statistics 28 for all analyses.¹⁸ Data were checked for completeness and data entry errors. Our first step involved checking the data for completeness and data entry errors. Baseline characteristics of participants and the impact of The Delirium OSCE Education Package on confidence and competence ratings were examined using descriptive statistics. To describe the variables, we used frequency, mean, and standard deviation. For evaluating the impact of the intervention on our self-perceptions of confidence and competence, we employed the Chi-squared test. We calculated a total score for the 16-item MCQ delirium knowledge quiz, assigning equal weighting to each answer and considering missing answers as incorrect. To compare the pre and post MCQ scores, we applied the Mann-Whitney U test. Percentage scores were calculated for the OSCE marking rubric and observations of delirium care in practice, and we paired participant data for these variables. We assessed the significance of the difference between the two scores using the Wilcoxon rank test. All statistical significance tests were two-tailed, with a threshold of p<0.05 for significance.

The first autor conducted conventional content analysis using an inductive approach as described by Hsieh & Shannon¹⁹ to explore the open-ended survey responses. The first author read and re-read the data to immerse and familiarise herself with the depth of the data, and to ascertain and identify common language and meaning. The focus was on understanding how the participant’s words were used to describe their planned delirium practice change and the perceived usefulness of the education. Exact words were highlighted from the text that appeared to capture key concepts and allowed for the creation of codes and themes.

RESULTS

A total of 81 participants completed the pre-education survey (T0), with 62 participants completing the educational intervention (completion rate 76.5%). Of the 62 intervention participants, 53 completed the 6-week post-survey (T1) (85.4% response rate). Demographic details of participants are reported in Table 1.

There were improvements in perceptions of confidence and competence in ability to use the delirium assessment tool and knowledge of delirium (Table 2). The MCQ scores improved from 13.6 (SD 1.7) at baseline to 14.6 (SD 1.1) six weeks post-intervention (p<0.001). Of the 53 participants who completed the T1 survey, 83.7% and 85.4% reported that the educational intervention greatly impacted their confidence and competence (respectively) to provide delirium care. None of the participants reported that educational intervention had a low impact on their confidence and competence.

At Site 2, 14 participants consented to an observation of practice. The mean OSCE score for these participants was 88.7% (SD 0.1), indicating a high level of competency in delirium care. The mean score for observing delirium care in practice was 81.5% (SD 0.1). There was no difference between the mean observation of delirium care in practice score and the mean OSCE score (p=0.07). This indicates that the high level of competency demonstrated and gained during the OSCE was translated and maintained into clinical practice.

Content Analysis

Thirty-six participants (67.9%) provided 77 free text responses to open-ended questions; ‘How do you plan to change your practice as a result of this education?’, ‘What do you consider worked best about this education and why?’, and "What do you consider could be improved about this education and how? These were summarised into the following three themes: (1) Assess for Delirium, (2) The usefulness of The Delirium OSCE Package and (3) A need for ongoing education on delirium care.

Assess for Delirium

A total of 72.2% (n=26) participants reported that following the educational intervention they would regularly assess and screen for delirium, due to a greater understanding of the seriousness of delirium, and the need for early detection and intervention (Table 3).

“Be vigilant about recognising patients at risk of becoming delirium. Act quickly when a patient appears delirious. Monitor more closely” (Participant 37).

Usefulness of The Delirium OSCE Education Package

72% (n=26) participants provided positive comments about the educational intervention. 14 out of the 26 participants (54%) specifically reported that the OSCE contributed to their learning and delirium care skills development. No negative comments were provided about the interactive group face-to-face session or the OSCE component (Table 3).

A need for ongoing education on delirium care

A total of 30.5% (n=11) of participants reported they wanted further and continued education on delirium care, with some asking for wider education of delirium care throughout the hospital (Table 3).

DISCUSSION

The DOE pilot study has demonstrated that The Delirium OSCE Education Package significantly increased the perceptions of confidence and competence in using delirium assessment tools and knowledge for registered healthcare practitioners. The participants also had a significant increase in delirium knowledge scores. The high level of competency in delirium care demonstrated and gained during the educational intervention were translated into clinical practice. The intervention positively impacted planned practice change, and the participants perceived the OSCE to be useful to their learning and delirium care skills development.

The baseline knowledge of participants in this study was high, though the reasons for this were unclear. Despite the high baseline delirium knowledge, self-efficacy ratings (perceptions of confidence and competence) for delirium assessment and knowledge were low before the education. This implies that knowledge is only one element in developing effective clinical practice. Self-efficacy plays a key role in how a person approaches goals, tasks, and challenges; those with high self-efficacy are more likely to view a challenging task as something to be mastered rather than avoided.^20,21 It is thought that increased self-efficacy is positively related to clinical competence.^20,22,23 The DOE pilot study demonstrated a significant improvement in self-efficacy and translation of delirium care skills into clinical practice. The transference of delirium knowledge into practice can potentially increase delirium detection rates.

Our results suggest that participants may have undervalued the importance of delirium assessment and screening before the educational intervention. This finding is similar to recent qualitative studies exploring barriers and facilitators to delirium screening that found clinical staff do not prioritise delirium screening,²⁴ and they are not convinced of the benefit of delirium screening tools.²⁵ Routine screening is the essential first step of delirium care. The educational intervention positively impacted planned practice change with participants recognising the importance of delirium detection using the validated delirium assessment tool, and early intervention. This impact could have a positive effect on delirium care.

The DOE pilot study contributes to the increasing evidence that OSCEs are helpful as an educational intervention, with the majority of participants reporting that the educational intervention had a high impact on their confidence and competence to provide delirium care. Similarly, a recent study found that medical students who participated in a group OSCE during clinical placement improved their delirium knowledge, communication and clinical reasoning skills compared to conventional education (reference by authors). This was the first study to implement a delirium OSCE intervention for registered healthcare practitioners across different hospital settings.

A strength of this study is the demonstration that the intervention was implemented across two different types of hospital settings, with healthcare staff willing to participate in the intervention and research. This is an important finding as the participants expressed a desire for ongoing delirium education to be embedded across the whole hospital. The recruitment of a senior registered nurse to facilitate the design and implementation of the intervention at each site likely contributed to the success of this pilot study. Facilitators employed at the site (site champions) played a fundamental role in advocating for practice change in the clinical setting and translating delirium knowledge into practice.^26,27 Furthermore, recruitment of onsite senior clinicians to facilitate educational delirium interventions was shown to be valuable in participant engagement.²⁸

The survey data were not paired, so it is unknown if more confident practitioners completed the post-survey. It is difficult to compare findings in this study to previous studies as no consistent tool has been used to measure delirium knowledge. As this was a pilot study, it is unknown if the intervention is superior to conventional methods of professional development education. Further research is needed to determine how the effectiveness of The Delirium OSCE Education Package compared to conventional methods of professional development education.

CONCLUSION

The Delirium OSCE Education Package was able to be implemented at two hospital sites demonstrating feasibly and acceptability. The DOE pilot study has generated evidence on further integrating The Delirium OSCE Education Package across contexts and acute healthcare settings. Following this study, an RCT has been designed using the tools and methods developed here to evaluate the effectiveness of The Delirium OSCE Education Package against conventional professional development education in aged care, medical and surgical wards across further hospital sites.

Funding

This work was supported by the Science, Medicine and Health Faculty at the University of Wollongong, Aged Care Department at St. George Hospital and APR Internship.

Corresponding Author

Amy Montgomery
School of Nursing, Faculty of Science, Medicine & Health, University of Wollongong
Northfields Ave, Wollongong, NSW, 2500
asaunder@uow.edu.au
@montgomery_amy3

Acknowledgements

Alera Bowden, Pamela McAllan, Carissa Saunderson, Janine Masso, Kristine Tobin, Olivia Paulik, Gillian Costello, Nicole Britten, Robyn Whittaker, Cherrie Barton and, the National Institute for Applied Statistics Research Australia (NIASRA). We would also like to thank all the participants and the participating sites. This works builds on a quality improvement initiative undertaken at Bulli Hospital, NSW, Australia and Agency of Clinical Innovation.

Conflict of Interests

There are no conflict of interests to declare

Author Contributions

All authors contributed to the conceptualisation of the study. AM, PS, AW & VT contributed to the methodology. AM & AW contributed to data curation. AM contributed to project administration and writing- original draft presentation. AM, AW & PS completed the formal analysis. PS, LH & VT provided supervision. All authors contributed to visualisation and writing, reviewing & editing.