Background
Delirium diagnosis is a common challenge in the general medical setting.1 Not only is delirium missed in most cases, but this omission also worsens outcomes.2 Criticisms of some of the commonly used delirium screening tools include the potential for re-test error and the requirement for extensive training.3 Many tools test single domains, such as attention, and fail to measure the full spectrum of DSM criteria.3 Technological solutions offer potential advantages in terms of the test construct. Examples include: (i) visual distractors to help evaluate sustained attention; (ii) randomised and rotating questions to help mitigate re-test error; and (iii) in app generated diagnosis to reduce user bias to final delirium status.4 Also, ergonomic design, adaptive features, and convenience can help lend user-friendly advantages to the uptake of mobile device-based screening tools.5
For these reasons, authors EE, OT, DF and and a multidisciplinary research group (Critical Care Research Group) have previously developed an electronic delirium screening tool [eDIS-ICU] that we successfully tested in the intensive care setting.6,7 eDIS-ICU incorporates user-centred, delirium-specific and technological advantages. Firstly, a large bank of potential questions is randomised after each test, thus avoiding re-test error. Secondly, eDIS-ICU is more dimensional than single attention tests as it targets all DSM criteria for delirium. Thirdly, adaptation to the delirium syndrome includes variable test frequency that reduces redundancy, test burden and potentially time to diagnosis.
Delirium presents very similarly, regardless of the place of onset. However, the characteristics of patients in the medical setting is distinct from that of the intensive care environment with more prevalent dementia and perceptual impairment.1,2 Here, we describe a process of redesigning eDIS-ICU, focusing on delirium screening within the medical setting.
Methods
We conducted a human-centred redesign, guided by: (i) the requirements of a screening tool and characteristics of a general medicine setting; (ii) the usability of DIS-ICU in its current format; (iii) implementing redesign based on user feedback (Figure 1). Human-centred design is a philosophy that directly involves users in designing a service or technology.8 Human-centred design approaches are often applied to redesign an existing service or technology.9 We conducted the redesign in February 2023 in a face-to-face setting at a metropolitan hospital, facilitated by one of the original eDIS-ICU co-designers (EE). An advanced trainee in medicine, physiotherapist, nurse, and nurse educator who all worked in a general medicine setting participated. A clinical researcher representative from the app development team (author OT), software developer and a patient consumer also participated. Four individuals were male, three were female. All clinicians had at least 5 years of postgraduate experience.
Redesign
Phase 1: Defining the Context and needs assessment. We brainstormed to: (i) establish the testing environment; and (ii) define the requirements of a delirium screening tool, both within an internal medicine setting.8,10 Affinity mapping was conducted to group information generated from brainstorming into related clusters, and we ordered ideas into themes.11
Phase 2: Usability inspection. We performed a pluralistic walkthrough, which involves end-users, software developers, and researchers stepping through interactions with a technology, focusing on and discussing its usability and identifying potential issues.12 These walkthroughs support a clearer understanding of user needs when working with the technology. In a pluralistic walkthrough, the reviewers progress through a simulation or run-through of the test under question and are invited to comment. Responses are captured and used to inform usability and potential areas for further development.
First, we provided a brief orientation to delirium and a standard approach to screening, courtesy of the Western Australia Clinical Training Network. Second, we presented an eDIS-ICU video comprising a run-through of delirium screening against different possible outcome scenarios (delirium absent, delirium present, delirium possible). The facilitator (EE) ran the simulation, stopping regularly for comments and questions.13 The walkthrough was voice recorded and later transcribed by the facilitator. A copy of DSM-5 delirium diagnostic criteria was made available to attendees for reference.14 Third, participants were asked to write down positive and negative attributes of eDIS-ICU.15 The facilitator collected these responses and grouped them into themes after discussion.
Phase 3: Implementing redesign. We identified actionable items based on the feedback generated from the pluralistic walkthrough. The development group (EE, OT and PW) used an iterative approach to decide on potential areas for change. Corresponding adaptations were made to eDIS-ICU. This modified version represented the prototype eDIS-MED.
Ethics approval was granted [HREC/2021/QPCH/74715].
Results
Phase 1 brainstorming generated eighty-seven statements. Context responses were clustered into patient factors (easier or more difficult to test), staff and ward characteristics (Appendix 1). Ideal features of a delirium screening tool were grouped into: diagnosis, links with management, functionality, and integration (Table 1).
Regarding functionality, participants identified that eDIS-ICU was short, portable, non-repetitive and adaptive in design, as highlighted by: [doctor] ‘eDIS-ICU asked fewer questions if the patient has delirium.’ By contrast, even though the number of questions was seen as an asset, the number of steps for each was noted as problematic: [doctor] ‘you have to click it three times before showing it to the patient.’ And [nurse] ‘yes, it’s very clicky.’
While there is an abort function embedded in eDIS-ICU if the patient is deemed too drowsy or agitated according to Richmond Agitation-Sedation Scale (RASS), this was still identified as an area of uncertain utility in medicine: [physiotherapist] ‘I have never seen RASS used in a medical ward.’16 Also, the functionality of eDIS-ICU was considered more research-focused than clinically useful: [physiotherapist] ‘in a practice partnership with nurses, how will we know that John in bed three hasn’t had the test ten times if it isn’t recorded somewhere?’. Only the delirium status was reported at the end of the screening; the lack of availability of the actual responses informing diagnosis was identified as a potential loss of valuable clinical data.
In terms of diagnosis, eDIS-ICU tested all the DSM criteria for delirium: [doctor] ‘it targets all the diagnostic domains affected in delirium.’ eDIS-ICU was felt to be appropriate for the target population: [doctor] ‘covers the majority of the general medical patient population.’
However, some groups were deemed at risk of being marginalised from testing with eDIS-ICU: [physiotherapist ‘what about visual problems and hemianopia?’. Suitability in the context of cultural diversity and limited education was another potential shortcoming, [consumer] ‘if you have someone who had no education or is from a rural background, then they will just stare at it.’
There was an impression that some of the questions were suggestible. In such an instance, an incorrect response may have arisen in the absence of knowledge rather than belief in the wrong response. For instance: [nurse educator] ‘if you had no idea where you are and saw a picture of an airport, then you would just answer yes.’ The importance of this distinction was debated: [physiotherapist] ‘it’s just an orientation question, so does it matter why it is answered incorrectly?’ Additionally, not all the questions were endorsed. Some were reported as being potentially [IT] ‘a bit confusing.’ It was suggested that the binary nature of submitted answers could be a possible confounder, [IT] ‘if they can’t answer the question then that’s it?’ Solutions to this potential problem were highlighted:[IT] ‘a skip function would work.’ (Figure 2).
Other future positive facets of eDIS-ICU included the potential ability for a simple translation into other languages for more general use. There were also general concerns around the feasibility of using an electronic device in the clinical setting. First, the availability of devices in the clinical setting was deemed problematic: [physiotherapist ‘there is a lack of access to iPads in the ward.’ Also, hygiene was expressed as a concern: [physiotherapist] 'personal phones in the clinical area are gross!’
The design team established the areas for actionable redevelopment and included and implemented focussed solutions in response to the findings of the pluralistic walkthrough and positive/ negative feedback.
Implementing Redesign
The following features were integrated into EDIS-ICU in response to the feedback, and the prototype eDIS-MED was created (Table 2).
-
Diagnosis: a skip function has been integrated so that an alternative is presented if a patient cannot respond to a question.
-
Functionality: The number of user steps per question was reduced.
-
Management: An introductory slide to affirm intended use and raise awareness of the abort function in case of agitation/drowsiness.
-
Integration: a process for making individual responses more readily available is underway.
Discussion
The results showed that eDIS-ICU was easy to use, had adaptive design features, tested the pertinent domains affected in delirium and potentially provided point-of-care diagnostics. Actionable areas for change were identified and implemented. The redesign process created the next prototype, eDIS-MED, and further validation work is planned.
Compared with other electronic delirium screening tools, the proposed skip function has also been successfully introduced into the mobile app for delirium screening.17 Additionally, DelApp, has built-in instructions, scoring criteria, and is designed for use by non-experts.4 Unlike eDIS-MED, an inability to answer questions is determined as incorrect, according to DelApp.
There are several limitations to our study. Firstly, the individuals in our pluralistic walkthrough did not have the opportunity to interact with eDIS-ICU directly and could only view the functionalities remotely. Despite this, respondent feedback clearly identified functionality issues. Further, the nature of a pluralistic walkthrough is not based on direct and independent interaction as it is not a ‘test’ of usability. Instead, it is a collaborative inquiry into usability to inform further development/ improvement of a technology.
Before clinical studies, eDIS-MED will be subject to further content evaluation against an expert group to assess further the issue of suggestibility and possible confusion in some of the questions. It should be emphasised that eDIS-MED is intended for use as a screening tool prioritising brevity and high-performance sensitivity. As in other studies, concerns about the general use of mobile devices in the clinical setting were raised.17 In order to mitigate hygiene concerns, eDIS-ICU/eDIS-MED has been designed for contactless use with the patient.
A collaborative human-centred redesign process has created a mobile delirium electronic screening tool (eDIS-MED) for use in the general medical setting. Once adequately validated, this has the potential to bring a user-friendly, adaptive and dimensional delirium screening tool to assist frontline clinicians in making an accurate and timely diagnosis of delirium.
Conflicts of interest
All authors declare that they have no conflicts of interest.
Acknowledgements
We would like to thank research assistant Anne Bucetti, The Critical Care Research Group, Advancing Queensland Fellowship and the multidiciplinary panel of consumer/ experts