Background

Quality improvement (QI) projects aim to implement evidence-based knowledge to ensure best nursing care.1,2 Stakeholders often use a plan-do-check-act cycle to allow specific planning, performing, communication and evaluation of each step.3 Evaluation can use a before / after approach.4 The process requires management skills, especially regarding implementation in an interprofessional and/or interdisciplinary team.5 The COVID pandemic influenced several ongoing studies and projects, leading to delay, modification, and cancellation.6–12 Affected projects needed to be balanced between less efficacy due to the pandemic and the urgent need for improving practice to protect vulnerable patients.

Patients with a stroke are most often cared for on specialised SUs, which offer a high level of care to initiate first therapies and reduce complications.13 Delirium is among these complications, affecting around 40% of patients.14–16 Since there is a dose-response relationship between duration of delirium and adverse outcomes such as prolonged stay in hospital, cognitive impairment or increased mortality, implementation of delirium management is recommended to reduce the negative impact of delirium.17–20 Several studies have addressed barriers and implemented delirium QI using a before/after comparison.4,21–25 We performed one such QI project to implement interprofessional delirium management on four affiliated Stroke Units (SU) in Germany.26

Since pandemics and other events occur unforeseen and can lead to several challenges in running scientific trials, we aimed to evaluate data quality in a QI project in study centres with vs. without pandemic-related disruption, report unplanned study modifications and regulatory approvals, and overall lessons learned.

Design and methods

This is a secondary analysis of a previously published, investigator-initiated, non-funded, nurse-led, observational, prospective QI project.26 The original study was planned before the pandemic and started after the first wave in autumn 2020. The local ethics committee (D459/20) approved and registered the original QI project in the German Register of Clinical Studies (DRKS00021436). The report of this secondary analysis is in concordance with Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement (Supplement).27

Setting

The design is reported in detail elsewhere.26 In brief, we conducted a QI project in four affiliated SUs to implement an interprofessional protocol for delirium management and evaluated it in a before/after comparison. The primary outcome was the median delirium severity over admission, assessed with the Nursing Delirium Screening Scale (Nu-DESC). The Nu-DESC is a delirium assessment tool including five dimensions with 0 to 2 points, and a total score ranging from 0 to 10 points; with a score ≥ 2 indicates delirium.28 Nurses screened 3x/24h all eligible patients for delirium; for positive results, physicians confirmed delirium using the Diagnostic and Statistical Manual of Mental Disorders fifth edition (DSM-5) criteria. Secondary outcomes were the presence and duration of delirium, length of stay in SU and hospital, and modified Rankin Scale (mRS). There were four phases: a) two weeks of interprofessional teaching of delirium assessment; b) four weeks pre-phase (phase 1) to measure the baseline of outcome parameters; c) four weeks of interprofessional teaching of staff in delirium management; and d) four weeks post-phase (phase 2) to measure the effect of delirium management after implementation of delirium management. Delirium management included the cooperation of physicians and nurses, evaluating possible reasons for delirium and the choice of appropriate, specific treatments, non-pharmacological interventions such as mobilisation, patient information, re-orientation, patient distress, administration of specific pharmacological interventions.26 All data were collected by local study coordinators, using standardised data collection forms. The QI project included nearly 500 patients, with 9.5% being having delirium.26

Sample

This secondary analysis included 14 participating local study coordinators in four centres. All centres were affected by the pandemic, including having patients with COVID-19 and stroke, re-structuring processes, extended working hours, visitor restrictions, and changes in staffing numbers. Coordinators were active clinicians, working as nurses, physicians, or students on SU and/or research in participating hospitals and related universities. Before the study began, all coordinators were asked to record all unplanned modifications related to the study. As mentioned above, two centres could perform the project as planned with no delay in all phases, and two centres had a delayed start and delivery.

Outcome parameters

The primary outcome was data quality of centres with vs. without delay in study delivery, measured as the rate of missing information in primary and secondary outcome parameters. The secondary outcomes were unplanned modifications as qualitative results, identified by clustering of observed themes. We defined an unplanned modification as an unforeseen modification resulting from a modifiable cause avoidable in future recurrence by reasonable adaption.29 Hence, we structured unplanned modifications in four different clusters: a) fatal modifications which would lead to ending the study in case of unexpected increased mortality, severe violation of ethical approval, or other reasons; b) serious modifications, requiring a revision of the registration and/or ethic approval, e.g. new or withdrawing study centres, revision of primary or secondary outcome parameters, or revision of other pre-registered information; c) minor modifications, requiring a revision of the study management, e.g. delayed study phases, adaption of information sheets due to local needs, modification of teachings from presence- into virtual-mode; and d) other modifications, e.g. revisions for improving study performance, and others.

Information was collected prospectively by participating study coordinators by a standardised data collection form, and supported by personal talks, real and virtual meetings, emails, and recorded timelines in reaching milestones.

Statistical methods

We conducted a descriptive analysis, using nominal data in absolute and relative frequency, and continuous data in median and interquartile range (IQR) or mean and standard deviation (SD), depending on the distribution. Inferential tests were conducted using Chi-squared, t-test, and Mann-Whitney-U tests. The level of significance was defined as alpha=0.05. The analysis was carried out with SPSS 23 (IBM, New York).

Results

All centres were able to implement delirium management in 2020 and 2021. Participating centres planned study initiation over different time frames, ranging from August to October 2020; but in November, the second wave started in Germany and led to several changes, such as reduced admission rates, other tasks and responsibilities for local study coordinators, especially participating nurses, restricted visiting policies, and others.30–33 These unplanned disturbances delayed some participating study centres (Figure 1).

Figure 1
Figure 1.Delay in performing the study due to pandemic

Data quality

The four study centres recruited 359 patients. Compared with planned recruitment, the rate of recruited patients was one-quarter lower than expected in all centres (Table 1). Data quality in delayed centres was significantly poorer than in centres without delayed study delivery. Missing primary outcome led to the exclusion of 28.8% (n=55) of recruited patients in centres with delays, compared with 0.3% (n=1) in centres with study conduct as planned (p<0.001). In secondary outcomes, data quality of length of stay in hospital, discharge destination, and admission diagnosis were significantly different (Table 1).

Table 1.Comparison of centres with/out delayed performance
Item Total
(n=4)		 Centres with performance as planned (n=2) Centres with delayed performance (n=2) p
Patient recruitment
Planned 492 301 191 0.5322
Actual 359 (73) 223 (74.1) 136 (71.2)
Data completeness
Primary Outcome missinga 56 (15.6) 1 (0.3) 55 (28.8) 0.0001
Secondary Outcome missingb
Length of stay in SU 1 (0.3) 0 (0) 1 (0.7) 0.379
Length of stay in hospital 25 (7) 2 (0.9) 23 (16.9) 0.0001
Discharge destination 19 (5.3) 7 (3.1) 12 (8.8) 0.0273
mRS at discharge from SU 49 (13.6) 12 (5.4) 37 (27.2) 0.0001
Missing patient data
Age 8 (2.2) 2 (0.9) 6 (4.4) 0.0574
Gender 29 (8.1) 14 (6.3) 15 (11) 0.1149
Admission diagnosis 16 (4.5) 5 (2.2) 11 (8.1) 0.0152
Delirious patientsc 30 (100) 20 (66.7) 10 (33.3) 0.844
Missing data in nursing/therapeutical interventions 2 (6.7) 0 (0) 2 (20) 0.143

Abbreviations: mRS modified Rankin Scale; SU Stroke Unit
a Primary outcome was delirium severity, measured by Nursing Delirium Scale; result is based on planned number of patients (patients without primary outcome were excluded), n=492 in total
b Based on number of included patients, n=359 in total
c Based on n=30 delirious patients in total. No data were available for number of anticholinergic drugs, complications, and pharmacological interventions

Modifications

There were no fatal modifications which would have led to ending the study in case of unexpected increased mortality, a major violation of ethical approval, or other reasons.

There were two serious modifications: a) one exchange of a study centre (before recruiting patients, one centre withdrew from the study due to the pandemic and was replaced by another centre); b) two outcome measures were added in two centres: polymorphic delta waves in electroencephalography and changes in cholinesterase samples. These modifications required a change in registration and ethical approval.

Minor modifications happened in six cases, requiring a revision of the study management, e.g., delayed study phases, adaptation of information sheets due to local needs, modification of teachings from presence into virtual mode, and others. Other modifications were short-term changes to virtual meetings due to new parental responsibilities for the local study coordinators (Table 2). Not all modifications were clearly related to the pandemic and might have happened anyway.

Table 2.Modifications in Study Design
Modifications Adaptions Lessons learned
0 Fatal modificationsa
None None required None
2 Serious modificationsb
Study centre declined Recruitment of another centre Withdrawal of participating centres can always happen, better to have more centres than needed to ensure sufficient power (calculate 25% patient than expected)
During pandemic, it is important to motivate centres, allow delay, stay in contact, enable frequent meetings
Adding new outcome parameters Electroencephalography and cholinesterase added in two centres with amendment for ethic approval and registration Consider additional questions during conduction, which must be suit into the frame of ethic approval
 
6 Minor modificationsc
Delayed pre/post phases, due to Covid-19 Two centres had a delayed start, requiring re-training and updates in team teachings. Mutual information of participating centres and study coordination
Refresh the knowledge about delirium and how to screen and treat before taking up the study again
Different work patterns due to pandemic Clinicians had to work at the bedside, leading to postponed research, delegated tasks to uninformed persons, or after-work-research To increase data quality, it would be better to interrupt the study and continue when life is normal again
Change from presence- to virtual-teachings Hygienic rules hampered team teachings, less clinicians participated in meetings, and virtual meetings and teaching materials were developed We recommend in trials to use virtual team meetings and trainings, and provision of digital training materials.
Adaption of information sheet for physicians Physicians had an overload of information at their office’ pinboards, and the information sheet had to be revised to be perceived as important Information should be adapted to local structures and perceptions
Active involvement of physicians Despite team training, physicians did not react to nurses screening results. Hence, a critical examination during morning rounds by leading physicians was used to improve adherence. Information about every professions’ role and responsibilities must be communicated and repeated, e.g. in rounds. Involvement of key opinion leaders is crucial
Interprofessionality Delirium management and the study became a lower priority due to the pandemic, with the risk of low-quality data collection Find leaders in every profession and shift, involve as much people as possible to have always one who can answer questions and remind the staff to follow the study.
Be very clear about which profession has which tasks to fulfil and that communication between the professions is absolutely necessary.
1 Other modificationsd
Babies born by local study coordinators Our congratulations! This did not lead to serious delay or reduced performance, but parents were overworked. Every local study coordinator should have a second one at her/his side

Abbreviations:
a fatal modifications which would lead to ending the study
b serious modifications, requiring a revision of the registration and/or ethic approval
c minor modifications, requiring a revision of the study management
d other modifications, e.g. revisions for improving study performance etc.

Discussion

In this secondary analysis of a QI project for implementing delirium management on four SU, the reported data quality from centres without delay due to the pandemic was significantly better, compared to centres with delayed start and conduction. Several regulatory approval amendments were needed on several levels, mostly due to pandemic conditions.

Involved clinicians reported anecdotally higher workload, up to 16h per day, other tasks such as caring for Covid-19 patients instead of doing research, working on other units, handling isolated patients and personal protection equipment, visiting restrictions and virtual communication while asking for consent, and other challenges. The key lessons learned were:

  1. Communication: Communication with all involved study coordinators is crucial to staying in contact and sharing information about challenges and facilitators. Video conferences for all were difficult due to limited time. Better were personal calls or WhatsApp messages.

  2. Data quality: Allowing for a delay in study delivery should be feasible, due to higher workload, changed local policies, and others. More staff would have been helpful to increase data quality, but this was impossible in pandemics due to workforce diversion. A delay after the pandemic or wave would result in better data;

  3. Recruitment: A pandemic may lead to lower recruitment for several reasons, e.g., lower admittance rate of patients, lower consent due to absent legal representatives, less time for talks about the study, and others. Hence, study duration should be extended, ongoing evaluations and feedback should be implemented, and a scoring for included patients should be communicated.

  4. Education: the pandemic led to changes from educational meetings in small rooms and bedside teachings on the units to virtual courses and the development of teaching videos. Meanwhile, virtual meetings seem to have several advantages, and the development of teaching videos and evaluation forms has become more feasible, especially in a non-funded study. Nevertheless, bedside teaching and interprofessional cooperation cannot be wholly replaced in real world settings. This required sustained engagement from key opinion leaders in every profession in each centre;

  5. Reflection: During the pandemic, clinicians had to cover more and different tasks and responsibilities. It became difficult to monitor all ongoing projects and tasks. Critical reflection within the teams about study progress, related tasks and responsibilities seem to be important, including talks for reflection, supervision, and emotional relaxation. A delay in study performance should be offered to prevent burn-out risks.

In summary, lessons were related to adapted communication and education, attention to data quality and recruitment, and ongoing reflection.

During this non-funded study, several revisions were required. Adaption of studies to ongoing processes is quite usual, e.g. change of outcome parameters, replacement of study centres, or others,34,35 and some of the reported changes might have happened anyway, such as adding more outcome parameters. Other changes can be strongly related to the pandemic, such as type of educational meetings, delay of study phases, data quality, and especially emotional stress for participating staff.36 Also, other authors reported challenges in reduced recruitment rates, reduced data quality or ensuring safety of patients and staff due to Covid-19, and reaction to suspension of a study, adaption of research processes, or use of new technologies.37–40 Since the original trial did not reach statistical significance in reducing delirium severity,26 the question raises if a better data quality might have led to a better outcome. This is plausible in a few cases, but given the lower-than-expected number of patients with delirium and the low effect size, we assume that better data quality might not have led to a substantial change in the main results.

Data quality fluctuations are not specific to pandemics.41 We observe similar phenomena in periods of staff shortage or staff changes. Here, we saw additive effects: research staff had to work at the bedside or were in quarantine. Though study coordinator communication was maintained due to electronic media, control and re-adjustment of data quality was difficult to monitor due to staff shortage. Recruitment rates dropped because of the pandemic as observed in other departments outside stroke services.30 The most important lesson learned was to conduct quality assessments of data quality, react, and communicate unforeseen changes.

Limitations

This study has several limitations. The generalisability may be limited to our setting. The study was unfunded, led by working clinicians and no dedicated staff were involved; hence, data quality may not be affected in studies with more staff and resources. It is possible that delayed centres were more affected by the pandemic, e.g., by serving more Covid-19 patients, compared to the other centres, but this effect has not been measured and hence, cannot be estimated. The study performance was not reviewed externally, so the lessons learned are limited to the reflection competency of the research team, albeit a multiprofessional and experienced one.

Conclusions

Pandemics may lead to delays in performing unfunded trials, with a significant difference in data quality between study centres with and without delay. Data quality tends to be lower in centres where the pandemic had caused a delay. An ongoing pandemic may affect a clinical trial negatively unless a more frequent monitoring of data allowing for more rapid interventions can be established.

Author Contributions

All authors have made substantial contributions to all of the following: PN, FB, DB, CB, DG, AH, HCH, UH, RI, NK, KK, MM-V, JO, FP, TP, FR, BS, HS, JM, NGM made substantial contributions to (1) the conception and design of the study, or acquisition of data, or analysis and interpretation of data; PN, FB, DB, CB, DG, AH, HCH, UH, RI, NK, KK, MM-V, JO, FP, TP, FR, BS, HS, JM, NGM made substantial contributions to (2) drafting the article or revising it critically for important intellectual content; PN, FB, DB, CB, DG, AH, HCH, UH, RI, NK, KK, MM-V, JO, FP, TP, FR, BS, HS, JM, NGM made substantial contributions to (3) final approval of the version to be submitted.

Ethics

The original QI project was approved by the ethic committee (D459/20) and registered in the public German Register of Clinical Studies (DRKS00021436).

Disclosures and declarations

All authors confirm that this study is not funded and had no financial or non-financial interests; had study-specific approval by the appropriate ethics committee for research involving humans and/or animals, had informed consent if the research involved human participants. Animals were not involved in this study.

Declaration of Conflicts

All authors declare to have no conflicts related to this work.

Data transparency

All data generated or analysed during this study are included in this published article [and its supplementary information files].