Introduction
Our institution used over 300 unique clinical decision support (CDS) alerts to address patient care. The high volume and breadth of alerts had the potential to overwhelm hospitalist providers, who are more likely to ignore alerts as the total alert volume increases.1,2 This problem was accentuated for repeated alerts, which disrupt provider workflow and negatively affect patient care.2,3 Many institutions have attempted to address similar alert burdens. One conducted surveys to find ways to refine alerts.4 Another categorized alerts by severity.5 A third explored whether alerts should be displayed in groups or individually.6 We sought to optimize our alert by increasing the time between repeat alerts to the same provider.
At our institution, an internal audit previously identified a venous thromboembolism (VTE) prophylaxis alert as a target for improvement as it triggered frequently (13% of all local alerts) and less than 5% of alert responses resulted in a change to patient care. This alert identified inpatients with risk factors for VTE but without orders for prophylactic medications such as heparin. The alert did not trigger for any patient currently receiving prophylaxis. When providers accessed an inpatient chart, the alert prevented closing of the chart until the provider responded with an order for prophylaxis, documentation that prophylaxis was not indicated, or a decision to defer until a later time. While providers complained of excessive alerts, this alert was directly responsible for 20% of local VTE prophylaxis orders.
During a previous intervention, a snooze option was added to allow providers to defer a VTE prophylaxis decision for one hour before the alert was retriggered.7 This option was intended as a rare deferment during emergent patient care or when providers lacked enough information to definitively address the alert. However, providers selected snooze for 67% of alert firings. While previous versions of our electronic health record (EHR) allowed only one-hour deferment periods when snooze was selected, more recent software updates allow for longer deferment periods. We hypothesized that alert burden would lengthen the time between alert firings to a specific provider after selecting the snooze response (“snooze delay”).
The primary objective of this study was to reduce weekly firing of the VTE alert by 25% within 60 days of project implementation without decreasing performance on VTE prophylaxis core measures or the percentage of patients receiving VTE prophylaxis.
Methods
Setting: The study took place at the University of Iowa Hospitals & Clinics (UIHC), an 800-bed tertiary care teaching hospital with approximately 37,000 inpatients annually. The VTE alert was integrated into our EHR (Epic Systems) and fired when providers attempted to close the charts of adult inpatients with at least one risk factor for VTE and no orders for VTE prophylaxis.
Intervention design and data collection: We created two EHR reports. The first monitored alert performance and included number of admissions, number of VTE alert firings, number of prophylaxis orders, and the interval between alert activation and a provider response to that alert (provider dwell time).8 The second monitored patient safety and included: number of admissions, pass rate of electronic clinical quality measures (eCQM), total number of patients with VTE pharmacoprophylaxis, and prophylaxis choices.9 We selected the VTE-1 (general inpatient VTE prophylaxis) and VTE-2 (intensive care unit VTE prophylaxis) core measures for this project as markers for patients receiving appropriate prophylaxis. Prior to the intervention, nine weeks of baseline data was collected with snooze delay set at the established setting of one hour. Then the snooze delay for the VTE alert was set to two hours for 5 weeks and 3 hours for 7 weeks with ongoing weekly data monitoring. These changes were made for all inpatient providers with the exclusion of those in the emergency room, procedural areas, or other departments not providing inpatient care. These reports were run weekly to identify any deterioration in core measure performance or risk to patient safety.
Outcome definitions: The primary outcome for this project was alert burden defined as the number of VTE alerts per 100 admissions. Secondary outcomes included the percentage of alerts that were “snoozed,” the percentage of inpatients given VTE pharmacoprophylaxis, and the percentage of admissions satisfying VTE prophylaxis eCQMs. We defined a patient as receiving pharmacoprophylaxis if they received any of the following medications at any dose: unfractionated heparin, low molecular-weight heparin, a direct oral anticoagulant (DOAC), or warfarin. These outcomes were calculated weekly using the two EHR reports. Prior to initiation, we decided to revert to a 1-hour snooze delay if we observed any statistically significant drop in VTE core measure performance or in the percentage of patients receiving pharmacoprophylaxis.
Statistical analysis: For analysis of alert performance chi-square analysis was used to detect statistical differences in proportions (e.g. eCQM pass rate) and a 2-sided t-test was used for comparison of continuous variables (e.g. alerts/100 admissions). For analysis of patient safety the QI Macros plug-in (KnowWare International Inc. Denver, CO) was used to create control charts.10 These charts were used to detect changes and were updated weekly basis. All calculations were performed using Excel (version 1902, Microsoft, Redmond, WA).
Ethical considerations: An exemption was submitted and accepted by the University of Iowa Institutional Review Board.
Results
We collected data for 21 weeks, including 9 weeks of baseline data (1-hour snooze delay), 5 weeks of 2-hour snooze delay, and 7 weeks of 3-hour snooze delay. Results of alert performance and patient safety measures are provided in Table 1.
Prior to the intervention, with a 1-hour snooze delay, VTE alerts fired 168 times per 100 admissions with 67.0% of providers selecting snooze. There was no significant change in either metric for a 2-hour snooze delay (179 firings, 67.5% snooze). For a 3-hour snooze delay there was no change in alerts per 100 admissions (176) but there was a marginal decrease in snooze percentage (63.2%, p < 0.001). Throughout the study period, there was no clinically significant change in the percentage of patients receiving VTE pharmacoprophylaxis (range 50.9-51.7%) or combined institutional performance on VTE-1 and VTE-2 eCQMs (range 86.9 to 88.0%) shown in Figures 1 and 2 respectively.
Based on the unexpected negative results, we performed a non-prespecified analysis to ensure that the intervention was working correctly and to better understand why the alert volume was unchanged despite the increase to the snooze interval. Post hoc analysis of the baseline data showed that 32% of VTE alert firings were repeated alerts to the same provider for the same patient shown in Figure 3. In addition, 55% of those consecutive firings were within a four-hour period. After extending the snooze delay to 3 hours, 28% of all firings were same-provider, same-patient alerts and less than 2% were within a 3-hour period. It appeared that the intervention was functioning correctly within the EHR.
We proceeded to evaluate alert performance based on specific departments and providers. Across the final period with 3-hour snooze delay 1.5% (10 of 661) of providers who used the snooze option most often accounted for over 17% (1502 of 8686) of snooze responses. These providers rarely selected a definitive response to prophylaxis even after assessing patients. Eight of these providers were within the internal medicine hospitalist group. Five of these providers had responded to alerts with snooze more than 90% of the time and four had never placed an order through the BPA. Internal medicine hospitalists combined for 22.2% of all alerts and 27.0% of all snooze responses to the VTE alert.
Discussion
Increasing the snooze interval did not significantly decrease provider alert burden but also did not impact performance metrics or prophylaxis delivered. While at longer intervals the percentage of snooze responses slightly decreased and the median interval between same provider consecutive snooze responses increased, the post hoc analysis shows that further increases in the snooze delay are unlikely to achieve our aim of a 25% reduction in alert burden.
There have been numerous published attempts to reduce alert burden.4–6 These attempts often target optimization of at least one of “the five rights of clinical decision support,” which include the right information, to the right person, in the right format, through the right channel, at the right time in workflow.11 For example, the study exploring how alerts should be displayed was focused on the “right format” of alerts. In contrast, our study sought to find the right time in workflow for alerts by manipulating the length of the snooze delay. This attempt was unsuccessful, and we returned to the “rights” to find a reason why.
We hypothesized that the problem stemmed from who (the right person) received the VTE alerts. Further analysis of the data showed that there are two groups of providers central to alert burden at our institution. The first of these groups involves providers who use the snooze option once. Even though these providers may use the snooze option appropriately to delay their response until assessing the patient, due to their large numbers they still account for one-third of all responses. While a mean of four providers were alerted per patient, many of these providers used the snooze option but did not receive a second alert. It is possible that these providers were not members of the primary team and therefore were not the right persons to receive the alert. Refining who receives alerts could have a significant impact on total alert burden at our institution.
Continuing to evaluate “who” received the alerts, we found that specific providers, including internal medicine hospitalists, appeared to be disproportionately responding to the alert with snoozes. These providers were members of the primary team and were likely the right providers to receive these alerts. Many of these providers were selecting snooze for a high percentage of total responses, implying that there was no time in their shift or point in their workflow when they were predisposed towards a definitive alert response. Rather than attribute alert performance to one of the five rights outlined above it is clear that there is a sixth “right” that must be addressed for this group of providers: the right culture.
Developing an appropriate culture towards responding to the VTE alert at our institution could have a significant effect on total alert burden by decreasing repeated alerts. While many expert reviews and collaborations look at the details of alerts to reduce burden,12–14 we suggest that the mindset of providers is an equally important factor. We believe that the lack of dedicated provider education with our implementation impacted our results.
Ten providers accounted for 17% of total snooze responses, suggesting a targetable group that could meaningfully change BPA performance through education. Alternatively, if these providers were targeted inappropriately by the alert this could be identified through direct interviews and workflow observations. Interviews may also allow exploration of reasons for repeated alert snoozing. Change in the EMR is a complex process and providers often express resistance to alterations in workflow.15
Providers may also be fatigued due to an overwhelming number of daily alerts.16 In contrast, if alerts are addressed immediately, this will eliminate BPAs for other providers who would otherwise receive duplicate alerts. Through educating and interviewing only a handful of providers we can create a culture focused on accepting responsibility for these alerts and eliminate alert burden for the hospital system as a whole.
Limitations: The data used in this study was limited to the EHR. Although we can analyze the number of times a provider chooses to snooze alerts, we have no patient-specific context to determine whether each decision was appropriate. Given the unique clinical context of each patient it is possible that some providers using the snooze option once are using it inappropriately and other providers who repeatedly snooze the alert have clinical reasons for their decision. Furthermore, while the EHR can detect many risk factors for VTE, the alert is based on provider-entered problem list data which may result in inappropriate over or under identification of patients. We cannot assess the validity of specific provider decisions to order or withhold VTE prophylaxis.
The study was also limited by its dependence on a provider’s use of the EHR. The alert only triggers when a provider accesses the chart. Therefore, if a provider did not access a patient’s chart for several hours, the increase of snooze delay from one hour to three hours would have no effect on alert burden. This would minimize the effect of increased snooze delay.
Conclusions
Increasing the snooze delay from one hour to three hours did not decrease VTE alert burden or impact prophylaxis use despite the interval between same-provider snoozes increasing dramatically. Alert burden varied by provider and specialty. Further optimization of the snooze delay is unlikely to reduce alert volume without a provider education component. Our next step will be to approach these provider groups with in-person education on alert management best practices.
Disclosures
None of the authors have identified a conflict of interest in relation to this manuscript.
Corresponding author
Ethan Kuperman
Clinical Associate Professor
University of Iowa
Department of Internal Medicine
200 Hawkins Dr., SE 622 GH
Iowa City, IA 52242, USA
Phone 319-353-7053
Fax 319-356-3086
Email: ethan-kuperman@uiowa.edu
ORCID: 0000-0002-7806-9759