INTRODUCTION

Patients near the end of life often are not recognized as imminently dying.1,2,3,4,5,6 Their suffering may not be appreciated or managed properly, and it may be exacerbated by usual medical care involving aggressive, futile, or iatrogenically harmful treatments.7,8 To reduce suffering and improve quality of care, it is imperative for clinicians to know how to identify imminently dying patients and implement supportive and comfort care treatment plans. 9,10

Implementing processes to enhance quality of care at end of life has been recognized as a key priority for decades. National and international task forces have embraced palliative care in response to a call to improve care at the end-of-life,11 leading to the development and testing of various upstream and downstream patient and family-focused interventions designed to change care across settings.12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27 Yet, implementation and dissemination of evidence-based practices in inpatient settings have shown limited success,14,21,24,25,28,29 including comfort care order sets25 and interventions to improve end-of-life care in intensive care units.21,28

In response to a lack of effective implementation strategies for palliative and hospice care at end of life, we developed an education-based comfort care intervention to teach providers in Veterans Affairs Medical Centers (VAMCs) how to identify actively dying patients in inpatient settings and to implement best practices of hospice home care.30 The “Best Practices for End-of-Life Care for Our Nation’s Veterans” (BEACON I) intervention was evaluated in six VA Medical Centers.30 Provider training included a hospice-based Comfort Care Order Set (CCOS) decision support tool embedded in the electronic medical record. This study demonstrated statistically significant changes in several processes of care, including availability of symptom-focused medications, including opioid pain medications (AOR (CI) = 1.39 (1.09, 1.76)), and a reduction in unhelpful or potentially harmful interventions.

Although the effectiveness of the Intervention was demonstrated, it was implemented with intensive involvement of the research team, who traveled to each site and spent 2 weeks providing staff training on all shifts throughout the hospital. Because this approach is not feasible or ideal for large-scale dissemination, it became necessary to identify alternative strategies for broader implementation. A formative evaluation was conducted to understand the processes facilitating adoption of best practices and to inform the design of future dissemination.31 Participant accounts detailed how small group instruction and interactive teaching methods facilitated active learning. They also indicated the need for a train-the-champion model, where training could be ongoing at each site, allowing for a more sustained delivery of training.

Hence, the purpose of this study was to evaluate and compare two methods of delivering the BEACON intervention utilizing the established infrastructure of VA Palliative Care Consult Teams (PCCT): a traditional implementation approach using a teleconference and an enhanced implementation approach utilizing in-person, train-the-champion workshops, both to review educational materials and prepare PCCT members to be on-site clinical champions.

METHODS

Overview

This study was a cluster randomized implementation trial in which 47 PCCTs were assigned to receive the BEACON Comfort Care Education Intervention via a teleconference or in-person workshop implementation approach. Data on processes of end-of-life care before and after implementation of the intervention were derived from the Computerized Patient Record System (CPRS) files of veterans who died in the VAMC hospitals. The study was conducted March 2015–April 2019. It was approved by the Institutional Review Board for human use. All participants signed informed consent forms.

Participants

PCCT teams at selected VAMCs were recruited. All VAMCs are mandated to maintain an interdisciplinary PCCT to serve facility-wide needs for palliative care.32,33 To avoid ceiling effects and lack of readiness, teams were not recruited if their VAMCs scored in the highest or lowest 10th percentiles on the Performance Reporting and Outcomes Measurement to Improve the Standard of Care at End-of-life (PROMISE) Bereaved Family Survey, a 19-item family satisfaction measure.34,35 Additional exclusion criteria were (1) no prescribing provider on the PCCT (enabling use of the CCOS); (2) PROMISE scores not reported; and (3) current use of a comfort care order set.

Randomization

To avoid baseline differences between groups of sites in quality of end-of-life care, PCCT sites were stratified on two factors: presence vs. absence of an inpatient palliative care unit, and higher vs. lower baseline scores on the PROMISE survey based on median split. Having an inpatient unit may affect the processes by which patients receive care. PROMISE scores should reflect the general quality of care provided. Within these strata, teams were randomized to the teleconference or in-person workshop implementation approach.

Power

Power calculations were based on primary endpoint data from the BEACON I study, change in proportion of patients with an active opioid order at time of death. Sample sizes of 2400 patients in each group (24 clusters per group with 100 patients per cluster) would achieve 86% power to detect a difference between group proportions of 0.05, where the proportion of patients post-intervention with an active opioid order at death in the teleconference group was 0.65 and the proportion in the workshop group was 0.70. The significance level was 0.05 and an intraclass correlation coefficient was assumed to be 0.005, as observed in the original trial. Calculations were performed using PASS 11.0.

Comfort Care Education Intervention

The BEACON Comfort Care Intervention targeted providers and used a multi-modal approach to changing provider practice patterns for end-of-life care. It included didactic information on how to identify and care for patients who are dying in the hospital setting, the CCOS, pocket cards, and other tools and training materials needed to train staff at their own hospital. The materials described a number of comfort care interventions appropriate for patients at the end of life. They were derived from the best practices for care in the last days or hours of life as practiced in the home hospice setting and modified for use in inpatient care settings.36,37 Training materials are available at https://www.uab.edu/medicine/palliativecare/training/resources/beacon.

The two groups of trainees received the same pre-implementation preparation and the same education intervention packet of materials. They differed only on the implementation approach used to deliver the education intervention (Table 1).

Table 1 Components of Comfort Care Education Intervention/Implementation
Full size table

Pre-implementation preparation included identifying a lead physician or nurse practitioner champion to spearhead ongoing palliative care education and the CCOS. Each site’s PCCT team was provided with the assistance of a computer applications coordinator (CAC) and a clinical provider at the coordinating site, to work remotely with the local PCCT and CAC to construct a Comfort Care Order Set (CCOS) tailored and integrated into the local VAMC’s CPRS. These order sets could be customized for individual patients, and all or some of the orders could be selected as appropriate and integrated into existing care plans.

For both groups, a trainer from the coordinating center reviewed the education packet materials and the basic functions of the CCOS with PCCT members who were coached in the use of the materials to train other staff. Trainees also reviewed their role as champions and drivers of culture change. To support the addition of medications or routes of administration to the pharmacy formulary, teams were assisted in working with local administration to change policies and procedures, particularly those related to morphine concentrate, sublingual administration, and use of subcutaneous lines.

Implementation Strategies

PCCTs in both arms of the study were exposed to their respective implementation strategy for 4 months. They were charged with training as many appropriate providers as possible at their home facility who may have contact with patients who are imminently dying. During implementation, the training team was available for consultation with teams requesting assistance.

Teleconference

The teleconference arm was designed to align with standard VA training methods. There was no limit to how many PCCT members could participate from each site. In this group, the intervention was delivered in an 80-min teleconference guided by a PowerPoint presentation providing didactic instruction and facilitating group discussion. The standardized curriculum included review of the education packet and teaching to become change agents.

In-Person Train-the-Champion Workshop

Two to three PCCT members from each team traveled to the coordinating center to attend a 2-day, in-person workshop. Ideally, this included a physician and advance practice practitioner with prescribing privileges, so that both would have access to the CCOS and be able to train others. The training team provided a standardized curriculum, including review of the education packet; case-based coaching to build care plans using the CCOS; hands-on computer lab experiences using the CCOS; and coaching to become champions (trainers and change agents). Small group discussions were used to review experiences with the CCOS and compare care plans.

PCCT members reviewed teaching concepts, techniques, and materials for training other providers. During the 4-month implementation period, regularly scheduled conference calls were available for the most recently trained workshop participants to discuss the team’s local implementation.

Measurement

Utilizing a chart abstraction tool designed for the study, a physician abstracted data from the electronic medical records on processes of end-of-life care during the last 7 days of life for veterans who had died as inpatients in acute care units. Data for long-term care units were not included. We collected data for all patients who died in the 9 months before and the 9 months after the intervention period at each site.

Several processes of care were identified a priori as primary and secondary endpoints to indicate quality of end-of-life care. The primary endpoint was presence of an active order for opioid medication at the time of death, because of its importance in the management of pain and dyspnea. Secondary endpoints were selected processes of care representing the development of a comprehensive care plan for actively dying patients. These included orders for and administration of appropriate medications, including antipsychotic medications for delirium or nausea and vomiting, benzodiazepines for anxiety and treatment or prevention of seizures, and medication to control excessive secretions (death rattle).

In addition, the following quality endpoints were abstracted: indicators of communication regarding goals of care (do-not resuscitate [DNR] order in place at the time of death, advance directive), palliative care consultation, orders for environmental changes to enhance patient safety and comfort (nasogastric tubes, intravenous lines infusing, restraints, and location of death), pastoral care visits, and family presence at or near the time of death. Clinical data included diagnoses (ICD-9 codes), locations of care within the hospital, and length of stay.

Inter-rater reliabilities for the primary outcome measures were established between the chart abstractor and the Director of Palliative Care at the coordinating center as the gold standard, based on review of 5% of charts. We calculated reliabilities using Gwet’s agreement statistic in the R package rel due to the large number of ties, which showed excellent agreement of 0.999 (95% CI 0.998, 0.999) across 381 records.38

Statistical Analysis

Processes of care were examined with generalized estimating equations (GEE) to account for clustering of participants, using a compound symmetric correlation structure to model the correlation of participants within sites.39 Initial univariate analyses estimated crude associations between processes of care and study phase (pre- or post-intervention), type of intervention (teleconference or workshop), and study phase X intervention interaction. Multivariate analyses examined the associations of processes of care with study phase and type of intervention. We adjusted the analyses for age, race, terminal condition (cancer/other), length of stay, and study year, because they could conceivably affect outcomes and showed small but statistically significant differences between groups.

Interactions between study phase and type of intervention were not included in multivariate analyses if these were not significant in univariate analyses. Secular trends were evaluated by including an interaction of study year and type of intervention. Sandwich estimators were used with adjustment of the degrees of freedom to account for the small number of clusters.40,41,42,43 Data were analyzed using SAS version 9.2 (SAS Institute, Cary, NC). Data reporting was guided by the Standards for Reporting Implementation Studies (StaRI).44

RESULTS

Among the 152 VAMCs nationwide, 41 were ineligible based on prior participation in the BEACON I implementation, not reporting PROMISE scores, or having PROMISE scores in the top or bottom 10%, leaving 111 sites eligible for active recruitment (Fig. 1). Eight of the remaining sites could not be contacted, 29 declined to participate, and 27 were ineligible based on current use of a palliative care order set. Forty-seven sites were randomized, 23 to the teleconference group and 24 to the in-person workshop group. One site dropped out prior to training (workshop) and one site dropped out after training (teleconference).

Figure 1
figure 1

CONSORT diagram.

Full size image

Characteristics of the Providers

Six waves of training were conducted with 132 providers from PCCTs at 46 VAMCs. Eighty-eight providers from 23 VAMCs received teleconference training, and 44 providers from 23 VAMCs received in-person workshop training (due to the limit on number of travelers). Sixty percent of participants were prescribing providers (42% physicians, 17% nurse practitioners), 40% were non-prescribing providers (17% RNs, 8% pharmacists, 7% mental health providers), and 8% were from other services.

Characteristics of Deceased Veterans

The medical records of 6,491 veterans who died in the VAMCs during the study period were abstracted. Table 2 presents the characteristics of the sample by intervention group (3,078 for teleconference, 3,413 for workshop). The overall sample was predominantly male (97.5%) with a mean age of 75.0 years and slightly over 20% were African American. The sample was balanced across sites by cause of death.

Table 2 Characteristics of Deceased Veterans
Full size table

Changes in Processes of Care

Changes in process-of-care endpoints are presented by group in Table 3. It is noteworthy that the use of the process-of-care interventions was already at or near optimal levels before the intervention was implemented. For example, before the implementation, 89.8% of deceased patients had an active order for an opioid (88.4% teleconference, 91.0 workshop), and 90.3% had a DNR order (88.8% teleconference, 91.7 workshop).

Table 3 Process of End-of-Life Care Endpoints in Teleconference versus Workshop Groups (No. (%))
Full size table

GEE analyses of between group differences found no significant differences between the teleconference and workshop groups on the primary endpoint or any of the secondary endpoints (primary endpoint AOR (CI) = 1.18 (0.74, 1.89) (Table 4). Furthermore, the pre-post changes were not significant for any endpoint (primary outcome AOR (CI) = 1.16 (0.92, 1.46).

Table 4 Multivariable Analyses
Full size table

DISCUSSION

The results of this trial indicate that using a 2-day in-person workshop to deliver an education intervention enabling providers to optimally implement a comfort care order set does not produce better process-of-care outcomes compared to using a traditional teleconference. Furthermore, neither implementation strategy demonstrated significant changes in the use of the comfort care interventions in the 9 months following the intervention period.

One might expect that the in-person workshop would yield better process-of-care outcomes than teleconference due to greater exposure to the training (2 days vs. 80 min), the interactive face-to-face teaching, sharing between teams, hands-on computer training on the CCOS, and the ability to focus on the training by being away from day-to-day duties. While we thought these elements might make a difference, they do not appear to be essential components for enhancing PCCT ability to implement CCOS interventions.

The lack of significant group differences may be explained by the fact that the trainees were all palliative care practitioners working in a healthcare system with a two-decade-long mandate to improve end-of-life care.45,46 Furthermore, the basic curriculum and materials, as well as the CCOS, were the same in the two groups. Perhaps the presence of the CCOS itself was the more salient component in shaping processes of care in the inpatient setting.

Given the significant improvements in process-of-care outcomes observed in the BEACON I trial, the lack of pre-post change in BEACON II was unexpected. It may be that the analyses were limited by high baseline values on some key endpoints leaving little room for improvement. These levels may represent a ceiling effect, given that 9.1% of deaths were unexpected, and it can be assumed that the interventions would not be appropriate in all cases. Furthermore, we may have set our PROMISE score threshold for avoiding ceiling effects too high. We did not foresee the high baseline values on some key endpoints, which were later identified in baseline CPRS chart abstraction.

In addition, the baseline levels of process-of-care endpoints were greater than post-intervention levels observed in BEACON I. These enhanced levels may be attributed to secular trends, possibly including dissemination of BEACON I in the VA Health Care System and other major VA initiatives, as well as continued efforts outside VA.45 The quality of palliative and end-of-life care has long been a priority for the VA Healthcare System.32,33 The VA was an early adopter of robust palliative care services distributed widely throughout the system which supports the effectiveness of palliative care interventions such as Palliative Care Consults for patient-centered outcomes and family satisfaction.46,47,48 Over decades, the VHA system, through a broad range of palliative care clinical and programmatic initiatives, has demonstrably increased access, use, and quality of palliative care services.49,50,51,52

One of the limitations of the study is its reliance on patient chart abstraction for measuring processes of care. One might question the selection of the process-of-care variables evaluated or the validity of the chart abstraction method of data collection. We do not believe these are salient issues, because the variables we examined were the same as in BEACON I, where improvements were demonstrated. Furthermore, inter-rater reliability was high and was assessed in the same way using the ratings by the lead investigator as a gold standard. Finally, changes in practice patterns can take time, especially in large organizations, and may not be reflected in the relatively short 9-month timeframe of the study. Finally, the study was conducted entirely within the VA which has a unique history of palliative care initiatives. Therefore, results could be different when implemented in other healthcare systems. Because changes in practice patterns can be challenging and slow, especially in large organizations, future implementation trials might consider other approaches to behavior change or outcomes at other points in the pathway from training to patient outcomes.

In conclusion, although an in-person workshop was the training modality preferred by providers from both groups in our analyses of qualitative interviews,53 the teleconference is perhaps more economical and resource efficient. These findings can inform considerations of the optimal approaches to palliative care education and training and optimize further nationwide dissemination of best practices for end-of-life care in both VA and non-VA settings.