Accuracy and interrater agreement of death event adjudications by physician trainees: validation of the ISTH definition of pulmonary embolism-related death in an autopsy cohort

Background: We previously determined good agreement and high specificity of the International Society on Thrombosis and Haemostasis (ISTH) definition of pulmonary embolism (PE)-related death among an expert central adjudication committee (CAC). CACs are often composed of experts in the corresponding research field. Involving physician trainees in CACs would allow investigators to divide the workload and foster trainees' research experience.

Objective: To evaluate the accuracy of the ISTH definition of PE-related death for PE- versus non-PE-related deaths as confirmed by autopsy and its interrater agreement among physician trainees.

Methods: This retrospective autopsy cohort included all patients with PE-related deaths between January 2010 and July 2019 as well as patients who died in 2018 from a cause other than PE at the New York-Presbyterian Hospital. Based on premortem clinical summaries, two physician trainees independently determined the cause of death using the ISTH definition of PE-related death. We calculated the sensitivity and specificity of the ISTH definition to identify autopsy-confirmed PE-related death and its interrater agreement.

Results: Overall, 126 death events were adjudicated (median age, 68 years; 60 [48%] women), of which 29 (23%) were due to PE, as confirmed by autopsy. Sensitivity and specificity of the ISTH definition for autopsy-confirmed PE-related death was 48% (95% CI, 29-67) and 100% (95% CI, 96-100), respectively. Interrater reliability for PE-related death was good (percentage agreement, 93%; 95% CI, 87-96, Cohen's Kappa, 0.67; 95% CI, 44-85).

Conclusion: Our findings are consistent with our previous validation study. They further support the use of the ISTH definition of PE-related death and revealed high agreement between adjudicators with varied experience.