Predicting Acute Exacerbations in Chronic Obstructive Pulmonary Disease

BACKGROUND: With increasing health care costs that have outpaced those of other industries, payers of health care are moving from a fee-for-service payment model to one in which reimbursement is tied to outcomes. Chronic obstructive pulmonary disease (COPD) is a disease where this payment model has been implemented by some payers, and COPD exacerbations are a quality metric that is used. Under an outcomes-based payment model, it is important for health systems to be able to identify patients at risk for poor outcomes so that they can target interventions to improve outcomes. OBJECTIVE: To develop and evaluate predictive models that could be used to identify patients at high risk for COPD exacerbations. METHODS: This study was retrospective and observational and included COPD patients treated with a bronchodilator-based combination therapy. We used health insurance claims data to obtain demographics, enrollment information, comorbidities, medication use, and health care resource utilization for each patient over a 6-month baseline period. Exacerbations were examined over a 6-month outcome period and included inpatient (primary discharge diagnosis for COPD), outpatient, and emergency department (outpatient/emergency department visits with a COPD diagnosis plus an acute prescription for an antibiotic or corticosteroid within 5 days) exacerbations. The cohort was split into training (75%) and validation (25%) sets. Within the training cohort, stepwise logistic regression models were created to evaluate risk of exacerbations based on factors measured during the baseline period. Models were evaluated using sensitivity, specificity, and positive and negative predictive values. The base model included all confounding or effect modifier covariates. Several other models were explored using different sets of observations and variables to determine the best predictive model. RESULTS: There were 478,772 patients included in the analytic sample, of which 40.5% had exacerbations during the outcome period. Patients with exacerbations had slightly more comorbidities, medication use, and health care resource utilization compared with patients without exacerbations. In the base model, sensitivity was 41.6% and specificity was 85.5%. Positive and negative predictive values were 66.2% and 68.2%, respectively. Other models that were evaluated resulted in similar test characteristics as the base model. CONCLUSIONS: In this study, we were not able to predict COPD exacerbations with a high level of accuracy using health insurance claims data from COPD patients treated with bronchodilator-based combination therapy. Future studies should be done to explore predictive models for exacerbations.

I n the current environment of increasing U.S. health care costs, cost management strategies have become a key focus. Traditional fee-for-service payment models, which have promoted quantity over quality, seem unsustainable given the continued rise in health care costs. U.S. health care expenditures were $3.2 trillion in 2015, yet health outcomes are not better than many other developed countries that spend considerably less. 1,2 More recently, payers have proposed alternative payment models that motivate health care providers to meet certain quality metrics. 3 These value-based payment approaches tie reimbursement to patient outcomes, putting a greater focus on effectiveness of care.
Chronic obstructive pulmonary disease (COPD) is a disease where some payers have implemented alternative payment models. 4,5 COPD has increased in prevalence with the aging population and now represents the third leading cause of death in the United States. 6 Direct COPD medical costs totaled $32.1 billion dollars in 2010 and are projected to increase to $49 billion dollars by 2020. 7 Exacerbations, which often require emergency department (ED) visits or hospitalization, contribute to a significant portion of spending on COPD. 6 A recent study • Outcomes-based payment models have used COPD exacerbations as a quality metric to determine reimbursement rates for providers and health systems. • Previous studies have identified factors that are predictive of chronic obstructive pulmonary disease (COPD) exacerbations including history of exacerbation, COPD disease severity, and COPD treatment; however, these studies have generally included all COPD patients, regardless of whether they were treated according to guidelines.

What is already known about this subject
• This study attempted to identify factors predictive of exacerbations among patients being treated for COPD with bronchodilator-based combination therapy as recommended by COPD guidelines. • When comparing patients treated with comparable treatment regimens, we were unable to develop a model that accurately predicted COPD exacerbations.

Characteristics of Patients with and Without Exacerbations (continued)
continued on next page patients across different severity levels and treatments. While it may be easier to predict exacerbations across patients with different levels of COPD disease severity, it may be more challenging to predict exacerbations in a COPD patient population of similar disease severity and which is treated according to established guidelines. The purpose of this study was to develop a model that predicts patients who are likely to have a COPD exacerbation among patients with similar COPD treatment regimens. Since administrative claims data are readily available and cost-effective for payers evaluating health outcomes, we used this information as the basis for developing a claims-based prediction model.

■■ Methods Data Source and Model Development
We used retrospective health insurance claims data from January 1, 2004, through December 31, 2014, from the Truven Health MarketScan Commercial Claims and Encounters and Medicare Supplemental databases. These data contain patientlevel demographics; enrollment information; and claims data for inpatient services, outpatient services, and outpatient prescription claims from over 230 million patients in the United States. Data were deidentified and so were determined to constitute nonhuman subjects research by the Institutional Review Board at the University of Illinois at Chicago. 13 Patients with a diagnosis code for COPD at any point before the index date (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] codes 491.xx, 492.xx, and 496.xx) were included in the study if they were aged 40 years or older and were first initiating a bronchodilator-based dual combination treatment based on prescription claim information. Bronchodilator-based dual combinations included long-acting beta2-agonist (LABA)/long-acting muscarinic has shown that patients with a single outpatient exacerbation in a 1-year period had mean all-cause annual medical costs that were $3,831 higher than patients without an exacerbation. 8 The increasing prevalence, high costs, and interest in optimizing outcomes of COPD patients have made this disease a target for value-based payment models. 9 To implement value-based payment models, it is necessary for payers to identify quality metric indicators of poor outcomes and then adjust payment based on these outcomes. The Prevention Quality Indicator (PQI) score is a quality metric developed by the Centers for Medicare & Medicare Services (CMS). The PQI score is a ratio of observed to expected COPD admissions that is calculated for hospitals and compared with a benchmark value. 5 Reimbursement to hospitals are adjusted based on their PQI scores. COPD readmission rates are also a quality metric used by CMS as a part of the CMS Hospital Readmissions Reduction Program. In this program, there are reduced payments to hospitals if a patient is readmitted within 30 days of a previous hospitalization for a COPD exacerbation. 4 In addition to quality metrics, there are costs of care measures that are sensitive to poor outcomes that can be expensive for the health systems, such as exacerbations. The Relative Resource Use measures by the National Committee for Quality Assurance are examples of cost of care measures. 10 In value-based payment models, health systems need to identify patients at risk for poor outcomes who are costly to the health care system. When health systems can identify these patients, they can target interventions in order to avoid the poor outcomes. Since exacerbations add significant costs for patients with COPD, several algorithms have been proposed to help identify patients at highest risk for exacerbations; however, many of these algorithms are based on data that may not be readily available to large health system organizations. 11,12 In addition, previous algorithms compare COPD

Characteristics of Patients with and Without Exacerbations (continued)
index date was identified from January 1, 2004, through July 1, 2014. Patients were required to have continuous enrollment during the 6-month period before the index date. Patients were excluded if there were claims for a medication within 30 days of the index date, which suggested that patients were being treated with a triple bronchodilator-based therapy (i.e., a claim for ICS for patients treated with LABA/LAMA or a claim for LAMA for patients treated with LABA/ICS). Patients were also excluded if they had claims for asthma (ICD-9-CM code 493. xx) during the 6-month baseline period or if they lost enrollment eligibility within 30 days after the index date. antagonist (LAMA) and LABA/inhaled corticosteroid (ICS). These combinations are generally prescribed at the same place in therapy in a more severe patient population at high risk for COPD exacerbations. Use was defined as more than 1 fill for the combination treatment. Combination use included a claim for a fixed-dose combination product or separate prescription claims for the 2 products within 15 days. The index date was the date of first use of the combination treatments. This date was the date of first fill for fixed-dose combination products or was the fill date for the second product when 2 separate products were used concurrently (i.e., fills within 15 days). The hospitalizations with primary diagnosis codes for cardiovascular/cerebrovascular events. COPD exacerbations were identified over a 6-month outcome period, starting 30 days after the index date. Thirty days between the index date and the outcome period start date were required to ensure that exacerbations occurring during the baseline period were not misclassified as study-related exacerbations. [16][17][18] We examined a 6-month time period in order to identify patients at risk for an exacerbation shortly after being prescribed the bronchodilator-based combination, since these are the patients who may benefit from an additional intervention in order to prevent an exacerbation. COPD exacerbations included outpatient exacerbations, ED exacerbations, and inpatient exacerbations. Inpatient exacerbations were defined as an inpatient hospitalization with a primary diagnosis code for COPD (excluding obstructive chronic bronchitis without exacerbation [ICD-9-CM code 491.20]). Outpatient and ED exacerbations were defined as outpatient or ED visits with a diagnosis code for COPD and prescription claims for an oral antibiotic or oral corticosteroid 5 days before or after the outpatient or ED visit. 16 Less than 30 days supply per claim was required for the antibiotic/corticosteroid because we assumed from this that the medication was not for chronic use.

Analyses
Logistic regression was used to predict the occurrence of exacerbations. The base model included the following variable categories collected during the 6-month baseline period: COPD combination treatment (LABA/LAMA or LABA/ICS); demographics; enrollment information (beneficiary status, prescription coverage, plan type, and Medicare); comorbidities;

Variables
We identified baseline patient demographic information, enrollment information, comorbidities, medication use, and health care resource utilization in the data during the 6 months before the index date. Demographics included age, sex, region, employment status, employee classification, and employment industry. Enrollment information included beneficiary relationship, health insurance plan type, Medicare enrollment, and prescription coverage. Comorbidity information was collected from baseline ICD-9-CM diagnosis claims on 47 distinct comorbidities categorized by the Clinical Classification Software from the Agency for Healthcare Research and Quality. 14 Medication claims were obtained from outpatient prescription claims on COPD medications, medications that may increase risk of COPD exacerbations, medications with cardiovascular effects, acute use of oral antibiotics (< 30 days supply), acute use of oral corticosteroids, and pneumococcal and influenza vaccinations.
Categories of COPD medications included short-acting beta agonists, short-acting muscarinic antagonists, LABAs, LAMAs, ICS, phosphodiesterase inhibitors, and methylxanthines. Medications that potentially increase COPD exacerbation risk included abatacept, zanamivir, adenosine, antihistamines, beta blockers, and opiates. Twenty-two drug categories were defined under medications with cardiovascular effects. 15 Measures of health care resource utilization included COPD-related and all-cause events. Specifically, baseline measures included medical claims for spirometry; all-cause physician visits (pulmonologist, cardiology, internal medicine, and family practice); physician visits for COPD (any diagnosis position); ED visits for COPD (any diagnosis position); hospitalizations with primary diagnosis codes for COPD; or  Comorbidities, medication use, and health care resource utilization were treated as separate binary variables (yes or no). COPD medications, antibiotics, corticosteroids, and COPDrelated health care resource utilization were not binary variables and, instead, were categorized as 0, 1, 2, and ≥ 3 claims, with 0 claims serving as the referent group. Baseline characteristics are detailed further in Table 1.

Histogram of Predicted Probabilities from Base Model Among Patients with an Exacerbation
Nominal variables, such as demographics and enrollment information, were treated as such and compared with a reference group. Variables with frequencies < 1% were excluded from the models. The dataset was randomly divided into a training set (75%) and a validation set (25%). Stepwise regression was performed on the training dataset, and covariates with a 0.3 significance level entered the model, while a 0.05 significant level was required to stay in the base model. We intentionally selected a more relaxed significance level for variable model entry (0.3) to ensure that all potentially important variables were tested for significance in the model, while more strict criteria were used for variables to stay in the model (0.05).
Coefficients generated from the model-fitting process were imposed back on the training dataset to generate a predicted probability for exacerbation based on the values of the covariates for each observation. 19 Prediction probabilities ranged from 0 to 1, and value ≥ 0.5 was used as an indicator of a predicted exacerbation. The validation dataset was used to evaluate the model developed from the training dataset. Model discrimination was evaluated by sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic (ROC) curve. Model calibration was evaluated with the Hosmer & Lemeshow, Pearson's, and deviance tests for the training and validation datasets.
In addition to the base model, other models were explored using the same model-building approach but including different sets of observations and variables. These models were developed to explore the best approach to predict exacerbations. While the base model included treatment regimen (LABA/LAMA and LABA/ICS) as a binary variable, in exploratory analyses, models were developed separately for patients treated with LABA/ICS and patients treated with LABA/LAMA.
To avoid potential collinearity between comorbidity, medications, and health care resource utilization variables, we created separate models that only included variables from 1 of the categories, along with demographics and enrollment information. We used a refined definition of exacerbation, including only inpatient exacerbations as the outcome. In the final model, we increased the predictive probability of exacerbation threshold from 0.5 to 0.7. Alternative model specifications were explored to evaluate the assumptions of the model-building approach. Specifically, we varied the significant level for variables to enter and exit the model (between 0.01 to 0.3), kept all variables in the model, and recategorized covariates. All analyses were conducted in SAS version 9.4 (SAS Institute, Cary, NC).

■■ Results
A total of 478,722 patients met all study criteria and were included in the final analytic sample (Figure 1). Mean age was 60.5 years, and 41.1% of patients were males. There were 473,388 patients treated with LABA/ICS, and 5,384 patients treated with LABA/LAMA. Exacerbations occurred in 40.5% of patients in the follow-up period, and among these, 2.2% were inpatient exacerbations.    Base Model Odds Ratios continued on next page exacerbation was much higher (85.4%). Positive and negative predictive values were moderate at 66.1% and 68.3%. The model had low to moderate discriminative properties, with an area under the ROC curve of 0.707. The Hosmer and Lemeshow test was statistically significant (P < 0.001), indicating poor fit of the predicted probabilities compared with the actual occurrence of events. The Pearson's and deviance tests were also statistically significant (0.0364 and < 0.001, respectively). In the validation dataset, predictive properties were similar to that of the training dataset. The area under the ROC curve was 0.706, and sensitivity and specificity were 41.9% and 85.3%, respectively. There was significant overlap of the predictive values for patients who had an exacerbation compared with patients who did not have an exacerbation, showing little ability to discriminate between the 2 groups ( Figure 2 and Figure 3). The variables, odds ratios, and confidence limits for the final base model are presented in Table 2. These values should be interpreted with caution, since the performance of the base model was poor. When we modeled exacerbations among patients treated with LABA/ICS, results showed similar properties to the base model, with low sensitivity and higher specificity (Appendix B, available in online article). Among patients treated with LABA/ LAMA, model sensitivity was higher; however, specificity was compromised, since only 253 patients out of 1,169 patients without an exacerbation were correctly classified.

Histogram of Predicted Probabilities from Base Model Among Patients Without an Exacerbation
When examining all patients regardless of index treatment, models adjusting for a subset of the covariate categories had similar predictive power as the base model. Sensitivity ranged from 34.4% to 38.9%, while specificity ranged from 84.9% to 87.7% (Appendix B, models 4 through 6). Results were similar in the validation datasets.
When focusing on inpatient exacerbations, the model correctly classified inpatient exacerbations for 4 patients out of 3,162. Increasing the predictive probability threshold for exacerbations in the base model resulted in improvements in specificity (96.6%) but at the expense of sensitivity (17.6%). Additional sensitivity analyses and alternative model specifications resulted in similar findings as models previously mentioned, including the full model without variables removed in a stepwise regression approach. Across all models, the validation datasets resulted in similar predictive properties as those from the training datasets.

■■ Discussion
The purpose of this study was to develop a predictive model to identify patients at risk for COPD exacerbation among those who were users of a bronchodilator-based combination treatment. Because reimbursement is more frequently tied to quality metrics such as COPD exacerbations, as with the PQI by CMS, 5 it is important for health systems to identify patients at risk for these events and target interventions to improve these outcomes.
Covariates levels were similar across the training and validation datasets. Baseline demographics and enrollment information were similar among patients with and without an exacerbation, with mean age slightly higher in patients with an exacerbation (63.4 years) compared with patients without an exacerbation (58.6 years). However, a much greater percentage of patients with an exacerbation were aged 65 years or older (42.9% vs. 26.9%; Table 1). Comorbidities were generally similar between the 2 groups, with the exception of lower respiratory disease, chronic airway obstruction, and obstructive chronic bronchitis having higher prevalence among patients with an exacerbation. Patients with a COPD exacerbation generally had more claims for COPD-related medications and COPD-related health care resource utilization. Cardiology claims were also slightly higher in patients with an exacerbation. Appendix A (available in online article) lists variables with frequencies < 1%.
The base model with the training dataset showed poor sensitivity to identify patients with a true exacerbation (41.7%), while the specificity to identify patients without a true  existing systems. Failure to identify predictive factors for COPD exacerbations could be because exacerbations cannot be predicted based on measureable indicators using technology currently available. Previous studies have focused on identifying predictors of COPD exacerbations, but none have found a single variable or subset of variables that consistently predict patients who will have an exacerbation among a subset of the COPD patient population managed according to the guidelines. 18 The poor ability to predict exacerbations from a large number of variables such as those included in this study leads us to question whether COPD exacerbations are an outcome that can be consistently predicted using claims data alone among patients treated according to guidelines.
Several different models were explored in our study, and all resulted in similar findings, suggesting that there may be other information needed to identify patients at high risk for exacerbations, such as clinical measures of lung function and symptoms. Low socioeconomic status, poor access to health care, and social stressors have also been shown to correlate with poor health outcomes 24 ; however, if this information is not obtainable, then it will be more challenging for health systems to implement interventions to improve these outcomes. Also, COPD exacerbations are complex and may involve a multitude of factors, including social and behavioral elements that may not consistently influence outcomes. If physicians and health systems are unable to predict those patients at risk for exacerbation and take action on this problem, we need to question whether reimbursement tied to COPD exacerbations is the appropriate approach.

Limitations
There are several limitations to this study that should be considered. First, this study focused specifically on patients who were treated with a bronchodilator-based combination treatment because we wanted to determine the predictors of exacerbation among a COPD patient population already at risk for exacerbations. Expanding this study to all COPD patients may lead to more differentiation and ability to predict exacerbations; however, we felt that the patients at risk for COPD exacerbations were the group of greater interest.
Second, exacerbations were defined based on health insurance claims data, which are primarily used for billing purposes. Although our definition is similar to that used in other studies, there may have been some exacerbations that were not captured or were misclassified. 16 Medical supplemental data were used for the Medicare patient population. There is the potential for missing claims in this dataset, if claims were processed without Medicare supplemental coverage. Follow-up time was limited to a 6-month period in this study; looking at shorter or longer follow-up times may change the ability to differentiate patients with and without an exacerbation. By requiring a 30-day washout period after the index date, we may have failed to capture any exacerbations that occurred We used widely available health insurance claims data to develop our predictive model. Our definition of exacerbations included only those events requiring health care intervention and considered to be the greatest burden to the health care system. A robust number of variables were considered for analysis, including demographics, enrollment information, comorbidities, medication use, health care utilization related to COPD, and health care utilization not related to COPD. Patients with exacerbations were slightly older and had higher number of COPD-and cardiovascular-related claims. The base model showed poor sensitivity to identify true exacerbations during the follow-up period. Several other models were developed to determine the best approach to predict exacerbations. All of these resulted in similar results as the base model, showing that it is difficult to predict those who would have an exacerbation among patients treated with a bronchodilator-based regimen using health insurance claims data.
Many studies have examined predictors of COPD exacerbations; however, most of these studies have focused on the predictive properties of individual variables. This approach contrasts with our study, in which we tried to use a set of influential variables to develop a predictive model. In other studies, variables that have been consistently associated with exacerbations include a history of COPD exacerbations and increasing COPD disease severity. 12,20,21 While health insurance claims data can capture a patient's history of COPD exacerbations, disease severity is not readily available in large datasets. A study published in 2016 by Stanford et al. explored COPD medication use in the health insurance claims data as a metric associated with exacerbations. 22 This study found that a high ratio of maintenance COPD medications to total COPD medications was associated with a lower risk of exacerbation. However, the study did not explore other variables that influenced risk of exacerbation. 22 Biomarkers have also been explored as another potential predictor of COPD exacerbations in an analysis of the SPIROMICS and COPDGene COPD study cohorts. 11 Clinical and biomarker information were analyzed for over 3,000 patients, but while some biomarkers were associated with exacerbations in subpopulations, these associations could not be replicated in the other cohorts.
Other studies, such as that by Moretz et al. (2015), have used predictive modeling to identify other events such as patients with undiagnosed COPD. 23 Although our model building approach was similar to the Moretz study, our model had poorer performance. This may, again, point to the difficulty of predicting COPD exacerbations, especially among COPD patients treated according to guidelines.
The realization of value-based payment models requires quality metrics that are measurable and actionable. Identification of appropriate indicators of quality care is a challenge, along with determining if that data are routinely available in immediately after initiating therapy. Because we based our predictive model on health insurance claims data, we were not able to capture clinical indicators of disease severity, including symptoms and measures of lung function.
Third, socioeconomic factors were not considered in this study. This information is not widely available in health insurance claims data, but previous research has shown these factors to be an important consideration when implementing health care interventions to improve patient outcomes. 25 Other databases, besides administrative claims data, may provide additional patient information that could be explored for improving the predictive power for exacerbations.
Finally, this study examined COPD exacerbations. Quality metrics for COPD and COPD exacerbations may be different than what we have captured in this study. There may be other quality metrics or measures of effectiveness of treatment that are important to examine.

■■ Conclusions
The model built in this study was not able to predict COPD exacerbations using data from a large health insurance claims database. Future studies may be needed to validate these findings or determine other variables that are necessary to predict COPD exacerbations. As payers move from fee-for-service to outcomesbase payment models, it is important to incorporate quality metrics that are predictable and actionable for health systems.