Risk of Nondherence to Diabetes Medications Among Medicare Advantage Enrollees: Development of a Validated Risk Prediction Tool

BACKGROUND: Low adherence to oral antidiabetic drugs (OADs) in the Medicare population can greatly reduce Centers for Medicare & Medicaid Services (CMS) star ratings for managed care organizations (MCOs). OBJECTIVE: To develop and validate a risk assessment tool (Prescription Medication Adherence Prediction Tool for Diabetes Medications [RxAPT-D]) to predict nonadherence to OADs using Medicare claims data. METHODS: In this retrospective observational study, claims data for members enrolled in a Medicare Advantage Prescription Drug (MA-PD) program in Houston, Texas, were used. Data from 2012 (baseline period) were used to identify key variables to predict adherence in 2013 (follow-up period). Members aged 65 years and older with a diabetes diagnosis, at least 1 prescription for OADs (biguanides, sulfonylureas, thiazolidinediones, dipeptidyl peptidase-4 inhibitors, or meglitinides), and continuously enrolled for both years were included in the study. Patients with insulin prescriptions were excluded from the cohort. The study outcome, nonadherence in 2013, was defined as proportion of days covered (PDC) < 80%. Multivariable logistic models using 200 bootstrap replications (with replacement) identified factors associated with nonadherence. The final model was tested for discrimination and calibration statistics and internally validated using 10-fold cross-validation. Using weighted beta coefficients of the predictors, the RxAPT-D was created to stratify nonadherence risk and was tested for sensitivity, specificity, positive prediction value, and negative prediction value. The predictive ability of the tool was compared with that of past PDC values using net reclassification improvement (NRI) and integrated discrimination improvement (IDI) indices. RESULTS: Data from 7,028 MA-PD members were used for tool development. Seven predictors (age, total OAD refills, total OAD classes filled, days supply of last filled OAD, pill burden, coverage of last filled OAD, and past adherence) statistically significant in ≥ 50% of the bootstrapped samples were identified from the logistic models. The final model demonstrated good discrimination (c-statistics = 0.75) and calibration (Hosmer-Lemeshow goodness-of-fit P < 0.05) statistics, with good internal validity (area under the curve = 0.73). The RxAPT-D demonstrated adequate sensitivity statistics: sensitivity = 0.73, specificity = 0.63, positive prediction value = 0.74, and negative prediction value = 0.62. Compared with use of past adherence measures, the RxAPT-D had higher prediction ability, relative IDI = 2.09, and user defined NRI = 0.16 with 24% events correctly reclassified. CONCLUSIONS: The RxAPT is an effective tool to identify patients who are likely to become nonadherent to OADs in the follow-up year. Pharmacists in MCOs can use this tool to identify patients expected to be nonadherent to OADs and develop targeted intervention programs to assist in improving MCO CMS star ratings.

D iabetes imposes a substantial burden on the economy of the United States-the total estimated cost of diagnosed diabetes in 2012 was $245 billion. 1 Approximately 59% of all health care expenditures attributed to diabetes are for health resources that are used by the elderly population, much of which is borne by the Medicare program. 1 A higher proportion of expenditures related to diabetes complications suggests that interventions to delay or prevent the development of complications and comorbidities in elderly Americans diagnosed with diabetes may be most effective in stemming the growing economic burden of diabetes. 2 The American Diabetes Association guidelines and the European Association for the Study of Diabetes recommend use of oral antidiabetic drugs (OADs) as first-line therapy in type 2 diabetes patients. 3 However, adherence is considerably low among Medicare beneficiaries. The Centers for Medicare & Medicaid Services (CMS) reported adherence to diabetes medications among only 77% patients in 2013. 4 A previous study of diabetes patients demonstrated association between decrease in medication adherence and increase in medical cost. 5 Other studies have reported an 8.3%-28.9% decrease in annual costs with every 10% increase in adherence among Medicare diabetes patients. 6,7 As part of an effort to increase medication adherence, CMS added medication adherence measures to the star ratings • Adherence to oral diabetes medications is low among the elderly Medicare population. • Many variables obtained from patient claims data are associated with nonadherence, including demographic, medication-related, and disease factors; cost factors that can be either related to cost of medications or income; and socioeconomic status.

What is already known about this subject
• A nonadherence prediction tool for oral antidiabetic drugs users was developed using claims data routinely collected by managed care organizations. • This tool, developed using a weighted combination of significant predictors, had higher sensitivity to identify nonadherent patients in the subsequent year, compared with use of a patient's adherence history as an adherence indicator.
population. 6,13 However, to use the information for risk prediction, a validated algorithm would be required, which this study aimed to achieve. The algorithm developed for this study included variables that have been significantly associated with nonadherence to oral medication in the literature. Proactive identification of patients at risk for future nonadherence can provide managed care organizations (MCOs) with a selective cost-effective approach to implement adherence intervention programs.

■■ Methods Study Design and Data Source
A retrospective study was conducted using the Cigna-HealthSpring administrative claims data for patients residing in Southeast Texas and enrolled in the Medicare Advantage Prescription Drug (MA-PD) program from 2012-2013. Five types of data files were used: (1) membership files that contained patient demographic information, (2) member summary files that contained enrollment and other information on a per-month-per-patient basis, (3) pharmacy claims files that contained prescription medication information, (4) professional files that contained information on physician visits, and (5) institutional files that contained information on hospitalizations. Data from 2012 were used to identify the predictor variables to develop the model to predict nonadherence in 2013. Following model development and validation, weighted beta coefficients from the final prediction model were used to develop the RxAPT-D to stratify nonadherence risk.

Study Population
This study included diabetes patients (International Classification of Diseases, Ninth Revision, Clinical Modification code 250.xx) with at least 1 prescription for oral diabetes medication from any of the 5 OAD classes (biguanides, sulphonyureas, thiazolidinedione, dipeptidyl peptidase-4 inhibitors, or meglitinides); continuous enrollment eligibility over 2 years (2012 and 2013) without any gap; and aged ≥ 65 years ( Figure 1). If any of the included patients had at least 1 claim for insulin, they were excluded from the study. All the inclusion and exclusion criteria were applied in the baseline year. The sample selection criteria was according to 2015 CMS star rating guidelines for diabetes patients on specific OADs and enrolled in the MA-PD program. 4

Measurement and Operational Definition of Variables
A literature search identified variables with either a causal or a correlational relationship with adherence to oral medications. All variables were considered categorical rather than continuous.
The dependent variable was adherence to OADs measured during the follow-up year and defined as conforming to provider recommendations. 14 Medication adherence was calculated using proportion of days covered (PDC), which was calculated for all the OAD claims of each patient together and not separately for each OAD class, using the CMS program in 2012, with a high scoring weight of 3. 8 The Medicare programs are rated for adherence to diabetes medications by calculating the percentage of plan members with prescriptions for diabetes medications who fill their prescriptions often enough to cover 80% or more of the time they are supposed to be taking the medication. High proportions of adherent patients lead to higher scores on the measure, which contribute to achieving higher star ratings. Higher star ratings can lead to significant economic gains for the Medicare programs. For a Medicare Advantage plan with 1 million members, moving from a 3-star to a 5-star rating would boost revenue by approximately $200 million. 9 Beginning in 2015, only plans scoring 4 stars and above are eligible for quality bonus payments. 9 The expected difference in payments compared with a 3-star plan is about 16% per member per month, which can have a significant effect on large health plans.
In this study, we developed a risk prediction tool called Prescription Medication Adherence Prediction Tool for Diabetes Medications (RxAPT-D) to identify risk for nonadherence to OADs. Claims data have been widely cited in medical literature for health risk assessment purposes, such as developing risk adjustment models or to predict future costs, inappropriate use, or risk of a disease/comorbidity. [10][11][12] There is no algorithm, however, to predict risk of nonadherence to medications in general and OADs in particular. There are many studies that identify predictors of nonadherence to OADs and few that predict nonadherence specifically among the Medicare  cross-sectional design for PDC calculation. 15 The outcome variable was dichotomized: patients with PDC ≥ 0.8 were categorized as "adherent," while patients with PDC < 0.8 were "nonadherent." In addition to alignment with CMS calculations, use of PDC and the cutoff value of 0.8 to measure adherence is justified because of its highest predictive validity for clinically meaningful outcomes in diabetes patients. 16,17 Predictor variables were calculated in the baseline year and included demographic characteristics, medication characteristics, and health care characteristics (Appendix A, available in online article). Medication characteristics included last filled class of OAD, total number of different classes of OADs that were refilled, total number of refills of OADS, days supply of OAD that was last refilled, last OAD duration, type of OAD most refilled, pill burden (defined as the total number of pills per days 18 ), total number of therapeutic classes, average dosing frequency of OADs, and past adherence (see Appendix A for operational definitions). Health care characteristics included duel eligibility status, comorbidity burden (assessed using the Deyo adaptation of the Charlson Comorbidity Index 19 ) and adjusting outpatient claims for overestimation of comorbidities, 20 diabetes severity (assessed using the adapted Diabetes Complications Severity Index [DCSI] 21 ), emergency room visits, type of physician most visited, number of physicians visited, and type of pharmacy.
In this study, the past adherence variable was calculated as a combination of 2 variables: adherence in baseline year and time of first prescription. Adherence in baseline year was calculated using PDC as previously discussed. The time of first prescription variable was divided into 3 categories: prescribed in the last half of baseline year, prescribed in the first half of baseline year, and prescribed before baseline year, since the prediction ability of PDC was associated with the time of OAD initiation. For example, if a patient initiated a medication in 2010 and his or her adherence was 0.9 in 2012, then the patient was more likely to be adherent in 2013 as well. However, if a patient initiated a medication on October 1, 2012, with 90 days of supply, the patient's PDC would be 1.00, but we cannot be sure if the patient would continue to be adherent. In short, the time from first prescription variable was introduced to avoid overestimation of PDC for patients who started OADs in the last few months of the baseline year.
The past adherence variable had 3 categories: category 1 (good), initiation time before baseline year and PDC ≥ 0.9; category 2 (poor); initiation time before baseline year and PDC < 0.4, or initiation time in the first half of baseline year and PDC < 0.6, or initiation time in the second half of baseline year and PDC < 0.8; and category 3 (average), the rest of the patients. The cutoff for the categories were selected so as to have stronger positive (for category 1) and negative (for category 2) correlation of past adherence with future adherence. The rationale was that patients who initiated before the baseline year and had high adherence in the baseline year were most likely to be adherent in the follow-up year. Patients who initiated before the baseline year and had low adherence or patients who initiated in the baseline year and had mediocre adherence were most likely to be nonadherent in the follow-up year. It would be difficult to predict nonadherence for other patients.
Although cost of prescription medication and out-of-pocket cost were found to be important predictors in the literature, they were not used in this study, since cost is highly variable and would require yearly adjustment, thus complicating the algorithm for pragmatic use. Predictor variables with the majority (≥ 95%) of patients in 1 of the 2 categories were not used in the prediction model. The model included whether brand or generic OADs were used (majority were on generic medications), dosing frequency of OAD (majority were on twice daily), and type of physician visited (majority visited primary physician).

Development and Validation of Predictive Model
All the relevant variables identified from the literature were included for model development to avoid predictor selection bias associated with forward/backward/stepwise selection. 22 The predictor variables were identified to cause multicollenearity problems if absolute value of the correlation coefficient was > 0.7, condition index was > 30, and variance inflation factor (VIF) was > 10. 23 All the independent variables were tested for correlation using Spearman correlation statistics. If the correlation coefficient between any 2 predictor variables was > 0.7, then the variable with comparatively greater correlation to nonadherence was included in the final model.
The predictive model was developed by bootstrapping the entire sample (with replacement) and conducting multiple logistic regression analysis in each of the 200 bootstrapped samples. Variables that were statistically significant (P < 0.05) in ≥ 50% of the logistic models constituted the final predictive model. The final predictive model was validated on the original sample, as well as using the 10-fold cross-validation method, and evaluated using discrimination (C-index) and calibration statistics (Hosmer-Lemeshow test).

Development and Evaluation of Prediction Tool
The average of the median beta estimates obtained from the 200 logistic models was calculated for all variables to obtain unbiased estimates. 24 The beta (β) estimates of the statistically significant variables were then summarized for each patient using the formula L = β 1 x 1 + β 2 x 2 + β 4 x 4 + β n x n , where β 1 , β 2 ,…β n were the median beta estimates of statistically significant variables and x 1 , x 2 ,…x n were the statistically significant variables. The final risk score for each patient was then obtained using exponential conversion of the summary score (L), using the formula ( e L 1+e L ) × 100. 25 A cutoff point was calculated at a point where sensitivity, specificity, negative prediction value (NPV), positive prediction value (PPV), and total accuracy were optimal. 26 Patients with scores below the cutoff were categorized as low-risk patients, and those with score above the cutoff were categorized as highrisk patients.
The predictive tool was tested for sensitivity, specificity, positive prediction value (PPV), and negative prediction value (NPV). 26 The predictive ability of the tool was compared with that of past PDCs using net reclassification improvement (NRI) and integrated discrimination improvement (IDI). 27 The operational definitions of the parameters used to test RxAPT-D performance are provided in Appendix B (available in the online article).
Descriptive statistics of the patients were conducted for the categorized predictor variables using the chi-square (χ 2 ) test. All analyses were conducted using SAS, version 9.3 (SAS Institute, Cary, NC).

■■ Results
A total of 7,028 patients were included for RxAPT-D development and validation. Mean age of patients was 74.78 years; females were in slight majority (56.3%; Table 1). The majority of included patients had moderate diabetes severity as defined by DCSI scores of 1-3 (54.8%); were on monotherapy (63.9%, mostly metformin); and were adherent to OADs in the baseline year (59.6%).
Variables that lacked adequate patient distribution across each variable category were not included in the final analysis. These variables were type of OAD refilled, dosing frequency of OAD, and type of physician most visited. Two predictor variables-total number of therapeutic classes and class of OAD last refilled-were dropped from the prediction model because of multicollinearity. In the predictive model, VIF values of all variables were less than 10, and the condition index value was < 30, which satisfied the criteria for absence of multicollinearity. Table 2 presents multicollinearity statistics (VIF and tolerance) for variables included in the logistic regression model for algorithm development.
Seven predictor variables, statistically significant (P > 0.05) in ≥ 50% of the 200 bootstrapped prediction models, were identified: age, total number of OAD refills, total number of different classes of OADS that were refilled, days supply of OAD that was last refilled, pill burden, last OAD duration, and past adherence (Table 3). When the final logistic model was used in the original sample, c-index value was 0.75, and Hosmer-Lemeshow goodness-of-fit was P > 0.05. The area under the curve (AUC) statistics for 10-fold cross-validation was above 0.7; AUC was 0.728 for the training sample and 0.727 for the validation sample, indicating good internal validity.
The mean RxAPT-D score in the total sample was 65.91, and it ranged from 33 to 91 (Figure 2). The increase in risk score was directly proportional to the increase in nonadherent patients. Using the cutoff of 0.70, 72.9% of the nonadherent patients were correctly classified as high-risk patients, and 37.2% of the adherent patients were misclassified as high risk.
The sensitivity statistics were optimum for not only the entire sample but also for randomly chosen subsamples (

■■ Discussion
The predictive model that was developed to identify nonadherent patients consisted of 7 significant predictors: age, total number of OAD refills, total number of different classes of OADS that were refilled, days supply of OAD that was last refilled, pill burden, last OAD duration, and past adherence. The predictive model had good validity, and RxAPT-D, the predictive tool developed using the significant variables, demonstrated adequate sensitivity and specificity. All patients included in the study were enrolled in MA-PD by Cigna-HealthSpring and resided in the southeastern part of Texas. In 2012, the West South Central region of the United States, which includes Texas, was reported to have the lowest likelihood of adherent beneficiaries in the Medicare population. 28 The proportion of patients with nonadherence (PDC < 0.8) to OADs in the study sample was around 40%, which is higher than the CMS-reported national average (25% in 2012 and 23% in 2013) among MA-PD patients. 4 The high rate of nonadherence to OADs in this population highlights the need for innovative strategies to identify nonadherent patients and justifies our efforts in that direction.

Logistic Regression Model for Prediction of Nonadherence in 2013
maintain the behavior. 39 Based on this definition, patients with low (high) PDC values calculated over a longer time could be assumed as habitual nonadherents (adherents), while the adherence of patients with limited habitual history may be difficult to predict. This, along with the fact that the predictive ability of past adherence to predict future adherence increases with the increase in the length of data available about past adherence, 40 helped us segregate patients based on how accurately their future nonadherence risk can be predicted.
The study results were as hypothesized-patients with low (high) past PDC and longer medication dispense history could be accurately predicted as high (low) risk patients, compared with patients with limited medication dispense history in the past. Consideration of duration of past medication dispense history would not be required if all patients had a constant baseline PDC information of 12 months (i.e., initiated an OAD before or at the beginning of the baseline year). However, this would eliminate many patients on OADs who would be eligible for star-rating calculation.
The results of RxAPT-D performance statistics (sensitivity, specificity, PPV, and NPV) should be interpreted with the understanding that the values will vary based on the cutoff point chosen. Ideally, one would want to have a test that is highly sensitive and highly specific, but this is not always possible. When the cutoff point between low risk and high risk is changed to increase either sensitivity or specificity, there is usually a concomitant decrease in the other. The cutoff point may therefore be varied to increase sensitivity (e.g., when important not to miss diagnosis) or specificity (e.g., when false positive diagnosis can be lethal), with concomitant decrease in the other, according to what is the purpose of the test. 41 significant predictor of nonadherence to OADs was the total number of OAD classes that were prescribed in the baseline year. No other studies have tested this association. Since a patient needs to visit a physician for drug switching, it may be postulated that decreased nonadherence in these patients is a result of higher physician visits because of increased involvement in health. Considering the high predictive power of this variable with future nonadherence, it would be interesting to investigate this relationship further.
The last OAD duration variable was assessed in this study to identify patients who were exhibiting high adherence behavior until the end of the baseline year. Patients with an OAD refill that provided medication coverage up to the end of the baseline year were more likely to be adherent in the next year. This assessment seems to hinge on the fact that behavior of human beings is constantly changing so the most recent behavior is a strong predictor of preceding behavior, compared with a behavior in the distant past.
The past adherence variable did not consist solely of past PDC values, but a combination of past PDC values and if the OAD was initiated the previous year (if yes, which half of the year). Past adherence has been used in the literature to predict future adherence, 32 but this is the first study where the time frame of past adherence is also taken into consideration. The time frame of past PDC calculations was incorporated to account for 2 factors: (1) if a patient's adherence level has now become a part of the patient's habit and (2) the period of availability of past adherence information. Habit formed through repeated behavior of medication use is an important predictor of future medication use. 38 Based on habit theory, medication adherence in the long term is theorized to occur when patients repeat or practice a medication-taking routine long enough to

RxAPT-D = Prescription Medication Adherence Prediction Tool for Diabetes Medications.
patients at high risk of nonadherence to OADs. By running the 7-variable model on a prescription-filling dataset at the end of a calendar year, subjects would be identified as soon as they met the model's criteria for high risk, which can be targeted for personalized adherence interventions. Of the 7 significant variables, 1 variable is from member demographics, and 6 variables are from pharmacy claims. This makes the algorithm highly feasible and applicable, since patient demographic information is available for all patients, and pharmacy claims are up to date and, for the most part, cleaner (less error and missing data) and easier to work with than other administrative health care claims databases (i.e., medical and laboratory claims). 42 Future studies testing the predictive algorithm for external and temporal validity are required. An important next step towards external validation is to test the predictive ability of the instrument on OAD users beyond the Cigna MA-PD Texas population, by using data from other MCOs and from other parts of the country. Studies experimenting with other approaches for predictive modeling are needed. Models based on artificial intelligence using neural networks are being used in financial, legal, and actuarial sectors, as well as health care payers and disease management. Subjecting the dataset to model-building methods such as these may help identify important predictors that are otherwise overlooked. This work can be extended to develop adherence prediction tools for other medications used in the CMS star rating measurements, such as statins and antihypertensives.

■■ Conclusions
The prediction statistics identified the RxAPT-D as an effective tool to predict patients at high risk for nonadherence to OADs, with adequate sensitivity statistics, and as a better predictor of nonadherence to OADs compared with past PDC. Future studies on external validation are required.
In this study, we demonstrated that, compared with the past PDC measure, RxAPT-D more accurately classified patients with high and low risk for nonadherence. Past adherence is an easy-to-use tool compared with RxAPT-D. However, considering that RxAPT-D gives 2 times the accurate classification, implementation of RxAPT-D may provide additional benefit for nonadherence prediction.

Limitiations
The results of this study should be interpreted in light of various limitations. The algorithm suffers from retrospective claims data limitations such as coding errors and incomplete claims information. Use of a dichotomous adherence measure may decrease sensitivity. However, the decision to use retrospective data and a dichotomous measurement of adherence was aligned with CMS methods of measuring adherence to OADs for star ratings calculations, thus, increasing its applicability for MCOs. It is possible that the Medicare diabetes patients in this study have characteristics that differ from those in other settings, which could affect the generalizability of our findings.
While RxAPT-D was tested to identify patients at high risk for nonadherence, we were unable to test if these patients could improve adherence or whether targeting the identified patients would improve star ratings. The group of potential predictors that we studied was broad but not all inclusive. These predictors included variables either not available or routinely captured in the claims data, such as medication preference, perceived benefit of medication, race, socioeconomic status, and ZIP codes. Also, patients enrolled in medication adherence programs could not be identified from the data. One of the significant variables, past adherence, was created for the purpose of increasing the predictive ability of the model and is not validated. Some of the nonadherence calculated from the claims data may be because of physician recommendation for reasons such as adverse events and may falsely provide lower adherence scores.

Implications and Future Research
In spite of these limitations, a prediction tool with the ability to identify nonadherent patients is another step towards ongoing efforts to increase medication adherence by MCOs. In practice, an automated system that uses patient demographic and pharmacy claims files could identify the vast majority of