Standardization Is Necessary in the Methods to Assess the Value of Electronic Prescribing Systems

Standardization Is Necessary in the Methods to Assess the Value of Electronic Prescribing Systems In the May 2005 issue of JMCP, we reported that an electronic prescribing system with integrated clinical decision support was effective at improving prescribing behavior and lowering prescription costs. In contrast, Ross et al. reported in the June 2005 issue of JMCP that a different electronic prescribing system (without clinical decision support capabilities) had no impact on generic drug utilization or formulary compliance.


ss Standardization Is Necessary in the Methods to Assess the Value of Electronic Prescribing Systems
In the May 2005 issue of JMCP, we reported that an electronic prescribing system with integrated clinical decision support was effective at improving prescribing behavior and lowering prescription costs. 1 In contrast, Ross et al. reported in the June 2005 issue of JMCP that a different electronic prescribing system (without clinical decision support capabilities) had no impact on generic drug utilization or formulary compliance. 2 When designing our study, we also considered assessing changes in generic prescribing rates as a surrogate marker for cost savings but, instead, we decided to focus on the actual differences in prescription costs. Since publishing our results, we have been asked many times about the impact of our system on generic drug utilization and have been involved in subsequent study design discussions in which customers wanted to focus primarily on this outcome measure. Given the ongoing interest in evaluating the impact of electronic prescribing systems, we decided to assess the changes in generic drug utilization observed in our study by conducting a post hoc analysis of the new prescriptions dataset. These results (1) highlight a potential limitation of prior studies assessing electronic prescribing systems, (2) confirm that an electronic prescribing system with integrated clinical decision support can increase generic prescribing rates, and (3) demonstrate how focusing only on savings related to increasing generic drug utilization may substantially underestimate the total savings associated with electronic prescribing.
Some studies attempting to quantify the impact of electronic prescribing systems have compared outcomes for physicians using the system with a group of traditional prescribers over a single time frame. 2,3 For example, Ross et al. retrospectively compared physicians using an electronic prescribing system with a group of closely matched controls and found no difference in generic drug utilization during a 12-month evaluation period (37.3% versus 36.9%). While these studies can provide useful information, the results are somewhat difficult to interpret without also knowing if the groups had similar prescribing practices at baseline (i.e., prior to implementation of the electronic prescribing system).
Potential pitfalls related to comparing groups from a single time period can be demonstrated by examining the generic drug utilization data from our study. During our 12-month follow-up period, from June 2002 through May 2003, the generic dispensing rates were 55.2% in the electronic prescribing group and 48.7% in the control group. If we had looked only at this time period, we would have incorrectly assumed that the electronic prescribing system resulted in a 6.5% difference in generic drug utilization.
When baseline generic drug utilization is considered, the results show that the generic dispensing rate increased from 50.8% to 55.2% (8.7% relative increase) in the intervention group and from 48.2% to 48.7% (1.0% relative increase) in the control group. To correct for the difference in baseline generic dispensing rates, the data can be adjusted using the methodology  proposed by the Centers for Medicare and Medicaid Services physician group demonstration project. 4 This methodology assumes that the change observed in the control group represents normal variation (i.e., what would have been expected in the intervention group if they had not started using the electronic prescribing system). In this case, the 1% relative change observed in the control group is applied to the intervention group' s baseline value of 50.8% to achieve an expected generic dispensing rate of 51.3% (i.e., 101% of the 50.8% baseline generic dispensing rate). The difference between the intervention group' s expected generic dispensing rate (51.3%) and their actual generic dispensing rate (55.2%) represents a net absolute increase of 3.9 points.

Savings on Brand-Name and Generic
Although these results confirm that the system was effective at increasing generic prescribing, as confirmed by the generic dispensing ratio, it is important to recognize that the savings related to generic drug utilization account for less than half of the total savings. As demonstrated in Table 1, the 3.9% higher generic dispensing rate in the intervention group translates into 824 fewer brand-name prescriptions being dispensed during the study period. While each avoided brand-name prescription saved an average of $76.55 (total savings: $63,077), these savings were partially offset by an increased utilization of generic medications (total cost: $21,290); therefore, the final savings associated with increased generic drug utilization was $41,787.
To estimate the final savings, however, differences in the average cost of brand-name and generic medications should also be examined. Our data suggest that the system was not only effective at decreasing utilization of brand-name prescriptions, but it also had a significant impact on the cost of each remaining brand-name prescription. The average cost of brand-name prescriptions was $5.86 (7.7%) lower in the intervention group; therefore, interventions that focused on improving prescribing behavior within these drug categories saved an additional $70,097. These savings were most likely related to electronic messages that focused on improving brand-to-brand conversions or helping clinicians choose the most cost-effective dosing regimens. These cost-related savings on brand-name medications were partially offset by a $0.44 higher cost for each generic prescription (total cost: $5,915); therefore, the total savings attributed to price differences for both brand and generic drug claims was $64,182.
Combining the savings estimates from interventions affecting utilization ($41,787) and price ($64,182) results in total savings of $105,969 (8.8%), or an average of $465 per prescriber per month. Because all calculations in the post hoc analysis were based on data adjusted for differences in per-member-permonth (PMPM) utilization, these savings are almost identical to the savings that we originally estimated based on the PMPM costs for new prescriptions. 1 We hope that these results will encourage future researchers to assess actual savings related to differences in the cost and utilization of brand and generic medications rather than simply to focus on changes in generic utilization. We also encourage future researchers to identify closely matched controls based on data gathered prior to the implementation of the electronic prescribing system because we believe that comparing 2 groups of clinicians during an isolated time frame may result in spurious conclusions.

DISCLOSURE
The author is an employee of Purkinje, distributor of the electronic prescribing system that was the subject of this study.