Lessons Learned from Randomized Trials and Recent Experience with Health Information Technology: Promising Interventions Meet Real-World Patient Care

“I was ordering Cortisporin, and Cortisporin solution and suspension comes up. The patient was talking to me, I accidentally put down solution, realized that’s not what I wanted. ... I would not have made that mistake, or potential mistake, if I had been writing it out because I would have put down what I wanted.”—U.S. physician, commenting in a 2003 interview on experiences in using a computerized prescription order entry system.

I n a 2010 commentary on the occasionally large gaps between the aspirations of researchers and the reality of study results, biostatistician Andrew Vickers recounted the frustrating experience of British statisticians who were trying to determine better ways to set and explode depth charges as countermeasures against German U-boats that were sinking supply convoys during World War II. 3 After months of working with data on "the direction in which the depth charge had been fired relative to the direction of the ship and then whether the submarine had been hit," the statisticians "had gotten precisely nowhere in working out how best to target depth charges"that is, until one brave analyst volunteered to go to sea and observe field conditions firsthand. Direct observation showed the statistician that battlefield winds made the firing data with which he had been working "totally unreliable." After the statistician wisely "[ignored] most of the data that he'd been sent" and performed a reanalysis using revised assumptions, battlefield tactics were adjusted successfully. 3 A recently published randomized controlled trial (RCT) of a popular health care intervention provides a potent reminder that real-world experience and rigorous tests of our suppositions can and often do produce challenges to conventional wisdom. Like the failure of statistical models based on erroneous battlefield data to produce accurate predictors of depth charge "hits," these challenges can-and should-lead to rethinking about how "field conditions" in health care affect the outcomes of managed care interventions. tion management activities. In addition to standards for data recording and transmission for 30%-80% of patients, depending on the specific patient care activity, core requirements address infrastructure necessary to meet future requirements, such as the implementation of at least 1 clinical decision support tool. 1,11 The decision support tools are notable because, as envisioned by U.S. Department of Health and Human Services (DHHS) and CMS, they reflect "the capability that undergirds much of the value of EHRs: using records to enter clinical orders and, in particular, medication prescriptions." 1 Despite popular press attributions of HIT incentives to the current Presidential administration, 9 enthusiasm for the anticipated effects of "meaningful use" of HIT on patient outcomes, especially safety, is neither new nor politically partisan. In the much-cited 1999 report, To Err Is Human: Building a Safer Health System, the Institute of Medicine identified "automated medication order entry" as a "known [system] for improving safety" that was being underutilized by hospitals. 12 The increased use who received alternative anticoagulation, a "failure to prescribe appropriate [TMP/Sulfa] prophylaxis for an otherwise critically ill patient," and a 3-day delay in "initiation of antibiotic therapy recommended by the Infectious Diseases Consultation Service." The IRB's decision to terminate the study because of potential harm to the intervention group is especially notable because in the study's initial planning stages, the IRB had expressed ethical concerns about "depriving" the control group of the patient safety intervention. 4 Strom et al. concluded that the unexpected outcome of their study "emphasizes the need for formal evaluation and monitoring of programmatic interventions rather than simply assuming that they will be effective." 4 In a subsequent interview with Reuters Health, Strom pointed to the study results as a "dramatic example" of the need to "study the side effects of these [CPOE decision support] interventions" prior to widespread implementation. 6

HIT and CPOE: What Does "Meaningful Use" Mean for Health Care Providers?
The findings by Strom et al. take on special importance in light of recent public policy developments related to health information technology (HIT), including the July 2010 announcement by the Centers for Medicare & Medicaid Services (CMS) of regulations to be enacted pursuant to the Health Information Technology for Economic and Clinical Health (HITECH) Act, a portion of the American Recovery and Reinvestment Act (ARRA) passed in February 2009. 7,8 Of the approximately $48.8 billion provided in the ARRA to encourage the development and adoption of HIT, 9 "up to $27 billion" 1 was allocated for incentive payments to Medicare and Medicaid providers that implement "meaningful use" of electronic health records (EHRs). 8 The rationale underlying meaningful use criteria, as described in a CMS summary, is that "[by] implementing and meaningfully using an EHR system, providers will reap benefits beyond financial incentives-such as reductions in errors, availability of records and data, reminders and alerts, clinical decision support, and e-Prescribing/refill automation." 8 Meaningful use will be defined by CMS in stages, with Stage 1 in effect in calendar years 2011 and 2012, Stage 2 estimated in 2013, and Stage 3 estimated in 2015. 8 Beginning at Stage 3, providers that are "not meaningful users of EHR technology" will incur Medicare "payment adjustments" (financial penalties). 10 Specific criteria for Stages 2 and 3 have not yet been announced by CMS but will "continue to expand on" the "baseline" criteria used at Stage 1. 8 During Stage 1, qualification for incentive payments will require meeting 20 of 25 meaningful use criteria, of which 15 are mandatory "core objectives" intended to represent "basic functions that enable EHRs to support improved health care" (Table 1). 1 Of the 15 core objectives, 5 pertain to medica- • Electronic recording of demographic data, vital signs, smoking status, and "chart changes" (e.g., childhood growth chart recordings)-more than 50% of patients • At least 1 entry for (a) an "up-to-date problem list of current and active diagnoses," (b) active medication list, and (c) active medication allergy list-more than 80% of patients • Clinical summary of the provider office visit given to the patient within 3 business days-more than 50% of visits • Electronic copy of discharge instructions by hospitals and emergency rooms upon request at time of discharge-more than 50% of requesting patients • Electronic copy of health information upon request within 3 business days-more than 50% of requesting patients • Generation and transmission of electronic prescriptions using certified EHR technology (physician offices only)-more than 40% of permissible prescriptions b • CPOE use for at least 1 medication in the medication list-more than 30% of patients • Reports of clinical quality measures to CMS or states • Implementation of infrastructure features, such as real-time edits to check for drug-drug and drug-allergy interactions, electronic exchange of key EHR information among providers, a privacy protection system, and at least 1 decision support rule of HIT has been enthusiastically promoted by a diverse group that includes former Speaker of the House Newt Gingrich, who proclaimed as early as 2005 that "paper kills" and described the use of paper medical records in the United States as an "utterly irrational national security risk," 13 and President Barack Obama, who declared in a February 2009 address to a joint session of Congress that investment in EHR use would "reduce errors, bring down costs, ensure privacy and save lives." 14 The popular press contribution to the chorus includes a 2009 CNN report claiming that the "first, most obvious cost saving comes from the time EHRs save just from turning them on" because, with an EHR system, "patients don't need to take the time to explain medical history to new doctors. EHRs help doctors diagnose faster, significantly cut down on the time it takes hospital staff to chart patients' information and ultimately slash the length of an average patient visit." 9

Exaggerated Cost Savings and Benefits from HIT?
Not all knowledgeable observers agree that HIT represents a longed-for "silver bullet" solution to problems of patient safety and cost in the United States. Critics of providing financial incentives for widespread EHR adoption argue that although computerized records can produce benefits, such as centralization of information and prevention of dispensing errors attributable to illegible handwriting, there is no evidence of economic benefit, let alone the "huge savings" 9 asserted by EHR proponents. 15,16 In an editorial referring to oft-cited EHR cost savings estimates as an "$80 billion exaggeration," Harvard Medical School faculty members Groopman and Hartzband accurately observe that these projected savings were based on economic modeling conducted by researchers who chose intentionally to ignore empirical evidence of problems with HIT systems in developing their estimates, a methodological approach that, in Groopman and Hartzband's assessment, "flies in the face of the scientific method." 15 Critics of financial incentives for EHR adoption have also pointed to the potential for harm due to unintended consequences that are not anticipated prior to widespread use in clinical practice, especially among providers unfamiliar with the technology. 15, 16 Groopman and Hartzband note that American "culture adores technology, so it is not surprising that the [EHR] has been touted as the first important step in curing the ills of our health-care system. But it is an overly simplistic and unsubstantiated part of the solution" based on "elegant exercises in wishful thinking." 15 The so-called "elegant exercise," a 2005 modeling study by Hillestad et al. that produced a much-vaunted estimate of $77.4 billion in annual savings nationwide due to HIT efficiencies, was based on assumptions that were acknowledged by its authors to represent "not predictions of what will happen but of what could happen with HIT and appropriate changes in health care." 17 As Groopman and Hartzband observed in their critique of the estimates, the choice by Hillestad et al.
"to interpret reported evidence of negative or no effect of HIT as likely being attributable to ineffective or not-yet-effective implementation" 17 was foundational for the large size of the cost savings estimates. 15 Moreover, in a separate technical document, the investigative group that produced the estimates transparently acknowledged marked limitations in the base of available empirical evidence about HIT: "Out of more than 1,400 screened articles, we were able to extract only 42 findings that could be used as input to our model of national savings." 18 Notably, the only RCT of EHR use in inpatient settings that was available to inform the 2005 theoretical estimates was published in January 1993 and studied one of the first EHRs in use in the United States. 18,19 Supplementing limited empirical evidence with expert opinion, Hillestad et al. estimated that 90% adoption of EHRs by providers in a mature system (i.e., in the final year of a 15-year adoption period) could result in an annual $57.1 billion in inpatient cost savings, mostly due to reductions in length of stay ($34.7 billion) and nursing time ($13.7 billion). 17 Length-of-stay reductions would accrue because EHR use would reduce "delays in the ordering process, including waiting for written orders to be transcribed and communicated; delays caused by errors, as when a needed test is inadvertently ordered late; delays in task prioritization; delays in ordering ancillary services following nursing assessment, in searching for paper documents before visiting a patient, and in coordinating all of the information and communications necessary for discharge planning." 18 Nursing time would be made more productive because the time that nurses would save on documentation by using the EHR could instead be spent "[taking] care of additional patients, keeping quality constant." 18 In both the inpatient and outpatient settings, Hillestad et al. estimated efficiency savings attributable to reductions in laboratory test use ($4.6 billion because of "the potential to reduce unnecessary tests by making physicians aware of current [laboratory test] results and by alerting them to new orders that may be superfluous"). 18 Estimated savings from using CPOE were $14.5 billion due to "structuring medication selections to align with formulary rules; advising physicians of the cost-benefit characteristics of specific drugs at the time of ordering; recommending less-expensive alternative drugs, including generics; encouraging providers to discontinue unneeded or contraindicated medications; and encouraging timely conversion from intravenous to oral medications." 18 CPOE would enhance patient safety by warning of interactions and reducing errors and, in "the longer term," would "[provide] the information needed to redesign the order-execution process so that errors become even harder to make." 17 Hillestad et al. estimated that the $77.4 billion in annual savings due to EHR efficiencies could be doubled if HIT were also used to facilitate "interventions intended to keep people healthy (or healthier)," using a series of assumptions that

Lessons Learned from Randomized Trials and Recent Experience with Health Information Technology:
Promising Interventions Meet Real-World Patient Care appear heroic in retrospect-for example, that EHR-generated reminders could increase compliance with U.S. Preventive Services Task Force recommendations to 100% and that EHR-facilitated identification of patients in need of disease management programs for asthma, congestive heart failure, chronic obstructive pulmonary disease (COPD), and diabetes could result in 100% program participation. By "controlling acute care episodes," the disease management programs would "greatly reduce hospital use at the cost of increased physician office visits and use of prescription drugs," generating "potential annual savings of tens of billions of dollars." 17 Notably, the Hillestad et al. projections were published about 2 years prior to the announcement in 2007 that the Medicare Health Support demonstration project, an RCT of disease management provided to Medicare beneficiaries, had not only failed to yield the hypothesized return-on-investment from savings on inpatient care, but had failed to yield even enough savings to cover vendors' fees, resulting in termination of the project in 2008. 20

Evidence About the Effects of CPOE on Safety and Costs
Like the RCT by Strom et al., observational studies of CPOE systems have shown that for the vast majority of prescriptions, CPOE represents an improvement over paper. [21][22][23] For example, Kaushal et al. (2010) used a prospective cohort analysis of prescribing errors in community-based office practices, comparing physicians who used paper prescriptions (n = 15 physicians, 3,684 prescriptions) with adopters of a decision-supported CPOE system that included dosing recommendations and checks for allergies, drug-drug interactions, and duplicate therapies (n = 15 physicians, 3,848 prescriptions). Error rates for e-prescribers decreased from 42.5% prior to CPOE adoption to 6.6% at 1 year post-adoption, compared with baseline and 1-year rates of 37.3% and 38.4%, respectively, in the paperprescribing group (P < 0.001 comparing 1-year error rates). 21 Using a pre-CPOE versus post-CPOE design with multivariate analysis to study prescribing error rates in a community-based multispecialty health care system, Devine et al. (2010) found a significant reduction in prescribing errors from 18.2% of 5,016 handwritten prescriptions to 8.2% of 5,153 computerized prescriptions but no significant reduction in preventable ADEs that caused harm. 22 However, underscoring the importance of the study by Strom et al., the potential for CPOE and EHRs to introduce errors-and the resulting effects on safety and cost overallare understudied in rigorous research. General weakness in the evidence base about CPOE, particularly with respect to its effect on ADE rates, was identified in 2 systematic reviews conducted by Eslami et al., one in inpatient settings (studies published through August 2006) and the other in outpatient settings (studies published through March 2006). 24,25 Both reviews identified few randomized trials, and both found inad-equate and conflicting evidence about the effects of CPOE on efficiency, health care cost, and patient safety.

Effects of CPOE in Inpatient Settings.
In the systematic review of CPOE in inpatient settings, 67 studies, including only 8 RCTs, were identified by Eslami et al. Of 4 RCTs in which cost/efficiency was the outcome, 2 produced statistically significant positive effects, 1 produced reports of positive effects with inconsistent reporting of statistical significance, and 1 produced no significant impact. 25 Of 22 nonrandomized studies, 18 suggested "a positive effect of CPOE on safety" as measured by prescribing errors. However, no randomized studies of the effect of CPOE on prescribing errors were identified, and there were no studies (using any research design) of ADEs at all. 25 More recent observational economic analyses of the use of HIT, including decision-supported CPOE, in "real-world" hospital settings have generally not supported projections of cost savings, often (but not always) finding modestly (usually less than 10%) higher costs associated with EHR use with or without decision-supported CPOE. [26][27][28] However, the base of evidence about the effects of HIT remains remarkably limited. Current (October 2010) PubMed searches limited to RCTs, conducted using combinations of a number of terms ([a] "hospital" or "inpatient" with [b] "cost" and [c] "electronic health record," "electronic medical record," "computerized prescription order entry," or "CPOE"), yield only 1 RCT of the economic impact of EHRs or CPOE in inpatient settings-the 1993 RCT that was used by Hillestad et al. in their 2005 EHR cost savings estimates. 18,19 Another systematic assessment of the effects of CPOE on prescribing errors in inpatient settings was provided in a systematic review by Reckmann et al. (2009) of studies published from 1998 through October 2007. 29 Overall, 9 of the 12 studies identified significant reductions, ranging from 29%-96%, in prescribing error rates with CPOE. However, 7 studies employed weak pre-versus-post designs and none was randomized. The authors noted that the base of evidence "reporting the effectiveness of CPOE to reduce prescribing errors is not compelling and is limited by modest study sample sizes and designs," concluding that a "stark comparison might be drawn between the quality of evidence on e-prescribing systems and the investment that is made in ensuring that drugs and other medical devices are safe and effective before wide-spread use." 29 Effects of CPOE in Outpatient Settings. In the systematic review of CPOE with decision support in outpatient settings, Eslami et al. identified 12 studies, including 4 RCTs, of cost/ efficiency. 24 Positive outcomes were documented in 5 of the 12 studies overall. However, no randomized studies found a positive effect of CPOE with decision support on cost/efficiency; 3 of the 4 RCTs found no statistically significant effect, and 1 reported that the effect of decision-supported CPOE on cost/ efficiency was negative (i.e., higher cost with decision support). That study, reported by Tierney et al. (2005), examined the effect of providing care suggestions based on "published evidence-based guidelines" for the treatment of asthma and COPD to providers based on patient-specific data as a supplement to an existing CPOE system. 30 The finding of higher costs with decision support was inconclusive because of "a small number of extremely high-cost hospitalizations" in 1 cohort. 30 Eslami et al. identified only 4 studies of medication safety in outpatient CPOE, of which only 1 was randomized. 24 All 4 assessed CPOE combined with clinical decision support: (a) The sole RCT, reported by Rotman et al. (1996), identified no significant effect for CPOE on the rate of clinically significant drug interactions, comparing decision-supported CPOE with paper prescriptions; however, the trial has no applicability today because it was conducted in 1994-1995 and because the intervention group used CPOE for only 2.8% of prescriptions. 31 (b) A pre-post assessment without a comparison group, conducted by Steele et al. (2005), found that implementation of an alerting system for drug-laboratory problems (i.e., notification of a missing test or an abnormal test value) was associated with statistically significant increases in ordering of the necessary laboratory test (from 38.5% to 51.1%, P < 0.001) and stoppage of the medication ordering process (from 5.6% to 10.9%, P = 0.03) but no significant change in definite or probable ADEs defined by Naranjo scoring (from 10.3% to 4.3%, P = 0.23). 32 (c) An observational analysis by Weingart et al. (2003) found that physicians overrode approximately 90% of drug allergy and high-severity drug interaction alerts, with no significant difference in ADE rates for overrides versus accepted alert cases. 33 (d) A prospective cohort study by Gandhi et al. (2003) found no significant differences in either prescribing errors or ADEs comparing computerized with handwritten prescribing. 34 However, the CPOE sites in the Gandhi et al. study used "basic" computerization only, without checks for allergies and drug interactions. 35 A later, more detailed examination of the 2003 Gandhi et al. study data, conducted by physician reviewers, indicated that CPOE with advanced decision support (e.g., drug-dose checking and drug-frequency checking) could have prevented an estimated 138 of 143 prescribing errors (97%) and 59 of 62 potential ADEs (95%). 35 A current (October 2010) PubMed search on the terms "CPOE" and "computerized prescription order entry" with "safety" limited to RCTs produces only 3 studies, all conducted in the same large health maintenance organization (HMO) in 2006 and 2007. [36][37][38] When all HMO members aged 65 years or older were randomized to a CPOE-based alerting system that triggered when any of 11 potentially inappropriate medications for the elderly were prescribed (n = 29,840) or to a "usual care" control group (n = 29,840), Raebel et al. found that the rate of potentially inappropriate dispensings per 100 patients dur-ing the 1-year follow-up was 1.85, compared with 2.20 in the control group (difference of 0.0035 dispensings per patient, P = 0.002). 36 However, in a similarly designed study targeted to pregnant women, the intervention was successful in reducing the use of pregnancy-risk medications but had to be discontinued because of false-positive alerts attributable to computer system limitations. 37 Additionally, an RCT of CPOE-delivered reminders found no significant effect on provider adherence to recommended laboratory testing guidelines at initiation of pharmacotherapy. 38 Commenting on the unfavorable results of their study of the effect of decision-supported CPOE on outcomes for patients with asthma/COPD in primary care, Tierney et al. reported that they "had hoped to show that computer-generated care suggestions would enhance adherence to evidence-based guidelines and have salutary effects on patient-centered and clinical outcomes. Unfortunately, none of this occurred, which surprised us …" 30 Noting that previous studies had shown more positive outcomes for interventions in which physicians were given suggestions to increase preventive care or use of cost-effective treatments, Tierney et al. concluded that despite the IOM's strong support for EHR use as a care improvement mechanism, "expensive and sometimes intrusive innovations need to be thoroughly tested in rigorous trials before broad implementation." 30

Evidence of "e-Iatrogenesis" in Published Research
The report by Strom et al. was not the first to identify patient care problems attributable to use of HIT. 2,[39][40][41] In 2004, Ash et al. reported the results of exploratory observational research including formal interviews with medical staff to assess the effects of patient care information systems (PCIS) on workflow and accuracy in hospitals located in Australia, the Netherlands, and the United States. 2 Although acknowledging the "promise" of PCIS to facilitate appropriate care, the researchers reported "unintended consequences" attributable to "a mismatch between the functioning of the PCIS and the real-life demands of health care work." Problems identified by Ash et al. included (a) juxtaposition error, in which the wrong medication was ordered because "something is close to something else on the screen and the wrong option is too easily clicked in error;" (b) new patient safety risks because of staff "workarounds" introduced in response to system inflexibility; and (c) "loss of overview" in which "the user had to switch among multiple windows" to obtain information about a patient.
For example, one U.S. clinician interviewed by Ash et al. reported that: "You would order [medication] on one patient and it would, [because] of the vagaries of the light pen system, you thought you were ordering it on one, and it was really ordered on somebody else and somebody got the wrong medication and that sort of thing." 2 Similarly, a U.S. pharmacist described an instance in which a drug that had been The median time since system implementation among study hospitals was 5 years (range 6 months to 25 years); and a median of 91% of provider orders were electronic. Additionally, Spearman's rho correlations showed no statistically significant relationships between responses to any of the 8 items and time since CPOE implementation. 41 Like Ash et al., Koppel et al. (2005) used a combined qualitative-quantitative method to assess medication errors attributable to a widely used CPOE system in a tertiary-care teaching hospital from 2002 to 2004, approximately 5 to 7 years after CPOE implementation. 42 Using a variety of data-gathering techniques including structured interviews, real-time observation, focus groups, and written questionnaires completed by house staff, Koppel et al. identified 22 "previously unexplored medication-error sources that users report to be facilitated by CPOE." The authors grouped these into 2 major categories, (a) information fragmentation and failure of systems integration and (b) human-machine interface problems.
Information fragmentation and systems integration failures that survey respondents reported as occurring at least once in the previous 3 months included use of CPOE displays by 73% of respondents to identify minimally effective or usual dosages, whereas the display actually portrayed dosages that "are based on the pharmacy's warehousing and purchasing decisions, not clinical guidelines," that is, the strength of the tablet stocked by the hospital pharmacy. 42 Koppel et al. also identified delays in cancelling medications because of an interface that required using multiple screens to change a single patient's medications (reported by 51% of respondents) and gaps in antibiotic therapy because of the absence of reminders that the system required ordered 3 times daily had been discontinued after a dose of the medication had already been administered, but the system would not permit the nurse to record the dose "because the system considered it an incomplete execution of the task." 2 Providers also reported information overload because of an excessive volume of alerts and problems arising because of the tendency of users to "cut and paste" information from one patient record to another.
Noting that their evidence was qualitative, not quantitative, Ash et al. called in 2004 for additional research into "subtle silent errors" in which "the intended strengthening of one link in the chain of care actually leads unwittingly to a deletion or weakening of others." 2 Subsequent work published by the same research team in 2006 and 2007 first used observation and semi-structured interviews at 5 U.S. hospitals to identify a framework for classifying major categories of unintended consequences attributable to "computerized provider order entry" (encompassing CPOE and other electronic orders), 40 then produced quantitative information about the rates of occurrence of each consequence type based on telephone interview data obtained from representatives of 176 U.S. hospitals (response rate = 47%). 41 Based on the earlier qualitative work, the 8 major categories identified by the research team for inclusion in the hospital survey were (a) additional workload for physicians to "enter required information, respond to alerts, deal with multiple passwords, and expend extra time;" (b) workflow problems because of mismatches between system requirements and work routine; (c) ongoing "demands" for more and improved hardware and software; (d) "illusion of communication" problems because "people think that just because the information went into the computer the right person will see it and act on it appropriately;" (e) negative staff emotions; (f) new errors that threaten patient safety, such as juxtaposition errors and information overload; (g) a shift in power (i.e., removing some autonomy from physicians); and (h) "overdependence on technology," leading to lost productivity in the event of system failure. 41 Categories rated by 80% or more of respondents as "moderately to very important" (defined by the research team as a response of "yes" to indicate that the problem existed plus a rating of 3, 4, or 5 on a Likert-type scale of 1 ["not at all important"] to 5 ["very important"]) included workflow problems, communication problems, overdependence on technology, system demands, and negative emotions. Increased workload, new kinds of errors, and power shift were rated as moderately to very important by 72%, 47%, and 36% of respondents, respectively. 41 Two aspects of the 2007 survey by Ash et al. are notable. First, questions were framed in a way that suggested to respondents that the problems existed before asking whether they were an issue for the respondent's institution. For example, the question about emotion asked: "We have seen many emotional FDA seeks to support the benefits that HIT can bring through improvements in individual patient care and the overall healthcare system," Shuren highlighted what he described as "serious safety concerns" attributable to HIT malfunctions and human interface problems. 45 Shuren testified that according to the provisions of the Federal Food, Drug, and Cosmetic Act, "HIT software is a medical device." Although the FDA has "largely refrained from enforcing our regulatory requirements with respect to HIT devices," limited data on patient safety are available because some HIT vendors and users voluntarily report HIT-related adverse events to the FDA using the Medical Device Reports (MDR) system that is typically required of device manufacturers. Analysis of these data by the FDA for the previous 2 years identified "260 reports of HIT-related malfunctions with the potential for patient harm-including 44 reported injuries and 6 reported deaths." 45 Examples cited by Shuren included the placement of a nuclear medicine study in the wrong patient's electronic file because of a software error; an "improper database configuration" that "caused manual patient allergy data entries to be overwritten during automatic updates of patient data from the hospital information system;" failure to display patient allergy information properly because of "a missing codeset;" and a vendor interface problem that caused computerized tomography images to "flip" (i.e., display in mirror image). Because the data were reported voluntarily, Shuren added that the incidents "may represent only the tip of the iceberg in terms of the HIT-related problems that exist." 45 Less than 2 weeks after Shuren's testimony, the Veterans Health Administration (VA) issued a patient safety alert indicating that access to all electronic Department of Defense (DoD) records had been disabled on March 1, 2010, because of the "potential for incorrect or incomplete display of [DoD] records" when using the VA's EHR system. 46 The problem had been discovered when a clinician had accessed the EHR of a female patient and noticed that it contained a prescription for vardenafil, which actually had been ordered for a different (male) patient. Investigation revealed that EHR queries "may display no data, a subset of data, incorrect data, or the complete data. The VA clinician may see the patient's data during one session, but another session may not display the data previously seen. This problem occurs intermittently and has been reported when querying DoD Laboratory, Pharmacy, and Radiology Reports." 46 Anecdotal reports of system shutdowns and user interface problems in other EHR systems are emerging among policy blog writers 47 but at this writing have not been explored extensively in the popular press or peer-reviewed literature.

CQI for CPOE
A potentially important but seldom discussed benefit of CPOE is the provision of information not just during the act of reapproval of antibiotics every 3 days (83% of respondents). 42 Human-machine interface flaws identified by Koppel et al. included difficulties in identifying a patient from the CPOE display (55%), uncertainty about the current medication list because it appeared on up to 20 screens (71%), delays in ordering medications because of system shutdowns (84%), and system inflexibility in "specifying medications or ordering off-formulary medications" (92%). 42 The percentages of respondents reporting that problems occurred "about a few times a week" or more often during the past 3 months were lower, but still troubling-for example, 40% for gaps in antibiotic therapy, 55% for difficulties in ordering medications because of system inflexibility, and 46% for uncertainty about patients' medications because of display problems. 42 An editorial in JMCP more than 5 years ago highlighted the important work of Koppel et al. regarding the potential for CPOE to introduce harm and urged the application of the principles of continuous quality improvement (CQI) prior to widespread adoption of CPOE and clinical decision support systems. 43 In their 2009 systematic review of inpatient prescribing, Reckmann et al. found that despite generally favorable results for CPOE, several errors were more likely with CPOE than with paper-based prescribing. 29 These included duplicate prescription orders because of "fragmented" screen design (i.e., the inability to view all medications simultaneously), selection of inappropriate or erroneous products or dosages from dropdown menus, and missed drug allergies. 29 In commenting on the work of Ash et al., Weiner et al. (2007) coined the term "e-iatrogenesis" to describe "patient harm caused at least in part by the application of [HIT]." 44 Noting the increasing importance of examining unintended consequences "as CPOE and other components of [HIT] logarithmically diffuse across the U.S. health care system," Weiner et al. described their work with a small consortium of HIT adopters to develop and improve HIT: "Universally, we are hearing reports that e-iatrogenesis, and the broader area of unintended consequences, is [sic] of concern at all of these topnotch organizations. What will happen as HIT is rolled out at organizations further down the diffusion curve?" 44  45 That Shuren's testimony raised concerns about HIT was ironic because the committee before which he testified had been created as part of the HITECH act to promote EHR use. 7 Although emphasizing that "the ranging from less to more intensive: (a) required registration of HIT devices and submission of MDRs to the FDA to enable postmarketing surveillance if needed; (b) required registration and submission of MDRs plus adherence to the FDA's Quality Systems Regulation (QSR) process for "minimum guidelines to assure the quality and consistency of products on the market" (e.g., procedures for handling complaints and for correcting problems); or (c) application of the FDA's "traditional regulatory framework, in which HIT device manufacturers would be required to meet all the same regulatory requirements as other, more traditional devices, including risk-based premarket review" of "the safety and effectiveness of high-and mediumrisk HIT devices before they go into market use." The third option might also include FDA approval of installation plans, hazard analysis, product label, or postmarket studies. 45 On April 21, 2010, the DHHS HIT Policy Committee made recommendations to David Blumenthal, the DHHS National Coordinator for Health Information Technology, to address the "vitally important topic" of "patient safety related to the use of [EHRs]." 52,53 Foremost among these were the adoption of a "national, transparent oversight process and information system" including "standardized data reporting formats that facilitate analysis and evaluation" and "a formal study to thoroughly evaluate HIT patient safety concerns, and to recommend additional actions and strategies to address those concerns." 52 These suggestions are consistent with the spirit of the ongoing CQI efforts within the CPOE industry. 48 However, a safety monitoring system established according to the committee's suggestions would likely require additional features in EHR and CPOE devices, as well as broader criteria and cultural changes for reportable events. For example, recommendations included certification criteria for EHRs incorporating "functionality that makes it easier for clinician-users to immediately report any problems/concerns with information that appears on screens (a 'feedback button');" the "[capacity] to monitor" not only actual adverse events but also "near-miss patient harms;" and "whistle-blower" protection for those reporting HIT-related adverse events. 52 At the same time, however, the committee expressed concerns about the "potential for increased [FDA] regulation of EHR systems." 52 These concerns included the recognition that HIT-related problems can occur even in a properly functioning device (i.e., human interface issues that might fall outside FDA's jurisdiction); the possibility that the costs of compliance with FDA regulation could be "a barrier to entry for small vendors;" and inconsistency between the FDA's QSR process and "the incremental nature of HIT development. … By hampering and slowing the ability of vendors to continuously improve systems, thus making them safer, such a process could actually work against the safety efforts we are proposing." The committee instead suggested collaboration between DHHS and the FDA on safety matters, including EHR certification criteria.

Limited But Growing Evidence of Challenges in "Real-World" Use of HIT
prescribing, but also in databases generated from the electronic prescriptions themselves. These databases permit ongoing nationwide monitoring and quality improvement "in a way that is simply not possible with paper … [representing] what amounts to a national air traffic control system for prescriptions," according to an industry quality officer. 48 The CPOE industry is moving toward a system in which information gathered from these databases can enable timely identification and correction of specific quality problems.
For example, monitoring studies using CPOE databases have pointed to discrepancies between the contents of "free text" and "structured" (e.g., pre-specified drop-down menu) fields in electronic prescriptions as a source of potential new safety problems. 49,50 In a retrospective review of more than 400,000 electronic prescriptions in a large outpatient health care system, Palchuk et al. (2010) found that 42.9% contained a free-text instruction. Of 2,914 prescriptions with free-text instructions that were sampled, 470 (16.1%) contained at least 1 discrepancy and 79 (2.7%) contained more than 1 discrepancy. Assessment by pharmacist reviewers indicated that 394 (83.8%) of the 470 discrepant prescriptions could lead to an ADE; these included under-or overdosing of up to 55-fold, wrong route (e.g., intravenous vs. intravaginal), and dose escalation or tapering problems. Reviewers judged that 79 of the 470 discrepancies (16.8%) had "the potential for an ADE severe enough to lead to a hospital admission and/or death." 50 Data of this type have permitted industry monitors to begin honing in on configurations that can foster quality problems. 48 The e-prescription network SureScripts issued guidelines for creating high-quality electronic prescriptions in ambulatory settings in 2010 and plans a future publication "that will address issues such as root-cause analyses of e-prescriptions that do not meet these guidelines, as well as outline preventative and corrective measures to mitigate any problems that such analyses identify." 51 Noting that "experience has shown that a number of prescribing data fields are particularly susceptible to improper use," the guidelines remind prescribers of the importance of consistency across fields, standard naming conventions, watching for truncation of fields when entering free text, and use of appropriate numeric measures. The latter category is particularly notable because of the potential for dosing errors; for example, the guidelines point out that trailing zeros (e.g., X.0 instead of 0.X in specifying tablet strength) can lead to 10-fold overdoses, and use of the abbreviation "ug" instead of "mcg" can lead to misinterpreting the intended "microgram" as "milligram. The committee also noted its position "that the biggest risk to patient safety would be to either avoid or delay the proper implementation of EHR and CPOE systems." 52 On April 29, 2010, in an apparent early response to the committee's recommendations, Blumenthal told attendees at a national conference on HIT that a "preliminary investigation" into the concerns raised by the FDA's Jeffrey Shuren in February 2010 had shown that the evidence was "anecdotal and fragmented." 54 The logical flaw in using the anecdotal nature of voluntarily reported safety evidence as a reason to reject a mandatory reporting process is obvious. Nonetheless, at this writing, no decision about the HIT committee's recommendations to protect patient safety has been announced.

Hubris, Humility, and Hippocrates
In health care policy making, as in any activity, it is tempting to assume that all our good ideas will yield the desired outcome-but often, they don't. It is also tempting to assume that it is acceptable to put well-intended interventions into practice without first subjecting them to rigorous testingbut often, this strategy results in unintended and undesirable consequences.
The truth is that we don't know everything about the effects of HIT (or any health care intervention), much as we would like to. More RCTs-the best way to distinguish supposition from sound policy decision making-are needed to assess the incidence and types of errors for various HIT configurations and decision support features as rapid adoption of these potentially practice-changing technologies moves forward. The limited evidence currently available suggests that a key issue may be the tradeoff between strength and flexibility-that is, how to design a system that harnesses the power of data in providing actionable safety-enhancing information to clinicians (e.g., treatment guidelines or drug interaction warnings) while permitting overrides of inappropriate care suggestions or allowing for instructions in free-text fields to handle nonstandard patient care situations. Additionally, like any good tool, HIT requires ongoing monitoring for gaps between ideal use and real-world use. In expansion of HIT to include decision support, especially "nearly hard stop" features such as that studied by Strom et al., perhaps a key guiding principle should be primum non nocere (first, do no harm).