HTA’d in the USA: A Comparison of ICER in the United States with NICE in England and Wales

BACKGROUND: The Institute for Clinical and Economic Review (ICER) has gained recognition for performing independent health technology assessments (HTAs) that include the cost-effectiveness of selected new technologies in the United States. ICER has similarities with the National Institute for Health and Care Excellence (NICE) in England and Wales, but the amount of overlap and new methods adopted to meet stakeholder needs in the complex U.S. health care system have not been fully analyzed. OBJECTIVE: To perform a comprehensive comparison of ICER and NICE. METHODS: We compared ICER and NICE using the same framework as Drummond et al. (2008), which suggests 4 dimensions for comparison of HTA organizations: structure of HTA programs, methods of HTA, processes for conduct of HTA, and use of HTAs in decision making. RESULTS: We found differences between ICER and NICE in the structure of HTA programs (setup of the organizations, governance issues, and funding); methods (perspective, costs, utilities, discounting, and thresholds); process (relationship with relevant stakeholders, deliberative decision-making processes, and timelines); and the use of HTA in decision making (the format and type of evidence generated, how the evidence is considered, and the format of the recommendations). CONCLUSIONS: ICER uses a different approach for clinical review but performs cost-effectiveness analysis using methods similar to NICE. The key differences between NICE and ICER arise because of important differences between the United Kingdom’s “single payer” health care system and the United States’s pluralistic system. ICER’s lack of mandatory power translates to substantial differences in terms of its processes and type of recommendations.

I n recent years, the Institute for Clinical and Economic Review (ICER) has gained recognition for performing independent assessments of cost-effectiveness of new technologies in the United States as part of its health technology assessment (HTA) reports, in addition to assessing clinical evidence using traditional evidence synthesis methods. Its recommendations are increasingly used to inform decisions by U.S. health payers, which include commercial insurance companies, state Medicaid agencies, and pharmacy benefit managers (PBM), such as CVS Health, one of the largest PBMs in the United States.
ICER uses incremental cost per quality-adjusted life-year (QALY) gained to estimate the cost-effectiveness of new technologies. 1 The National Institute for Health and Care Excellence (NICE) is one of the most well-known organizations to use this cost per QALY approach. It has been suggested that ICER is indeed modeled after NICE, given their similarities in undertaking clinical and economic reviews and the use of cost per QALY. 2 While previous efforts at HTA, which included the use of cost-effectiveness evidence in the United States have not lasted, [3][4][5][6] ICER has thus far been resilient with increasing relevance in health care decision making.
Given the influence of NICE on HTA in the United States and on pricing discussions, the connected history of NICE and ICER, and the level of U.S. discourse about HTA and cost-effectiveness, it is of interest to explore the amount of • The Institute for Clinical and Economic Review (ICER) is a nonprofit organization that produces reports with clinical and cost-effectiveness evidence for use by health payers in the United States. • It has been suggested that there are similarities between ICER and the National Institute for Health and Care Excellence (NICE) in England and Wales, one of the most well-known organizations to use incremental cost per quality-adjusted life-years (QALYs) for cost-effectiveness.

What is already known about this subject
• A comprehensive comparison of ICER and NICE is provided that focuses on the structure of health technology assessment (HTA) programs, methods of HTA, processes for conduct of HTA, and use of HTAs in decision making. • ICER is similar to NICE in its use of the incremental cost per QALY metric and in its methods for undertaking clinical and economic reviews; however, there are some key differences that arise because of important differences between the United Kingdom's "single payer" health care system and the United States's pluralistic system. • Despite the lack of mandatory power, ICER has successfully managed to create and adapt an HTA process tailored to provide relevant clinical and cost-effectiveness evidence to U.S. health care payers.
Organizational Structure. NICE is a governmental organization that makes recommendations regarding the use of technologies in a single payer health care system, whereas ICER is an independent, nonprofit, nongovernmental organization that produces recommendations for value-based price benchmarks. 23 Governance and Funding. NICE is publicly funded and receives most of its money from the Department of Health but overlap between these 2 organizations and the new approaches that have been adopted to meet the unique stakeholder needs in the complex U.S. health care system. The few articles that have compared ICER with NICE are either brief summaries 7 ; conference presentations 8 ; or relate to specific issues, such as recent updates at ICER and NICE 9 ; or focus on specific aspects, such as price implications of cost utility and budget impact methodologies. 10 To our knowledge, there is no published study that has performed a comprehensive comparison of ICER and NICE, which is the aim of this study. The scope of the comparison was mainly restricted to pharmaceuticals and relates to evaluation of single technologies (i.e., single technology appraisal [STA] process) at NICE, although key issues associated with the multiple technology appraisal (MTA) process are also highlighted, where appropriate.

■■ Methods
We used the organizing framework set out by Drummond et al. (2008), 11 which suggests 4 dimensions for comparison of HTA organizations: structure of HTA programs, methods of HTA, processes for conduct of HTA, and use of HTAs in decision making.
The methods to support the comparison of ICER and NICE involved complementary approaches: review of documents, personal experience, and communication with key stakeholders.

Review of Documents
Key documents relating to HTA methods and processes from the ICER and NICE websites were downloaded and reviewed. The

Structure of the HTA Program: Overview of the Comparison Between ICER and NICE
is independent from the decision-making body. Currently, NICE has an annual (as of 2019-2020) HTA budget of GBP (£) 12.6 million, roughly 167 whole time equivalent staff, 24 publishes around 50 appraisals each year, 25,26 and charges the manufacturers for appraisals (£126,000 for an STA, with small companies receiving a 75% discount). 27 ICER receives most of its funding from not-for-profit foundations, typically as short-term grants, with a 3-year, $13.9 million grant from the Laura and John Arnold Foundation in 2017 being the biggest thus far. 28 Currently, ICER has around 30 staff and typically performs less than 10 assessments each year.
Use of External Independent Organizations. NICE works with evidence review groups (ERGs), which are mainly independent academic organizations that review the clinical and cost-effectiveness evidence submitted by the manufacturer of the technology under review. ICER, on the other hand, works with independent academic organizations to develop the clinical and cost-effectiveness evidence.

Methods of HTA
The comparison of methods used is outlined in Table 2. These methods relate to type of clinical and cost-effectiveness evidence, including the consideration of elements of costeffectiveness analysis.
Clinical Evidence. NICE requires that evidence on outcomes be obtained from a systematic review. In the case of STAs, these reviews are provided by the manufacturers and are then critiqued by ERGs. In the case of MTAs, ERGs put together the evidence. ICER uses systematic review methods and evidence synthesis techniques and also assesses the strength of the entire body of evidence available using its Evidence Rating Matrix.
Utilities. NICE and ICER suggest EQ-5D as their preferred measure for estimating utility values, with other measures (such as SF-6D) or mapping studies to be used in the absence of EQ-5D values. However, U.S. tariffs are preferred for ICER evaluations, and NICE recommends the use of UK tariffs. 29,30 Costs. NICE recommends the use of list prices of drugs or prices estimated from the drugs and pharmaceutical electronic market information tool (eMIT), 31 the National Health Service (NHS) reference costs for hospital-based care, 32 and the costs reported by the Personal Social Services Research Unit (PSSRU) for health and social care costs. 33 All costs are converted into the current year pound sterling using the PSSRU health services index.
ICER recommends the use of RED BOOK and SSR Health for drug costs and publicly available data for other health care  Perspective. NICE uses a health and social care perspective, that is, the costs that fall on health care and social care budgets (e.g., costs of community care) are included in the estimates of incremental cost per QALY. Other costs, such as informal caregiver costs, can be noted if they are considered particularly important. ICER's reference case for economic evaluations uses a health system perspective and a modified societal perspective as a "co-base case" when the societal costs of care (e.g., productivity, caregiver costs, and educational costs) are large relative to the direct health care costs, and the effect of treatment on these costs is substantial. Discount Rate. NICE recommends a discount rate of 3.5% per annum for costs and QALYs, while ICER recommends a discount rate of 3% per annum for costs and outcomes.
Cost-Effectiveness Thresholds. NICE typically uses a threshold of £20,000-£30,000 per QALY, with a threshold of £50,000 per QALY for technologies that meet end of life criteria, 41 and a higher threshold of between £100,000 and £300,000 per QALY when appraising technologies for rare diseases. ICER uses $100,000 and $150,000 per QALY to estimate the value-based price benchmarks (recently renamed as health benefit price benchmarks) and provides incremental results up to $200,000 per QALY and per equal value of life years gained (evLYG).
Budget Impact Thresholds. If the estimated budget impact of adopting a given technology approved by NICE exceeds £20 million in any of the first 3 years, NHS England may engage in commercial discussions with the manufacturer to mitigate the effect that funding the technology would have on the rest of the NHS (e.g., by seeking a longer implementation period). ICER also uses an annual budget impact threshold ($819 million/ year for 2019-2020), designed to alert policymakers that funding the new technology may be difficult without displacing other needed services or increasing the health care insurance premiums at a rate faster than growth in the overall national economy. Table 3 compares the process elements of ICER and NICE, including timelines, relationship with key stakeholders, approach to transparency, and the nature of the deliberative decision-making process.

Process for the Conduct of HTA
Timelines. NICE typically publishes recommendations for STAs 12-14 months after the UK marketing authorization. 42 In contrast, ICER begins its review approximately 8 months before approval by the U.S. Food and Drug Administration (FDA) is expected, so the final report and public hearing can coincide with the period of initial insurance coverage and pricing negotiations.

Process for Conducting HTAs: Overview of the Comparison Between ICER and NICE
a health economic model. This submission is reviewed by the ERG to produce a report that includes a critique of the clinical and cost-effectiveness evidence, exploratory analyses with alternative assumptions or data, and the "ERG most plausible" incremental cost per QALY. Although ICER requests clinical, economic, and other evidence from the manufacturer, ICER either commissions an independent modeling group or performs an evaluation internally to generate cost-effectiveness evidence. The bulk of an ICER evidence report presents the clinical and costeffectiveness evidence on technology developed by ICER and the academic collaborators, respectively.
Approach to Transparency. In the NICE submission, the manufacturer includes an executable electronic copy of an economic model, with full access to the programming code, which is reviewed by the ERG. The ERG amended model is shared with the manufacturer to aid model transparency. The NICE website also includes full details of all relevant appraisal documents (with the exception of commercial or academic in confidence data, which is redacted in the documents so that the data cannot be back calculated).
As part of ICER's program on economic model transparency, modeling groups provide a preliminary methods presentation to manufacturers and to patient stakeholder groups and have developed a process for sharing models with interested manufacturers involved in the review during the draft report public comment period. The ICER website includes all relevant documents with confidential or academic in confidence data redacted so that the data cannot be back calculated. 43 ICER is also developing an interactive model program where end users will be able to change key inputs and estimate results. 44 Committees. The NICE Technology Appraisal Committee members include representatives from the NHS, patient and carer organizations, academia, and pharmaceutical and medical devices industries but are independent of any vested interests. The committee meetings are open to the public during the initial evidence presentation, followed by a closed session for deliberation.
ICER also facilitates a public discourse for its reports, and the regional appraisal committees include clinicians, health services researchers, health economists, patient and public representatives, and clinical experts and patient representatives specific to the topic under review. The meetings are conducted in public; they are also live streamed and archived on the ICER website. Table 4 compares the use of HTA in decision making, which relates to the format and type of evidence generated, how the evidence is considered during decision making, and the format of the recommendations.

Use of HTAs in Decision Making
Format and Type of Evidence. For NICE, the ERG report highlights critical issues related to clinical and cost-effectiveness evidence submitted by the manufacturer and "the ERG most plausible" incremental cost per QALY. The ERG report also includes a number of scenarios, one-way sensitivity analyses (OWSA), and probabilistic sensitivity analyses (PSA).
ICER assesses the quality of the clinical evidence using its Evidence Rating Matrix, which assesses the level of certainty and the magnitude of difference in the net health benefit. The price benchmarks are reported using the standard range of $100,000-$150,000 per QALY. Extensive scenario analyses,

Use of HTA in Decision Making: Overview of the Comparison Between ICER and NICE
Following the ICER public meeting, a final report is generated along with a "meeting at a glance" report (a condensed version of the report) that includes the evidence presented in the final draft evidence report, updated based on any new evidence, or relevant discussions during the public meeting. The price benchmarks for the technology; the discounts in the price of the technology needed; the results of the voting, along with the additional information on the deliberation surrounding the votes; and the policy roundtable discussions are published in the full report, along with recommendations for future research.
■■ Discussion ICER and NICE are similar in their use of cost per QALY and methodologies used; however, NICE is a governmental organization making recommendations (which have to be implemented within 3 months), whereas ICER does not have mandatory power but produces evidence and guidance on pricing for use by payers. This difference results in differences in the type of recommendations, timeliness of decisions, thresholds, transparency, and the format of deliberative discussions.

Type of Recommendations
NICE is in a single payer health care system, so it can make recommendations for the whole NHS in England and Wales. In contrast, the U.S health care system is decentralized, with multiple public and private payers and different budget levels and decision-making processes. ICER acts as an information provider, producing price benchmarks in the evidence reports, which can be used by individual payers as they see fit. Recent survey data suggest that U.S. health care payers, while highly variable, are using ICER reports as part of their appraisal and negotiating processes, 46 which may take the form of discounts needed to achieve the price benchmarks or outcomes-based contracts.

Thresholds
It should be noted that thresholds used by NICE and ICER are not empirical estimates of opportunity costs. The opportunity cost of health in England has been estimated to be around £13,000 or 0.43 gross domestic product (GDP) per capita, while the range used by NICE varies from approximately 0.66 GDP per capita (£20,000) to 1.64 GDP per capita (£50,000) using an exchange rate of $1.3 to £1. The range used by ICER is from 1.84 GDP per capita ($50,000) to 2.76 GDP per capita ($150,000), 47 which are reflective of a body of literature used to estimate willingness to pay in the United States. 48 This suggests more generous threshold values in the United States, which is in line with higher U.S. spending on medical care.
OWSA, and PSA are also presented; ICER reports also present results of incremental cost per life-year, incremental cost per evLYG, and incremental cost per other effectiveness measure (e.g., disease-specific measures such as exacerbations for chronic obstructive pulmonary disease).

Consideration of Evidence.
The NICE committee members discuss the key clinical and economic uncertainties and select the base case among the different ERG exploratory analyses. The incremental cost-effectiveness ratio from this base-case analysis, along with the consideration of other factors, including uncertainty, is used to determine if the technology should be reimbursed. These discussions typically happen over multiple meetings over a few months, 42 between which there may be new evidence submitted by the manufacturer.
In contrast, consideration of evidence at an ICER public meeting is at a more general level (i.e., less detailed discussion), with the members using the base-case analyses performed by the ICER modeling group as a basis to perform value for money judgements. Following the evidence presentation and public comments, the council votes on key questions concerning the comparative clinical effectiveness, potential other benefits and contextual considerations, and the overall long-term value for money of the treatments. All the discussion is completed within a single (albeit longer) 1-day meeting.
Deliberative Decision Making. After consideration of the manufacturer's submission, the ERG report, and testimony from experts and other stakeholders, the NICE Appraisal Committee formulates its preliminary guidance, on which stakeholders are invited to comment. These consultation comments are discussed at subsequent meetings where the committee may also consider additional evidence submitted by the manufacturer in response to consultation.
Following deliberation and voting on the evidence, the ICER public meeting concludes with a moderated policy roundtable, with members, including patient advocates, clinical experts, payers, and representatives from pharmaceutical manufacturers, to explore how to best apply the evidence on the new treatments to policy and practice.
Format of the Recommendations. NICE can make 5 types of recommendations: recommended, optimized, recommended for use in the Cancer Drugs Fund (cancer appraisals only), only in research, or not recommended. NICE cannot negotiate on the price of technologies; however, the manufacturer can decide to take part in pricing agreements through the Patient Access Scheme (PAS) to enable patients to gain access to highcost drugs, which would otherwise not be recommended by NICE. 45 The price implied by the PAS is kept confidential, and only the list price of the drug is publicly available. The final appraisal documentation details if and for what populations the technology is recommended for use. and cost-effectiveness evidence, provide responses to clarification questions, and even provide access to further data, where possible. However, the reliance of ICER on voluntary submissions means that there are implications if manufacturers do not cooperate. For example, the lack of access to individual patient data or unpublished data with longer follow-up, which only the manufacturer can provide, can hinder the ability of ICER to develop a robust economic model. On the other hand, manufacturers do have the incentive to participate in order to ensure that their data are accurately included and the most favorable data and assumptions are considered, given the highly competitive landscape.
Funding. Along with most northern European HTA bodies, NICE is publicly funded and is completely independent from the decision-making body/payer (e.g., it would require an act of Parliament to dissolve NICE). In contrast, ICER obtains most of its funding from private foundations, and grants are typically for the short term (around 3 years), which could potentially raise governance issues in the long term. Long-term sustainable funding outside of private foundations maybe possible from private health plans or other sources but are not yet in place.
Criticism of QALYs. The use of QALYs by NICE is fairly well accepted in England and Wales. In contrast, there has been significant opposition in the United States to ICER's use of QALYs (see Appendix, available in online article). ICER has defended the use of QALYs; however, it should be noted that ICER has now formally adopted evLYG as a complement to QALYs. 49

Limitations
There are some limitations to this review. Given the many different elements of comparison, it is not possible to go into the details of all the issues, and as such, some interesting aspects may have been missed. Readers are encouraged to review the references to get an in-depth understanding of the issues. Also, HTA is a rapidly growing field, and while caution has been taken to ensure that the most recent information is included in this review, it is possible that some may become out of date by publication. Again, the readers are encouraged to review the original source materials from ICER and NICE to ensure accuracy.

■■ Conclusions
This is the first published comprehensive comparison of ICER and NICE, focusing on the structure of HTA programs, methods of HTA, processes for conduct of HTA, and use of HTAs in decision making. ICER is similar to NICE in its use of the incremental cost per QALY metric and in its methods for undertaking clinical and economic reviews. However, key differences between NICE and ICER arise because of important differences between the United Kingdom's single payer health care system and the U.S. pluralistic system, and the need to Deliberative Decision Making NICE Appraisal Committee meetings are quite detailed; the members discuss in depth the issues associated with clinical and cost-effectiveness evidence to select the "base case" incremental cost-effectiveness ratio, which is used to determine if a technology is cost-effective. In contrast, discussion regarding cost-effectiveness evidence at an ICER public meeting is less detailed, with the members using base-case analyses performed by the ICER modeling group to perform value for money judgements. ICER models go through extensive phases of iterative feedback from manufacturers and other stakeholders before being finalized for the evidence report. As such, it could be argued that ICER does most of what NICE appraisal committees do in their meetings as preparatory work for the public meeting. The ICER committees can, however, deviate from the recommendations suggested by the base case in their voting, reflecting the influence of the uncertainty, scenario analyses, and other benefits and contextual considerations.

Transparency
Although NICE and ICER make all documents available to the public (redacting any confidential information), there is a difference in transparency around the committees' deliberative discussions to formulate recommendations. All ICER meetings are public and are live streamed and archived on the ICER website to ensure legitimacy and uptake of its recommendations. However, while part of the NICE committee meetings are open to the public (e.g., during evidence presentation), sessions on the deliberation of recommendations are closed.

Timeliness
Given its governmental role and mandatory power, NICE can schedule technology appraisals without fear of major resource allocation decisions in the NHS being made in advance of its recommendation. As such, NICE typically initiates drug appraisals after regulatory approval from the European Medicines Agency (EMA), although it often contacts manufacturers for information before EMA approval. ICER, on the other hand, needs to ensure that the timing of its reports aligns with upcoming FDA approvals, when U.S. payers establish their initial coverage policies and negotiate prices. This can create additional challenges for ICER reviews related to evidence availability, especially drug price information.

Challenges
Given its lack of mandatory power and government support, ICER faces a number of challenges, which include potential lack of engagement from the manufacturers, the uncertainty regarding long-term funding, and the criticism of QALYs.

Engagement with Manufacturers.
Manufacturers in the United Kingdom are engaged with NICE-they submit clinical tailor an HTA process to the U.S. context. NICE has developed a comprehensive and robust HTA process, which has evolved since its inception, in order to meet the needs of England and Wales. ICER has leveraged the work done at NICE and other HTA bodies to create a new process tailored to the United States. ICER's increasing relevance in U.S. health care decision making reflects the gap that needed to be filled, as well as the success of ICER in navigating the complex U.S. health care landscape.