Payer perceptions on the use of economic models in oncology decision making

BACKGROUND: To support oncology formulary decisions, especially with accelerated regulatory approvals and niche populations, payers desire data beyond what regulators review. Economic models showing financial impact of treatments may help, but data on payers’ use of economic models in oncology are limited. OBJECTIVE: To assess payer perceptions regarding use of economic models in informing oncology formulary decisions. METHODS: A multidisciplinary steering committee involving health economists and payers developed a survey containing singleanswer, multiple-answer, and free-response questions. The pilot survey was tested at a mini-advisory board with 5 US payers and revised based on feedback. In February 2020, the survey was distributed to 221 US payers through the AMCP Market Insights program and 10 additional payer panelists, who were invited to discuss survey results. Results were presented primarily as frequencies of responses and evaluated by plan size, type of health plan, and geography (regional vs national). Differences in categorical data responses were compared using Pearson chi-square or Fisher’s exact tests. Two-tailed values were reported and an alpha level of 0.05 or less was used to indicate statistical significance. RESULTS: Overall, 106 of 231 payers completed the survey (45.9%); 45.5% represented small plans (< 1 million lives), and 54.5% represented large plans (≥ 1 million lives). Respondents were largely pharmacists (89.9%), and 55.6% indicated that their job was pharmacy administrator. Payers indicated moderate/most interest in cost-effectiveness models (CEMs; 85.3%) and budget impact models (BIMs; 80.4%). Overall, 51.6% of respondents claimed oncology expertise on their pharmacy and therapeutics committees. Large plans were more likely to have expertise in reviewing oncology economic models than small plans (55.6% vs 31.1%, P = 0.015). The most common reasons for not reviewing economic models included “not available at time of review” (44.1%) and “potential bias” (38.2%). Overall, 43.1% of payers conduct analyses using their own data after reviewing a manufacturer-sponsored economic model. To inform formulary decisions, 62.7% of payers use BIMs and 66.7% use CEMs sometimes, often, or always. When comparing therapies with similar safety/efficacy profiles, 68.6% of payers reported economic models as helpful a moderate amount, a lot, or a great deal. Over one-third of payers (37.3%) were willing to partner with manufacturers on economic models using their plans’ data. Payers valued preapproval information, data on total cost of care, and early access to models. Concerns remained regarding model transparency and assumptions. CONCLUSIONS: Most US payers reported interest in using economic models to inform oncology formulary decision making. Opportunities exist to educate payers in assessing economic models, especially among small health plans. Ensuring model availability at launch, transparency in model assumptions, and payer-manufacturer partnership in model development may increase the utility of oncology economic models among US payers.

the survey was distributed to 221 US payers through the AMCP Market Insights program and 10 additional payer panelists, who were invited to discuss survey results. Results were presented primarily as frequencies of responses and evaluated by plan size, type of health plan, and geography (regional vs national). Differences in categorical data responses were compared using Pearson chi-square or Fisher's exact tests. Two-tailed values were reported and an alpha level of 0.05 or less was used to indicate statistical significance.

RESULTS:
Overall, 106 of 231 payers completed the survey (45.9%); 45.5% represented small plans (< 1 million lives), and 54.5% represented large plans (≥ 1 million lives). Respondents were largely pharmacists (89.9%), and 55.6% indicated that their job was pharmacy administrator. Payers indicated moderate/most interest in costeffectiveness models (CEMs; 85.3%) and budget impact models (BIMs; 80.4%). Overall, 51.6% of respondents claimed oncology expertise on their pharmacy and therapeutics committees. Large plans were more likely to have expertise in reviewing oncology economic models than small plans (55.6% vs 31.1%, P = 0.015). The most common reasons for not reviewing economic models included "not available at time of review" (44.1%) and "potential bias" (38.2%). Overall, 43.1% of payers conduct analyses using their own data after reviewing a manufacturer-sponsored economic model. To inform formulary decisions, 62.7% of payers use BIMs and 66.7% use CEMs sometimes, often, or always. When comparing therapies with similar safety/efficacy profiles, 68.6% of payers reported economic models as helpful a moderate amount, a lot, or a great deal.
What is already known about this subject • Payers may use economic models to help inform coverage decision making.
• In oncology, accelerated approvals and small patient populations limit the amount of data available for new products, making economic models more helpful to payers than for nononcology drugs.
What this study adds • More than half of payers have expertise in evaluating economic models in oncology and find value in using budget impact models and cost-effectiveness models in oncology formulary decision making.
• Factors that may increase the utility of oncology economic models among US payers include ensuring model availability at launch, transparency in model assumptions, and payermanufacturer partnership in model development.
When making formulary and coverage decisions about new therapies, payers may benefit from economic information in addition to traditional safety and efficacy data. As stated by the US Food and Drug Administration (FDA) guidance on payer communciations, payers commonly request additional information regarding effectiveness and cost-effectiveness to support formulary management and coverage and reimbursement decisions across their plan membership. 1 This may be especially true in oncology where innovative new therapies address unmet needs for small niche populations, some of which may have received accelerated regulatory approvals.
The "AMCP Format for Formulary Submissions" has acknowledged that cost-effectiveness models are an effective means to assess the overall potential value of health care technologies. As a result, the AMCP Format includes a section on economic information that describes economic models, such as simple financial models, cost-effectiveness models (CEMs), and budget impact models (BIMs). 2 Simple financial models enable payers to estimate the cost of a drug based on price (eg, average wholesale price, wholesale acquisition cost, and average selling price) and possible discounts but do not address clinical outcomes or other economic impacts. 2 A CEM compares the estimated costs and benefits of a new intervention with the current standard of care, watchful waiting, or other available drugs/ therapies over a specific time frame, which helps the payer estimate the incremental changes in cost and the relative value of a new drug compared with existing therapy. 3 BIMs help payers estimate the financial impact on their health plan's overall budget by calculating how use of a new drug may change the usage rate for existing standard of care therapies, and how those changes will impact the plan's overall disease outcomes, resource usage, and costs. 3 A 2016 study by Fazio et al showed that payers use manufacturer-sponsored economic models for a variety of reasons (eg, to learn about manufacturers' clinical data, to scrutinize cost-effectiveness analysis, to help conduct their own analyses, to learn background information on the disease, or to repopulate the model with their own local data). 4 However, that study was not specific to oncology, is somewhat dated, and was conducted before the recently updated guidelines on preapproval information exchange (PIE). 5 Decision making in oncology is somewhat unique, with multiple considerations involved, such as overall survival, cost, and cultural and legal considerations. Therefore, we conducted our own survey to understand current US payers' perception and use of economic models in oncology formulary decision making.

Methods
An interdisciplinary steering committee of payers, academics, and industry health economics and outcomes research experts was formed to develop a survey to gauge payer understanding of the value and use of economic models in oncology decision making.
The steering committee created a pilot survey based on literature reviews and current understanding of the payer landscape regarding the use of economic models in oncology decision making. The pilot survey was distributed via email to 5 US payers in October 2019. The results were tallied, then a follow-up in-person discussion was held a week later with the steering committee and the 5 payers. Discussions focused on questions where the answers showed disagreement or confusion. Survey questions were revised based on feedback from the payers and then reviewed by the authors.
The revised survey questions included 13 questions about payer perceptions and use of economic models. The survey included a variety of question formats that included questions with single choice and/or multiple responses (select all that apply), as well as ranking responses on a scale of 1 = least interest to 5 = most interest. Of the 13 questions, 1 question asked about payers' interest in using different types of economic models, and 8 questions addressed respondent organizations' experiences with using economic models in coverage decision making. Two questions asked respondents to identify barriers to using economic models. Two additional questions explored respondents' interest in partnering with manufacturers to develop or validate an economic model in oncology using the organization's plan data.
Over one-third of payers (37.3%) were willing to partner with manufacturers on economic models using their plans' data. Payers valued preapproval information, data on total cost of care, and early access to models. Concerns remained regarding model transparency and assumptions.

CONCLUSIONS:
Most US payers reported interest in using economic models to inform oncology formulary decision making. Opportunities exist to educate payers in assessing economic models, especially among small health plans. Ensuring model availability at launch, transparency in model assumptions, and payer-manufacturer partnership in model development may increase the utility of oncology economic models among US payers.

CURRENT PAYER PERCEPTION AND USE OF ECONOMIC MODELS
Compared with small plans (< 1 million covered lives), a higher proportion of large plans (≥ 1 million lives covered) reviewed economic models in relation to oncology products within the last 2 years (57.4% vs 35.6%, P = 0.043).
Large health plans were more likely than small plans to have individuals with expertise in evaluating economic models in oncology (55.6% vs 31.1%, P = 0.016; Figure 1). Payers' reasons for not reviewing economic models included lack of available models at the time of product review (44.1%), potential bias (38.2%), lack of necessary personnel/ training to review the model (33.3%), lack of transparency (31.4%), and models being too complicated (28.4%; Figure 2).
Overall, after reviewing a manufacturer-sponsored economic model, 43.1% of payers said they used their own internal data to conduct models. 4 The panel of 10 payers and the steering committee members convened virtually in March 2020 to discuss survey findings.

SURVEY ANALYSIS AND STATISTICS
The survey results were presented primarily as frequencies of responses. For multiple-response questions, each response was coded as "yes" or "no." A ranking question asked respondents to rank different models as of most interest, moderate interest, and least interest. Categorical responses were evaluated using Pearson chi-square or Fisher's exact test by plan size, plan type, and scope of service (regional vs national). Statistical significance of responses based on plan size, plan type, and scope of service were based on 2-tailed values with an alpha level of 0.05 or less.

DEMOGRAPHICS OF RESPONDENTS
From the national sample of 221 AMCP survey recipients plus the Subsequently, in February 2020, the national online survey was distributed through the AMCP Market Insights program to a panel of 221 payers. This managed care panel consisted of AMCP members who were pharmacy leaders involved in medication product evaluation and/or utilization management who had agreed to participate in payer-based market research. Results of the online survey were collected using Survey Monkey.
The same survey was also distributed to 10 additional US payers who were not involved with the AMCP Market Insights panel and who agreed to participate in a live panel discussion about the aggregate survey results. The survey results from these 10 panelists were combined with the results from the broader payer panel previously mentioned. After the panelists completed the survey, but before the panel discussion, each panelist was sent the Fazio et al article on the use of economic models in US payer decision making (nononcology-specific) to provide a common basic level of understanding around economic the time of initial formulary decision making, especially for oncology drugs with accelerated approvals. They also understood that economic data often vary over time as a product enters the market. Despite these limitations, payers still welcomed any early economic data made available to them from clinical trials, especially data that factor into total cost of care, such as adverse events and hospitalizations. Payers wanted economic models that incorporate overall survival and progression-free survival, since those rates are highly relevant and may help in outcomes-based contracting. There was an appreciation that this evidence is often not available at launch and should be provided through follow-up real-world evidence studies.
Panelists indicated that they rely on manufacturer liaisons for much of this data, although they also consider AMCP dossiers and literature reviews. When possible, payers often validate the pharmacoeconomic data they receive from manufacturers using data from their own populations, but with small oncology populations and a lack of in-plan data at new product launch, such validation is not always feasible.
During the post-launch stage, payers wanted to continue to see real-world economic data, which they can use to verify initial decisions and evaluate potential changes in coverage decisions or treatment pathways.
Panelists reported concerns about economic models, specifically regarding the transparency and assumptions used in the models. Payers were especially concerned about the following assumptions: (1) prevalence in select trial populations rather than the plan's wider population; (2) lack of accurate and inclusive estimates for all components of total cost of care; and (3) treatment patterns in the trial that may not reflect actual treatment patterns practiced by physicians in a moderate amount or more when considering therapies with a similar efficacy/safety profile.
After the national survey was completed, a panel of 10 payers discussed the results with the steering committee, adding additional insights to the survey findings. Panelists confirmed their interest in economic models and remarked that, in general, MCOs had more access to internal economic data, while PBMs were often able to access only pharmacy claims data.
Panelists indicated interest in planning for the effect of a new drug on their health plans. They valued data on standard of care, the drug's anticipated place in therapy, estimates of target populations, adverse event costs during clinical trials, hospitalizations during clinical trials, and total cost of care. Panelists acknowledged that such data are often limited at additional analyses to inform their oncology decision making (Figure 3). Only 9.8% said they made decisions based on the manufacturer-sponsored model without using their own data to validate the model. Nearly a third (31.4%) of payers did not make coverage decisions at all based on the manufacturer-sponsored model.
When asked to rank their interest in the 3 types of economic models, payers ranked CEMs (85.3% moderate/most interest) and BIMs (80.4% moderate/most interest) much higher than cost calculators (34.3% moderate/most interest; Figure 4). Approximately two-thirds of respondents reported that they use CEMs (66.7%) and BIMs (62.7%) sometimes, often, or always to inform oncology formulary decisions. Over two-thirds of payers (68.6%) believed economic models can help inform decisions

FIGURE 2
Payers information, they are made available near a product's approval date, which may be too late for consideration by payers. 6 Therefore, payers would likely welcome any economic data that contributes to total cost of care forecasts as early as possible (preferably before FDA approval, under PIE guidelines). 5 As a result of these early conversations with the manufacturers, payers could then look at the predicted cost for a drug with certain assumptions regarding the patient population, adverse events rates, hospitalizations, and medical costs. From the survey results, large plans (≥ 1 million covered lives) were more likely than smaller plans (< 1 million covered lives) to have in-house expertise available to evaluate and validate economic models. This is likely because of limited resources, such as analytical personnel and access to data, that is not always available to smaller plans. Given these findings, opportunities exist for manufacturers to work with health plans, especially small plans, to bring economic models to their attention.
Leading up to formulary review, payers need as much information as possible for preparatory work with finances, pricing, and overall cost of care. However, at the time of initial formulary review, such data are not always available in the public domain or payers may need to spend considerable time searching for this data. Manufacturer liaisons may play an important role in helping payers understand early product information, including preliminary economic data, through preapproval information exchange guidance. 7 Additionally, payers may also request AMCP dossiers before and after approval.
While payers desire economic models, they have expressed concerns about quality of the data, as well as the appropriateness of assumptions made in the economic models. Payers through FDA-accelerated approval. Additionally, patient populations may be small, and often individual plans do not have enough data to inform their pharmacy and therapeutic committee decisions.
Despite difficulties in gathering required inputs, economic models serve as an important tool to narrow uncertainty around the financial impact. Therefore, from the survey and the panel discussion, it is apparent that economic models are considered valuable additions to the traditional trial data that payers review when making coverage decisions for oncology products.
While clinical trial data remains the primary driver for formulary decisions, payers are often forced to make formulary decisions with minimal information regarding the economic impact of their decisions. Although final evidence reports from the Institute for Clinical and Economic Review provide such the real world. Panelists said they welcomed models where the assumptions were referenced and they would like to see more sensitivity data around the assumptions being presented. Some panelists suggested that enlisting the help of health plans in the formation of models may address some of those concerns.

Discussion
In general, payers have a strong desire to understand a new drug's effect on the disease's standard of care and its change to the total cost of care for those patients. 6 Although information on how economic models are used in health plans has been previously reported, 4 our study specifically explored this topic as it relates to decision making in oncology. There are unique limitations for payerbased analyses in oncology because some products enter the market

FIGURE 3
Payers' Actions After Reviewing a Manufacturer-Sponsored Economic Model an economic model in oncology using their plans' data. A second step in addressing concerns may be to increase transparency about the assumptions used in economic models. This can be achieved by providing clear references for the selected assumptions and sensitivity analyses. Another way to increase transparency and comfort with models is through live presentations. For these presentations to be successful, manufacturer outcomes liaisons should be trained to explain where the assumptions in the model came from, why they were chosen, and how the assumptions may change over time. Additionally, manufacturer outcomes liaisons should be knowledgeable about how the model was built, how the data were incorporated, how inputs can be changed, and where to find data to feed into the model. Although much of this is currently being done, it is important to make sure that it is done routinely. Using the AMCP dossier, which includes the model report, is also helpful in presenting the information.

LIMITATIONS
Although MCOs, PBMs, and large payers were well represented in the survey, the data presented may not be generalizable to all types of health plans. In particular, integrated delivery systems were not well represented in this survey. Most respondents were pharmacy directors of health plans; other decision makers, such as medical directors, may have a different perspective. Although the survey response was approximately 45%, the respondents may not be representative of the entire panel.
As with all surveys, the length of the survey was limited and did not allow collection of additional information to explain answers provided. Not requiring that all respondents answer all questions resulted in a different number of new product approvals in oncology increases; payers will continue to look for additional information to help differentiate between products or to help inform outcomes-based contracts. Established guidelines for economic models should be communicated as part of such education to support health plans of all sizes in understanding how to use economic models. 8 One step towards addressing payer concerns about models, assumptions, and transparency would be for manufacturers to involve payers in the development of the economic model for oncology and other drugs. In doing so, it is more likely that the parameters chosen by manufacturers are relevant to the payers and their decision makers. In fact, over a third of the payers (37.3%) reported in the survey that they would be willing to partner with a manufacturer to develop/validate have perceived that manufacturersponsored models may be biased in favor of a manufacturer's product. As a result, payers may want to validate the model by using their own data. Options to change the model inputs and sensitivity analyses may help increase validity of the model.
Since only half of payers reported having experience reviewing economic models or conducting their own economic studies in oncology, there may be an opportunity for better education regarding the value of economic models and how to create, validate, and use them effectively for decision making. This is especially true for smaller health plans, of which only 35.6% reported having reviewed an economic model in the last 2 years compared with large health plans (57.4%). The utility of economic models in formulary decision making is likely to grow as the

FIGURE 4
Payers' Level of Interest in 3 Types of Economic Models making. Economic data may be useful at initial coverage reviews, as well as for post-launch reviews.

DISCLOSURES
Pfizer provided funding for this research, and Pfizer employees led the development of the survey instrument, were involved in the analysis and interpretation of the data, and contributed to the manuscript as authors. Arondekar and Niyazov are employed by Pfizer. Biskupiak, Oderda, and Brixner are managers of Millcreek Outcomes Group and were paid as consultants on this project. Burgoyne was a consultant for Pfizer on this project.
number of responses for many of the questions. Although this likely increased participation, it may have introduced a bias related to which questions were not answered. It is likely that multiple respondents answered from the same organization. Since the survey results were deidentified by AMCP, we do not have specific information on the respondents. Additionally, only a small portion (< 10%) of the respondents participated in the virtual panel discussions; therefore, their opinions may not represent all respondents.

Conclusions
Economic models can be a valuable supplemental tool that payers can use when making oncology formulary decisions. However, many payers reported a lack of available models at the time of product review. This could be improved by requests for preapproval information under PIE guidelines. While payers are especially interested in total cost of care data, such data are largely unavailable at the time of initial decision making. Therefore, any data that are available from clinical trials may be useful in informing formulary decision making. Education for payers in the application and limitations around the use of economic models in oncology decision making would be useful.
As payers evaluate innovative oncology drugs coming to the market, they appreciate well-designed and highly transparent economic models that can help inform their decision