Skip to main content
Free access
Research Article
30 April 2019

Toward Relevant and Credible Cost-Effectiveness Analyses for Value Assessment in the Decentralized U.S. Health Care System

Publication: Journal of Managed Care & Specialty Pharmacy
Volume 25, Number 5

Abstract

In the United States, there is an increased interest to understand the value of health technologies. Cost-effectiveness analysis is arguably the most appropriate framework to quantify value and to inform reimbursement decision making regarding medical interventions; however, a thorough analysis is resource intensive and complex. In many countries, the cost-effectiveness of medical interventions is evaluated by expert agencies at the national level, but in the United States, reimbursement decision making occurs at the local level. This raises the question of how we can provide a means to transparent cost-effectiveness analysis that reflects the local context and patient population and is based on the latest evidence and scientific insights. In other words, how can we maximize the relevance and credibility of cost-effectiveness evaluations in the context of a decentralized decision-making environment? Published cost-effectiveness analyses typically fail on these dimensions. Access to transparent open-source models that can be adapted to reflect the local setting in a relatively straightforward manner is an essential step toward such a goal. However, no model for cost-effectiveness analysis is ever truly “right” or “complete,” and it must evolve along with clinical evidence and improvements in scientific methodology to ensure that its credibility remains.
We propose a transparent approach of iterative development and collaboration between content and methodology experts to produce up-to-date, open-source consensus-based cost-effectiveness models that account for parameter and structural uncertainty to help local decision makers understand the confidence with which they might make a decision. Our proposed approach provides a way to adapt formal assessments of value—long the province of centralized health care systems—into the decentralized U.S. health care landscape.
DISCLOSURES: This research was funded through the Innovation and Value Initiative, a nonprofit multistakeholder research organization. The Innovation and Value Initiative contracted with Precision Medicine Group for research activities related to this article. Jansen and Incerti are salaried employees and shareholders of Precision Medicine Group. Curtis is a paid consultant for the Innovation and Value Initiative. Curtis also reports consulting fees and grants from Amgen, AbbVie, BMS, Corrona, Janssen, Lilly, Myriad, Pfizer, Roche/Genentech, Radius, and UCB, unrelated to this article.
The continuing increase in U.S. health care costs has stimulated the introduction of novel payment arrangements and other initiatives to promote the use of high-value care.1 Cost-effectiveness analysis (CEA) can inform efficient use of health care resources by formally computing costs and benefits to identify the most valuable treatment options for a given disease. In many countries, a single health technology assessment (HTA) agency assesses the value of health care technology by means of CEA and recommends a utilization strategy. In the United States, however, utilization decisions are decentralized and made by a variety of payers and provider organizations. Value frameworks are gaining prominence to guide utilization of therapies, but they vary in perspective, the evidence considered, and approaches.2 Formal CEA is a well-established approach for value assessment and has been recommended by the International Society for Health Economics and Outcomes Research as the most appropriate framework for public and private reimbursement decision making.3 However, CEA only has relevance for decision making when it reflects the totality of the latest evidence, is transparent, and is representative of the local context and patient population. This raises the question of how we can provide a means to relevant and credible CEA in the context of a decentralized U.S. health care environment.

Relevant Cost-Effectiveness Evaluations Require Access to Transparent Adaptable Models

Typically, there is no empirical study with sufficient long-term follow-up that compares all treatments for a particular disease regarding relevant clinical outcomes and costs. Thus, CEAs generally rely on mathematical models that integrate evidence on the course of disease, treatment effects, and the relationship between clinical outcomes and costs from a variety of studies. Users of published CEAs typically do not have access to the actual models used. This lack of transparency poses problems for decision makers who might question the scientific rigor of the analysis, or whose perspective, local context, or patient population varies from that of the published analysis. In the absence of public access to the actual models, updating a CEA is cumbersome, if not impossible, for someone other than the original model developer. As a result, published cost-effectiveness findings risk immediate irrelevance to some stakeholders and growing irrelevance to all stakeholders as new clinical evidence emerges.
Against this backdrop, several authors have argued for open-source cost-effectiveness models.4-6 The typical cost-effectiveness model, however, can be complex. Providing public access to the software code to implement the open-source model may not be sufficient for a local decision maker to perform a CEA reflecting the local setting, for example, by using specific information regarding the patient population and unit cost estimates, when there are limited resources or limited expertise with the technicalities of the model or software used. In order to promote a relevant CEA, it is important to make the models easily accessible for local decision makers. A web-based, user-friendly interface to interact with an open-source model allows a decision maker to tailor the evaluation to the local setting in an efficient manner.
Physicians are a sometimes-neglected stakeholder group with regard to cost-effectiveness modeling. However, clinicians have growing interest in having access to accessible and transparent cost-effectiveness models. Beyond modeling the value of new medical technologies or interventions, interest is increasing in the ability to demonstrate value-based care. For example, clinicians are increasingly focused on the value of services provided by their specialty, care provided through integrated delivery networks that manage the costs and quality of the care they provide, and care pathways that are designed to reduce unwarranted variability in patient management.
Unfortunately, there are few real-world examples of open-source models and significant institutional barriers to create these. Open-source models are administratively complex to manage and update, and many key players do not have an incentive to take on this task. Despite private disincentives, collectively different stakeholders would benefit from access to high-quality, transparent, and publicly available cost-effectiveness models.

Credible Cost-Effectiveness Evaluations Need to Reflect the Current State of the Science

The nature of model-based CEA can lead to disputes about its credibility in the scientific literature and health care community. Model inputs are typically informed by systematic literature reviews and meta-analyses to ensure that all relevant evidence is considered. However, decisions regarding the model structure relating model inputs to outputs are frequently made arbitrarily based on the idiosyncratic expertise of the model developer and can have large impacts on findings.7 While robustness can be assessed by means of sensitivity analyses, these are typically limited to studying the impacts of varying model inputs. For any given disease, a variety of modeling approaches have typically been proposed in the literature. In order to evaluate the effect of these different approaches on estimates of cost-effectiveness in a systematic way, flexible open-source cost-effectiveness models that not only capture the uncertainty in model input parameters (i.e., parameter uncertainty), but also capture the range of proposed alternative model structures (i.e., structural uncertainty) are needed. This facilitates demonstrating the implications of different areas of uncertainty and leads to a better understanding of the reasons why value estimates vary. A nontechnical and publicly available user interface to interact with this flexible open-source model enables a more constructive dialogue between stakeholders (e.g., patients, payers, providers, and manufacturers) with different beliefs about relevant clinical data, modeling approaches, and value perspectives.
No model for CEA is ever truly “right” or “complete,” and it must evolve along with clinical evidence and improvements in scientific methodology to ensure that its credibility remains over time. Scientific evidence grows and improves through the efforts of many individual researchers. The open-source flexible model that captures the range of modeling approaches can be a platform to crowdsource expertise. Based on input from clinical and methodological experts and formal debate, the model can be updated and revised over time in an iterative manner to ensure that it remains sound and up to date. In Table 1, we outline an example process. Over time, this collaborative approach will hopefully lead to a flexible model that captures a set of scientific defensible approaches based on the latest evidence and state of the science, which we like to call the open-source, consensus-based model.
TABLE 1 Iterative and Collaborative Development of Credible Cost-Effectiveness Models that Reflect the State of the Science
Step 1:For a given disease of interest and associated treatment alternatives, create a transparent, open-source, flexible cost-effectiveness model that reflects the full range of scientifically defensible modeling approaches and variation in preferences and perspectives regarding value. Release model code, data used, and detailed documentation in the public domain (e.g., as an R package with source code on GitHub) and provide public access via a web-based user interface to allow nonexperts to evaluate the model.
Step 2:Invite feedback and changes to the model by other researchers and stakeholders (i.e., patients, payers, providers, and manufacturers) collected via a website in an open comment period. Changes to the model will be considered when supported by peer-reviewed evidence, that is, evidence-based suggestions for improvement.
Step 3:A panel of experts, representing different stakeholders, determines which of the evidence-based suggestions for improvement suggested in Step 2 should be implemented by means of a Delphi process.
Step 4:Revise the model according to the outcome of Step 3 and release a new model in the public domain. If someone’s contribution is incorporated, he/she will be either mentioned as a contributor or included as a co-author, depending on the extent of the contribution, for any published report.
Note: This process can be repeated over time (e.g., every 12 months) if sufficient new evidence-based suggestions for improvement have been collected or if the clinical literature has been updated in a major way.
The primary aim is to build credible interactive models for local decision makers that account for all scientific uncertainty (due to gaps in evidence and different modeling beliefs) to help them understand the confidence with which they might make a decision. The predicted outcomes and cost-effectiveness estimates for the different scientifically defensible approaches included in the this flexible open-source model can be “averaged.”8 With a model-averaging approach, we can overcome the confusion among decision makers about what kind of model structure to use for cost-effectiveness evaluations and properly capture scientific uncertainty. To be sure, using this open-source, consensus-based model as the basis for relevant local cost-effectiveness evaluations does not eliminate all the variation in cost-effectiveness estimates, since perspectives on value will differ,9 and arguments about relevant model input, given the local context, will persist.

Practical Challenges

The development of open-source models in an iterative and collaborative fashion has multiple logistical and practical challenges. First, there needs to be a platform where the latest version of the model is available, and where feedback and proposed modifications can be shared and tracked among different researchers interested in contributing. Solutions used in software development come to mind. For example, GitHub, a repository hosting service designed around the version control system Git, can be used to manage and store revisions of the model and facilitate contributions from multiple developers. Second, simulation models can be complex, which makes providing feedback a labor-intensive process. Representative stakeholders need to be motivated to contribute, which cannot be achieved without active and ongoing communication and engagement efforts, as well as meaningful recognition of individual contributions. Third, collaboration requires communication among experts from diverse backgrounds who may not necessarily understand each other’s jargon. This is a challenge that should not be underestimated. Finally, intellectual property has been raised as a potential concern in the context of developing open-source models and will require a serious discussion to find acceptable solutions among producers and consumers of the models. That being said, if we are serious about improving the credibility and acceptance of model-based value assessment, limiting access to models in the name of intellectual property seems to go directly against this goal.
Despite its challenges, the benefits of a collaborative and transparent approach to model development are worth the effort. Initial progress is being made through case studies in rheumatoid arthritis and non-small-cell lung cancer, which will provide information on the feasibility of this approach.10,11

Conclusions

CEA is a well-established and arguably the preferred approach to inform efficient use of health care resources. However, a thorough analysis is resource intensive and complex. The question that arises is how we can maximize the relevance and credibility of cost-effectiveness evaluations in the context of a decentralized health care environment with many local decision makers. Access to credible evidence-based, open-source models that can be adapted to reflect the local setting in a relatively straightforward manner is essential. No model for CEA is ever truly “right” or “complete,” so it must evolve along with clinical evidence and improvements in scientific methodology to ensure that its credibility remains.
We propose a transparent approach of iterative development and collaboration between different content and methodology experts to produce up-to-date, open-source, consensus-based cost-effectiveness models that account for parameter and structural uncertainty to help local decision makers understand the confidence with which they might make a decision. This proposed approach provides a way to adapt formal assessments of value—long the province of centralized health care systems—into the decentralized U.S. health care landscape.

REFERENCES

1.
Garrison LP, Carlson JJ, Bajaj PS, et al. Private sector risk-sharing agreements in the United States: trends, barriers, and prospects. Am J Manag Care. 2015;21(9):632-40.
2.
Willke RJ, Neumann PJ, Garrison LP Jr, Ramsey SD. Review of recent U.S. value frameworks—a health economics approach: an ISPOR Special Task Force report [6]. Value Health. 2018;21(2):155-60.
3.
Garrison LP Jr, Neumann PJ, Willke RJ, et al. A health economics approach to US value assessment frameworks—summary and recommendations of the ISPOR Special Task Force report [7]. Value Health. 2018;21(2):161-65.
4.
Dunlop WC, Mason N, Kenworthy J, Akehurst RL. Benefits, challenges and potential strategies of open source health economic models. Pharmacoeconomics. 2017;35(1):125-28.
5.
Schramm W, Sailer F, Pobiruchin M, Weiss C. PROSIT open source disease models for diabetes mellitus. Stud Health Technol Inform. 2016;226:115-18.
6.
Cohen JT, Neumann PJ, Wong JB. A call for open-source cost-effectiveness analysis. Ann Intern Med. 2017;167(6):432-33.
7.
Madan J, Ades AE, Welton NJ. An overview of models used in economic analyses of biologic therapies for arthritis—from current diversity to future consensus. Rheumatology. 2011;50(Suppl 4):iv10-iv8.
8.
Negrín MA, Vázquez-Polo FJ. Incorporating model uncertainty in cost-effectiveness analysis: a Bayesian model averaging approach. J Health Econ. 2008;27(5):1250-59.
9.
Garrison LP, Kamal-Bahl S, Towse A. Toward a broader concept of value: identifying and defining elements for an expanded cost-effectiveness analysis. Value Health. 2017;20(2):213-16.
10.
Incerti D, Curtis JR, Shafrin J, Lakdawalla DN, Jansen JP. A flexible open-source decision model for value assessment of biologic treatment for rheumatoid arthritis. Pharmacoeconomics. February 9, 2019 [Epub ahead of print].
11.
Incerti, D, Jansen, JP. A description of the IVI-NSCLC model. Updated January 2019. Available at: https://innovationvalueinitiative.github.io/IVI-NSCLC/index.html. Accessed March 19, 2019.

Information & Authors

Information

Published In

cover image Journal of Managed Care & Specialty Pharmacy
Journal of Managed Care & Specialty Pharmacy
Volume 25Number 5May 2019
Pages: 518 - 521
PubMed: 31039069

History

Published online: 30 April 2019
Published in print: May 2019

Authors

Affiliations

Jeroen P. Jansen, PhD* [email protected]
Innovation and Value Initiative and Precision Medicine Group, Los Angeles, California.
Devin Incerti, PhD
Innovation and Value Initiative and Precision Medicine Group, Los Angeles, California.
Jeffrey R. Curtis, MD, MS, MPH
University of Alabama at Birmingham.

Notes

*
AUTHOR CORRESPONDENCE: Jeroen P. Jansen, PhD,Innovation and Value Initiative, 11100 Santa Monica Blvd., Ste. 500, Los Angeles, CA 90025. E-mail: [email protected].

Metrics & Citations

Metrics

VIEW ALL METRICS

Citations

Export citation

Select the citation format you wish to export for this article or chapter.

Cited By

There are no citations for this item

View Options

View options

PDF

View PDF

Get Access

Restore your content access

Enter your email address to restore your content access:

Note: This functionality works only for purchases done as a guest. If you already have an account, log in to access the content to which you are entitled.

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media