Typically, there is no empirical study with sufficient long-term follow-up that compares all treatments for a particular disease regarding relevant clinical outcomes and costs. Thus, CEAs generally rely on mathematical models that integrate evidence on the course of disease, treatment effects, and the relationship between clinical outcomes and costs from a variety of studies. Users of published CEAs typically do not have access to the actual models used. This lack of transparency poses problems for decision makers who might question the scientific rigor of the analysis, or whose perspective, local context, or patient population varies from that of the published analysis. In the absence of public access to the actual models, updating a CEA is cumbersome, if not impossible, for someone other than the original model developer. As a result, published cost-effectiveness findings risk immediate irrelevance to some stakeholders and growing irrelevance to all stakeholders as new clinical evidence emerges.

Against this backdrop, several authors have argued for open-source cost-effectiveness models.^4-6 The typical cost-effectiveness model, however, can be complex. Providing public access to the software code to implement the open-source model may not be sufficient for a local decision maker to perform a CEA reflecting the local setting, for example, by using specific information regarding the patient population and unit cost estimates, when there are limited resources or limited expertise with the technicalities of the model or software used. In order to promote a relevant CEA, it is important to make the models easily accessible for local decision makers. A web-based, user-friendly interface to interact with an open-source model allows a decision maker to tailor the evaluation to the local setting in an efficient manner.

Physicians are a sometimes-neglected stakeholder group with regard to cost-effectiveness modeling. However, clinicians have growing interest in having access to accessible and transparent cost-effectiveness models. Beyond modeling the value of new medical technologies or interventions, interest is increasing in the ability to demonstrate value-based care. For example, clinicians are increasingly focused on the value of services provided by their specialty, care provided through integrated delivery networks that manage the costs and quality of the care they provide, and care pathways that are designed to reduce unwarranted variability in patient management.

Unfortunately, there are few real-world examples of open-source models and significant institutional barriers to create these. Open-source models are administratively complex to manage and update, and many key players do not have an incentive to take on this task. Despite private disincentives, collectively different stakeholders would benefit from access to high-quality, transparent, and publicly available cost-effectiveness models.

The nature of model-based CEA can lead to disputes about its credibility in the scientific literature and health care community. Model inputs are typically informed by systematic literature reviews and meta-analyses to ensure that all relevant evidence is considered. However, decisions regarding the model structure relating model inputs to outputs are frequently made arbitrarily based on the idiosyncratic expertise of the model developer and can have large impacts on findings.⁷ While robustness can be assessed by means of sensitivity analyses, these are typically limited to studying the impacts of varying model inputs. For any given disease, a variety of modeling approaches have typically been proposed in the literature. In order to evaluate the effect of these different approaches on estimates of cost-effectiveness in a systematic way, flexible open-source cost-effectiveness models that not only capture the uncertainty in model input parameters (i.e., parameter uncertainty), but also capture the range of proposed alternative model structures (i.e., structural uncertainty) are needed. This facilitates demonstrating the implications of different areas of uncertainty and leads to a better understanding of the reasons why value estimates vary. A nontechnical and publicly available user interface to interact with this flexible open-source model enables a more constructive dialogue between stakeholders (e.g., patients, payers, providers, and manufacturers) with different beliefs about relevant clinical data, modeling approaches, and value perspectives.

No model for CEA is ever truly “right” or “complete,” and it must evolve along with clinical evidence and improvements in scientific methodology to ensure that its credibility remains over time. Scientific evidence grows and improves through the efforts of many individual researchers. The open-source flexible model that captures the range of modeling approaches can be a platform to crowdsource expertise. Based on input from clinical and methodological experts and formal debate, the model can be updated and revised over time in an iterative manner to ensure that it remains sound and up to date. In Table 1, we outline an example process. Over time, this collaborative approach will hopefully lead to a flexible model that captures a set of scientific defensible approaches based on the latest evidence and state of the science, which we like to call the open-source, consensus-based model.

The primary aim is to build credible interactive models for local decision makers that account for all scientific uncertainty (due to gaps in evidence and different modeling beliefs) to help them understand the confidence with which they might make a decision. The predicted outcomes and cost-effectiveness estimates for the different scientifically defensible approaches included in the this flexible open-source model can be “averaged.”⁸ With a model-averaging approach, we can overcome the confusion among decision makers about what kind of model structure to use for cost-effectiveness evaluations and properly capture scientific uncertainty. To be sure, using this open-source, consensus-based model as the basis for relevant local cost-effectiveness evaluations does not eliminate all the variation in cost-effectiveness estimates, since perspectives on value will differ,⁹ and arguments about relevant model input, given the local context, will persist.

The development of open-source models in an iterative and collaborative fashion has multiple logistical and practical challenges. First, there needs to be a platform where the latest version of the model is available, and where feedback and proposed modifications can be shared and tracked among different researchers interested in contributing. Solutions used in software development come to mind. For example, GitHub, a repository hosting service designed around the version control system Git, can be used to manage and store revisions of the model and facilitate contributions from multiple developers. Second, simulation models can be complex, which makes providing feedback a labor-intensive process. Representative stakeholders need to be motivated to contribute, which cannot be achieved without active and ongoing communication and engagement efforts, as well as meaningful recognition of individual contributions. Third, collaboration requires communication among experts from diverse backgrounds who may not necessarily understand each other’s jargon. This is a challenge that should not be underestimated. Finally, intellectual property has been raised as a potential concern in the context of developing open-source models and will require a serious discussion to find acceptable solutions among producers and consumers of the models. That being said, if we are serious about improving the credibility and acceptance of model-based value assessment, limiting access to models in the name of intellectual property seems to go directly against this goal.

Despite its challenges, the benefits of a collaborative and transparent approach to model development are worth the effort. Initial progress is being made through case studies in rheumatoid arthritis and non-small-cell lung cancer, which will provide information on the feasibility of this approach.^10,11

CEA is a well-established and arguably the preferred approach to inform efficient use of health care resources. However, a thorough analysis is resource intensive and complex. The question that arises is how we can maximize the relevance and credibility of cost-effectiveness evaluations in the context of a decentralized health care environment with many local decision makers. Access to credible evidence-based, open-source models that can be adapted to reflect the local setting in a relatively straightforward manner is essential. No model for CEA is ever truly “right” or “complete,” so it must evolve along with clinical evidence and improvements in scientific methodology to ensure that its credibility remains.

We propose a transparent approach of iterative development and collaboration between different content and methodology experts to produce up-to-date, open-source, consensus-based cost-effectiveness models that account for parameter and structural uncertainty to help local decision makers understand the confidence with which they might make a decision. This proposed approach provides a way to adapt formal assessments of value—long the province of centralized health care systems—into the decentralized U.S. health care landscape.