Health care systems are being flooded with a slew of digital innovations, both internally and externally sourced. The promise of remote sensors, machine learning, artificial intelligence, and personal health records, together with investment from the National Institutes of Health and private investors, has resulted in a tsunami of interest in these solutions. Yet, health care systems across the country are struggling with their approach to innovation, which has resulted in a tremendous proliferation of pilot studies but little adoption of solutions at scale. The mismatch between the promise of the technology and current organizational strategy requires further examination.
Why Is This So Hard?
The first challenge is that executing on innovation that impacts value (cost and quality) requires both technology and business model transformation. While we have had significant focus on the technology aspects of the digital revolution, the business model transformation aspect has received much less attention. Business transformation requires significant investment of time by senior leadership, resources, the development of new business models, and the retirement of legacy business models. Most organizations will not undertake this level of investment in a novel concept without significant justification.
Another dilemma stems in part from leadership’s inability to evaluate novel technologies and pilot studies appropriately from an investment perspective. Rather than being seen as a point solution, digital innovations should be viewed as a starting point for investment in novel business processes, novel business designs, and potentially, novel organizational structures. Senior leadership, which must assess whether the digital innovation has reached a level of maturation appropriate to this level of investment, is often the bottleneck in these decision-making processes. Leaders at lower levels of decision making are often not empowered to make decisions regarding business transformation, and few health systems have an alternative “strategic” decision-making pathway that includes alternative perspectives, leaders, and metrics.
Leaders of organizations are rewarded for predictable performance, achieving revenue targets with little room for deviation. Innovation inherently involves uncertainty: time frames slip, enrollment lags projections, and unanticipated conflicts arise with other aspects of the organization. Given the clear prioritization of predictable performance, innovation stands in conflict with organizational objectives unless expectations are adjusted to balance predictable performance with a more strategic innovation agenda.
From the outside, the willingness of leadership to engage in efforts to transform the core business model can be described as the “activation energy” of the organization. From this perspective, the strategy, the quality of the evaluation, and the urgency of the need can all help to lower the activation energy required to pursue adoption of a digital innovation.
This innovation challenge is not unique to digital technology; it has been confronted in other non-biomedical fields, including surgery, where factors such as operator, team, and setting can play a huge role in assessment of quality. For example, in 2000, the UK Medical Research Council set out guidance for evaluating surgical innovation; these recommendations included development and evaluation through iterative phases, measurement of both outcome and process, and reporting detailed descriptions of interventions to improve reproducibility. Later, the IDEAL recommendations updated this framework by pairing the scope of analysis to the maturity of the innovation.
A Staged Approach Is Needed
We believe a similar staged approach could foster acceleration of digital innovation across health care systems. At each stage, we suggest that the evaluation should be appropriate to the state of technology implementation and should provide a framework for further investment decisions. We can think of this staged approach as encompassing discovery (our pilot testing stage), development (building a more robust organizational investment case), and dissemination (evaluating implementation at the enterprise level). By setting clear learning goals for each stage of innovation, senior executives can more clearly determine whether the promised innovation is achieving appropriate milestones. Innovations that achieve objectives can be considered for further investment, while innovations that are not achieving their objectives should lose their funding. (An important aspect of this evaluation function is to recoup the resources expended on extended pilots to channel those resources into other aspects of the innovation portfolio.)
The Discovery Stage
For example, in the discovery or pilot stage, in which the technology is being tested in one clinic or one service line, evaluation of clinical efficacy may be limited by the challenges of infrastructure, workflow, or culture change. At this stage, assessment of any meaningful impact may be very difficult. Many approaches to technology development are iterative, following the Lean Start-Up principles of minimum viable product testing. Others built on machine learning platforms are also iterative since they require use and feedback to improve model performance. As such, analysis should be limited to such elements as technical feasibility and user acceptance.
At Stanford University, computer scientists, in collaboration with physicians, created an algorithm using a deep neural network to evaluate electronic health record (EHR) data to predict all-cause 3- and 12-month mortality of admitted patients. In the pilot phase of the project, the EHR data of all admitted patients were evaluated every night by this algorithm, and the palliative care team was automatically notified of the list of patients with a positive mortality prediction. In the pilot test, which entailed a chart review of 235 patients flagged by the algorithm over a three-month period, the palliative care team found that most (about 85 percent) were appropriate for a referral.
Given this success, the researchers hope to extend their pilot into a prospective study that collects data on the change in rates of palliative care consults, and rates of goals-of-care documentation, resulting from the use of the model. Scientific publications from this study befit the proof-of-concept stage of evolution and included detailed case studies describing rationale and background for the technical solution.
The Development Stage
In the second stage, which we have called development, the technology has achieved key milestones from the pilot, and a more detailed evaluation can be proposed including integration with existing enterprise software and processes. At this stage, we can begin to evaluate the impact of the digital technology more broadly on the organization. Again, evaluation should focus on the intent of the innovation, key proof points (such as addressing access, quality, or cost-related measures such as workflow and productivity).
For example, researchers at Stanford recently created a machine learning tool that would mine EHR data to flag patients for screening for familial hypercholesterolemia (FH). Given FH’s low prevalence, manual chart review followed by genetic testing had been cost prohibitive. The introduction of algorithmic review raised the likelihood of a flagged individual of having FH to 80 percent. Having demonstrated that the technology worked during the pilot stage, management can now choose to embed the algorithmic tool into the EHR, providing the ability to scale across the organization.
With such an organization wide rollout, proof points such as time or dollars saved in screening and genetic counseling can be measured and evaluated. In the absence of a full implementation of the technology and implementation of an appropriate and novel business model, however, it would be premature to evaluate the impact of a new technological solution on global outcomes; that happens in the final stage of digital innovation.
The Dissemination Stage
The third stage, dissemination, is directed toward technologies that are sufficiently evolved to warrant a full evaluation. Only at this stage can the organization make the investment in the business model transformation to allow a full test of the innovation. At this point, the evaluation should focus on the original investment hypothesis, as well as the requirement to continue refinement of the technology and optimization of the business model.
For example, Stanford medical center piloted a specialty consultation service staffed by informaticists to solve the problem of “evidence gaps”—situations in which treatment guidelines do not provide clear recommendations. The consultation service uses data from the EHR to summarize the medical center’s own experience with treating similar patients and provides actionable reports to the requesting clinician. The physicians at the helm of this project have piloted the service on a small scale at their single center and hope to generate data on which clinical service lines or patients it would have the most impact.
At its preliminary stage, success meant that the software and workflow worked as intended. By limiting the scope of the pilot evaluation, it became easier to understand the underlying clinical hypothesis and for management to invest in scaling this solution hospitalwide, which has now occurred. The team is now rolling out utility testing to demonstrate return on investment and disseminating the shared software and workflow with other academic medical sites. Unlike many pilots, this intervention benefited from staged assessments and is able to “bridge the last mile” for such studies.
Making The Hard Work A Little Easier
The rise of corporate venture funds sponsored by provider systems may force reconsideration of the innovation model at health systems. Early versions of this concept were an elaboration of existing technology transfer offices, pairing promising innovations developed within the same health system with a funding runway to target potential commercialization. In the past 10 years, however, this segment has grown, with health systems now using their venture arm to source and evaluate new products and services. This exposes the investment side of the organization to the challenges of innovation from the perspective of the innovator. Here, extended sales cycles, unpredictable sales processes, and diffusion of decision-making authority can be understood as strategic challenges to both the venture and the operational arms of the system.
When venture funds are charged with strategic investments in innovation to improve internal health systems operations, they can pair their investments with a willingness to be an early customer of the same startup. This aligns the incentives of the innovator and the health system, and encourages the health system to develop a more structured evaluation and adoption process for the innovation. Relying on an investment approach for innovation, however, is limiting since many innovations might not be fundable from the perspective of a professional investor, and the requirement for an investment opportunity as a precondition to engagement would limit the rapid diffusion of innovations across the market.
Another development designed to lower the activation energy in the innovation ecosystem is the emergence of third-party organizations that serve to screen and validate novel technologies. One such example is NODE.Health, a nonprofit organization composed of more than two dozen health systems. NODE.Health seeks to de-risk the process of being an early adopter of new technology by aggregating deal flow, screening for viable solutions, offering guidance and expertise of the NODE.Health board members, and ultimately supporting chosen companies through the clinical trial process. Another such example is the Chicago-based AVIA, which directly contracts with health systems to source solutions to their pressing needs within the innovation community. Although AVIA does not participate in evidence generation, it harnesses the power of third-party validation to build confidence in the potential of an external digital solution.
The Way Forward
The excitement about digital solutions in the health sector will only continue to grow. Evaluating and implementing these solutions is a challenge for all of our health care systems. Given the complexity of the process we have outlined, it is clear that different organizations will approach this challenge with different levels of effort and enthusiasm. We believe that those organizations that make the effort to develop and implement an innovation pathway will be the ones that find the greatest impact from these solutions.