“Vaccines are tugboats of preventive health” - Dr. William Foege
Can we accurately forecast vaccine consumption? Is it possible to preemptively identify vaccination dropouts? And ultimately, can we make all of healthcare predictive?
In Isaac Asimov’s Foundation trilogy, the genius mathematician Hari Seldon leverages statistical laws of mass action to predict the future of large populations. Then, foreseeing the great galactic empire’s imminent demise succeeded by a prolonged dark age, he hatches a plan to preserve knowledge and save humanity.
I distinctly remember chills running down my spine when I first read this spectacular sci-fi trilogy some two decades ago. Not only did that story capture my imagination, but it seeded a lifelong ambition in me to become a futurist.
Fast-forward to today: how far have we come in predicting the future? Such strong or general Artificial Intelligence (AI) that may enable us to see into the future of all mankind seems far-fetched, like an Asimov novel. Yet we have come leaps and bounds in developing powerful models for making highly accurate predictions, albeit on much smaller groups of individuals about specific events.
We’re now able to use prediction in fields like e-commerce, financial technology, information technology and healthcare. Personally, as Head of Applied AI at macro-eyes, an AI company aimed at transforming the delivery of care, I am focused heavily on the relationship between health and machine learning (ML). The importance of optimizing healthcare cannot be overstated because of the direct impact on lives, I’ve seen this firsthand through my work in five different countries.
Saving thousands of lives and millions of dollars through vaccine forecasting - how does it work?
On one end, having less stock than is needed leads to shortages, or even the complete exhaustion of vaccine inventory altogether in a phenomenon known as a stockout. This can subsequently result in a disruption of vaccination schedules, leaving entire populations vulnerable to infectious disease.
On the other end, having too many vaccine supplies results in wastage. This is of special importance in the developing world where frequent adverse events such as power outages make longer-term storage of large quantities of vaccines and other temperature sensitive pharmaceuticals unfeasible. Wastage results in funder hesitancy, high program cost, and a lack of program sustainability.
At the end of the day, maintaining an efficient vaccine supply chain depends on accurate forecasting of utilization.
What do we need to make accurate forecasts? Surprisingly not what you’d think.
Broadly, there are two essential components to accurate forecasting – a quantitative framework (typically machine learning) that can crunch data to make these predictions. And of course, the data itself.
In sharp contrast to domains such as social media, finance or, e-commerce, where large, multidimensional datasets can be accessed relatively easily, using healthcare data is frequently fraught with multiple, unique challenges:
-
Challenges to data access due to issues surrounding privacy and sensitivity. Generally speaking, from health facility level all the way up to country governments, access to patient and caregiver data is heavily vetted. This safeguarding of personally identifiable data is rational and of course, necessary. However, it also creates an additional and at times an insurmountable barrier for institutions that intend to use this data solely for developing healthcare solutions for public benefit.
-
Issues of data quality. Ranging from grossly incomplete or missing data for key fields, all the way to badly built data tables, data quality can often become a bottleneck to any other downstream analyses. Data with a large fraction of missing values raise questions ranging from “What is the best method to fill up or impute missing data?” to “Should we even try filling up the missing values or just drop the whole column or rows of data?” Some of these questions are made more complex by additional issues related to data quantity.
-
Inadequately small datasets. Datasets, especially those that relate to on the ground vaccine consumption, are typically small. This, combined with issues of data quality, can lead to very difficult data preprocessing choices such as choosing a very liberal data filtering threshold to keep most of the data (even if it is of low quality).
-
Imbalanced data. When attempting to model vaccination predictions, more than 70-80% of the data may represent fully immunized individuals, while only the remainder will contain data for individuals who drop out. Highly custom-tailored analysis protocols such as the use of sophisticated data rebalancing techniques, assigning weights during machine learning model building and using meaningful forecasting metrics all need to be leveraged.
-
Data integration. For effective vaccine utilization forecasting, relevant data may include details of the vaccine, characteristics of the catchment population in the region of interest, number and nature of hospitals or other health facilities in the region, real-time information regarding local events or campaigns, weather and even ease of physical access to the health facility. This translates to gathering diverse types of data including tables, images (both on the ground and Satellite based), text messages from health workers and other ground personnel and, even audio-video data. Each data type warrants different quality assessment protocols and preprocessing steps before they can be used as input to ML models downstream. Again, developing optimal ML frameworks to integrative machine learning across such diverse datasets in an active area of research.
If data collection, quality assessments and preprocessing are the prerequisite initial steps in building a prototype forecasting solution, then the next core step is machine learning.
Assume that we managed to gather enough, usable data. How then do we start making predictions?
What is required here is a mathematical framework embedded inside algorithms that is powerful enough to not just crunch large numbers but is able to make non-linear connections between them.
An attractive paradigm already exists to do this – Machine learning (ML)
Drawing analogies with software programs is one useful way of understanding ML. Software programs, or “code”, take input data, process it and gives us the output. The processing machine inside a software program, however, is hardwired in code and has no predictive ability. In contrast, ML accepts both input data and their corresponding outputs generated by any real world or artificial process, then figures out the program to connect the two. In other words, ML attempts to learn the mathematical rules or the equation that connect two different sets of numbers, on either side.
In vaccine utilization forecasting, variables such as details of the vaccine, the type of health facility where it’s being administered, accessibility of the facility, etc. constitute numbers on the left side of the equation while regional daily or monthly vaccine utilization are numbers on the other side of the equation.
ML, then, tries to discover the exact mathematical formulae or rule that maps the first set of values (the variables) to the second set of values (daily or monthly utilization). It does so by learning those rules from previous, archival data. We are then able to use that formula to predict future vaccine utilization for any health facility or region.
Machine learning & vaccination. Match made in healthcare.
Using ML to forecast demand for vaccines and pharmaceuticals, and preemptively identify immunization dropouts, offers a powerful paradigm to optimize supply chains. But how extrapolate and robust are such models ? ML models are highly successful at capturing the mathematical relationship between a bunch of variables from data at any location. However, the fundamental causal events defining the relation between different variables might vary significantly between different regions. This warrants us to embrace some element of causal thinking to reinforce our belief in ML model building and improve its robustness. Instead of relying blindly on big data to answer our questions, causal inference which is rooted in counterfactual thinking, should also be made part of any grand predictive frameworks.
"The importance of optimizing healthcare cannot be overstated because of the direct impact on lives." - Ramkumar Hariharan
What are some other attractive healthcare venues for path breaking innovations in AI?
From automated pathology slide evaluations to cardiac arrhythmia detecting smart wearables, we have already ushered in a revolutionary era of AI powered healthcare solutions. Right now, the majority of these projects are happening in the developed world and in settings where data is bountiful. It is our duty to bring this powerful technology to also bear upon problems in developing countries and where high-quality data is hard to come by. Apart, vast improvements in computer vision ML, powerful natural language and network ML models will all continue to drive progress in AI for the next few years. The real challenge then is to try to envision the long-term future—What might AI look like in 2050? To answer this question, we revisit the beginnings of this article in a different light.
The invention of the submarine (20,000 leagues under the sea), human exploration of the moon (First men on the moon), and even the rise of robots (I, Robot) were all milestones envisioned many decades before the actual events by prolific, and imaginative creators of fiction. With startling developments in AI happening at breakneck speed, intellectually provocative science fiction might provide the only way to catch a glimpse of AI 2050; we can always temper it with reality by using the known laws of science to place an upper bound! Let me wrap all this up with my favorite Asimovian quote:
“Science fiction writers foresee the inevitable, and although problems and catastrophes may be inevitable, solutions are not.”