Back to the Future: What Predictive Decision Support Can Learn from DeLoreans and The Big Short

In the third blog in our series on artificial intelligence (AI) and machine learning (ML)-driven predictive models (data analytics tool or software) in health care, we discussed some potential risks (sometimes referred to as model harms) related to these emerging technologies and how these risks could lead to adverse impacts or negative outcomes. Given these potential risks, some have questioned whether they can trust the use of these technologies in health care. We are encouraged to see that some stakeholders are demonstrating that a predictive model is fair, appropriate, valid, effective, and safe (FAVES), rather than amplifying biases or harms. Some stakeholders are indicating this through descriptions of the processes used to develop the model and minimize risks, evaluation of the model’s performance (often described in peer-reviewed literature and according to nascent reporting guidelines), and clear description of how and when the model should be used.

Medigy Insights

There are potential risks and negative outcomes associated with the use of artificial intelligence and machine learning-driven predictive models in healthcare, which has led to concerns about their trustworthiness. To minimize these risks, it is important for stakeholders to demonstrate that these models are fair, appropriate, valid, effective, and safe. However, the information necessary to assess the quality of these models is often unavailable, leading to a lack of consistency and trust in their use. This lack of information can lead to a "market for lemons" in which only lower quality models are available and established models are difficult to replace with newer, potentially better models. To optimize the use of these predictive models in healthcare, it is important to address these issues of information availability and quality uncertainty.