Posted on January 26, 2022

This paper represents a study of algorithmic bias, specifically, the underlying racial bias in a predictive model widely used in healthcare. It shows that even though an specific way of modeling the predictive algorithm seems to work, such a way could implicitly be disregarding important factors that might translate into sensitive biases.

The motivating research question is: “Can some proxies for ground truth be a source of algorithmic bias?”; while the concrete operationalization is: “What other metric, or metrics, can we take into account when developing predictive models related to health in order to reduce racial bias?”

The authors make several relevant recognitions. For instance, they suggest that there is not just one source of bias (i.e. the people behind the algorithm and the data itself). Furthermore, they are quick to mention the assumption made by the manufacturers of the algorithm (i.e. “those with the greatest care needs will benefit the most from the program”). That means that the manufacturers assumed the decisions they took when developing the model were right, while not realizing they were missing a bigger picture.

The authors’ approach is very descriptive throughout the entire process. They discuss the different metrics for algorithmic bias, suggesting they assessed them and chose to go with calibration; and clearly express how to calculate C and H from their analysis, so that other researchers can easily reproduce the results.

This is a study that aims to shine a light on racial biases from an ‘observational analysis’ system that applies forecasting as its research strategy. In order to replicate the results, the authors also make use of such a strategy.

The authors showed that a widely used algorithm in the US healthcare system is racially biased. They explain that basing the system on future costs predictions is not entirely wrong; but, it’s also not the only reasonable choice. The fact that, conditional on health, Black patients generate lesser medical expenses than White patients might be a possible reason to explain the model’s bias: even though the total costs are similar on average, the hidden truth is that, on average, Blacks individuals have more critical illnesses with lesser costs. Thus, there is a disparity when choosing to whom to provide services given their needs (i.e. a Black person with more illnesses might not be chosen because the predicted costs from a less ill White person might be higher).

Some characteristics of the big data used in this research are: big (i.e. in the early stages of the study they had around 130,000 patient-years data points, afterwards the results were replicated with a dataset of about 3.6 million commercially insured patients), always-on (i.e. yearly measurements of patient’s health that show unexpected events), non-reactive (i.e. measuring illnesses don’t usually cause illnesses). Also, the data is inaccessible (i.e. not everyone can access this particular dataset), as well as sensitive (i.e. medical data). It’s important to mention as well that in the early stages, the real data that the model was trained with was inaccessible since this was an independent study, not related to the manufacturer.