AI for health: the impossible necessity of unbiased data

under science society health AI chronicle AI chronicle   Read time: 3 min.

Is unbiased data important to build health AI? Yes!

Can there be unbiased data? No!

The notion of bias depends on the intended use.

---

In medicine, we have seen the importance of tuning devices and decisions for the target population. The problem is not limited to AI: pulse oximeters, that measure oxygen saturation, do not work well on dark skins; cardiac procedures were adjusted to symptoms and anatomy of men, while those of women differ. These issues arose because the corresponding groups were underrepresented in the clinical studies.

So when we build AI, we need to make sure that they are not trained on biased data.

But unbiased data is hard, as it goes beyond sampling the right population of individuals. Indeed, the data we have is the result of a historical set of choices: Who do we measure? Which measurements? And what led to their condition? Beyond health, consider for instance salaries: we can train a model from historical data to tell us what should be the right compensation for a given individual. But it is just going to capture and repeat historical biases, such as paying less women that are as qualified as their male counterparts.

Here we see that the notion of being unbiased embeds societal and ethical values: Should Olympic-level gymnasts and football players be paid the same thing? How about men and women with the same job description?

And now to go back to medicine, there is another critical aspect, which is that of cause and effects, which is central to taking decisions. To take a simple example, if we were to compare the health outcome of individuals after two days at the hospital to those who did not go to the hospital, we would conclude, incorrectly, that a hospital is a very dangerous place, as individuals there are in a worse shape. The problem is, of course, that we are comparing individuals that are not comparable as they have a different baseline health. A health intervention is given for a reason, so it is given to a specific population: insulin is given to diabetics. Building a model, an AI, that can decide on health intervention requires compensating for the difference between the treated and non-treated individuals.

Here also we have a case of bias. The bias is with regards to the data required to answer the question on the effect of the intervention, where both populations are comparable. More generally, we are seeing once again that the data are always the result of a historical set of choices, and these choices condition the statistical relationships in the data. And AIs build on the statistical relationships.

What we see here is that the notion of bias depends on the intended use: it depends on the target population, but also on the target intervention. So there really is no absolute notion of unbiased data. There is just the notion of data that are well suited to a particular goal.

Go Top