Inaccurate oxygen readings: the problem with pulse oximeters

Pulse oximeters are medical devices that measure how much oxygen is available in the body. This measurement, called the oxygen saturation, is a vital sign as important as heart rate or blood pressure. A low oxygen saturation can raise suspicion for lung or heart disease, and a very low saturation is life-threatening.

Invented in the 1970s, pulse oximeters have become ubiquitous. They’re used in almost every healthcare setting and are also increasingly found at home. Inexpensive, portable, battery-powered pulse oximeters are available from a variety of retailers and are becoming integrated into smartwatches and other wearable devices. It’s easy to see why the technology has been widely adopted: they’re convenient, small, noninvasive and provide near instantaneous information about an important bodily function.

However, pulse oximeters can be inaccurate. Movement, poor blood flow and outside light interference decreases their accuracy. And because the technology relies on light absorption, differences in skin pigmentation affects it, too.

Researchers have been reporting that pulse oximeters are less accurate among darker-skinned people for decades: among non-White patients, the device tends to overestimate the true arterial oxygen saturation, reporting a number higher than truth and making individuals seem healthier than they really are. As practicing intensive care physicians who finished training at the height of the pandemic, we’ve both witnessed firsthand how this inaccuracy can lead physicians astray when making clinical decisions.

The danger is obvious – a falsely reassuring oxygen saturation in a patient who is short of breath can lead to inadvertent misdiagnosis or undertreatment. In a recent study in JAMA Internal Medicine, we showed that pulse oximeters overestimate the true oxygen saturation in Black, Hispanic and Asian patients, leading to systematic underdetection of severe COVID-19 and delayed recognition of eligibility for COVID-19 treatment among Black and Hispanic patients.

In large databases of critically ill patients, such inaccuracies have been associated with less supplemental oxygen and predicted death and organ injury among racial and ethnic minority patients.

In response, pulse oximeter manufacturers have claimed that the problem is with others’ devices, in one case citing internal testing data reflecting tiny and “clinically insignificant” differences in accuracy between those with light and dark skin pigmentation. Yet, our investigation joins the growing chorus of real-world studies demonstrating that the problem is greater than originally believed and that it affects those whoare Black, Asian and Hispanic.

How could this be? One possibility may be the disconnect between how these devices are tested versus how they’re used. To be legally marketed in the United States, pulse oximeters do undergo clinical testing for accuracy. However, this is typically done in a controlled laboratory experiment with young, healthy volunteers lacking the litany of medical conditions characteristic of the patients we treat in the intensive care unit. If the device works perfectly in the laboratory but becomes more inaccurate among dark-skinned patients in the real world, can it be trusted?

Another possibility is the lack of diversity among the volunteers who come forward to test the devices. Current policies only require a minimum of 15% of volunteers to be “darkly pigmented”— in a validation study of 10 individuals, this amounts to only two—which is far below the number needed to correctly calibrate an algorithm for equal accuracy across the pigmentation spectrum. Even with more equal representation, the low numbers of volunteers typical of these studies is concerning; one person can contribute hundreds of data points, but this provides far less useful data than one hundred people contributing one data point each.

A final possibility is that unknown factors that align with race and ethnicity contribute to the inaccuracy. Investigators have shown that racial bias in pulse oximetry is variable over time, which would not be expected if skin tone produced a consistent bias. Race is a poor proxy for skin pigmentation, and understanding these other factors is an important step toward developing an oximeter that performs equitably in all patients.

In response to our and others’ studies, the Food and Drug Administration recently announced plans to convene the Medical Devices Advisory Committee to review the data and explore whether changes to regulatory requirements for pulse oximeters are warranted. We encourage the FDA to develop inclusive validation and testing policies for these devices that recognize the complexities and diversities of the people who depend on them.

Ultimately, the problem with pulse oximeters arose from individual and systemic biases that continue to shape the inequitable way minorities are treated by our healthcare system. Versions of this problem are shaping the conversation around how we define normal for lung or kidney function and in how to apply computer models to predict health outcomes. Addressing the problem is complex and critical. Acknowledging it is only the modest, first step.

By Dr. Tianshi David Wu, assistant professor, Baylor College of Medicine and Dr. Ashraf Fawzy, assistant professor, Johns Hopkins University