Fitness Trackers: Good at Measuring Heart Rate, Not So Good At Measuring Calories

By Patti Neighmond

Published May 24, 2017 at 2:33 PM EDT

Researchers had participants wear the fitness trackers while walking or running on a treadmill and while riding an exercise bike to determine how well the trackers measured heart rate and energy expenditure.

Sleek, high-tech wristbands are extremely popular these days, promising to measure heart rate, steps taken during the day, sleep, calories burned and even stress.

And, increasingly, patients are heading to the doctor armed with reams of data gathered from their devices. "They're essentially asking us to digest the data and offer advice about how to avoid cardiovascular disease," says cardiologist Euan Ashley, associate professor of medicine at the Stanford University Medical Center and Stanford Hospital and Clinics in northern California. And, being somewhat near Silicon Valley, he says he gets a lot of tech-savvy patients bringing fitness-tracker data to appointments.

The problem, he says, is that he just didn't know how reliable that data was. So, he and colleagues decided to study seven of the most popular devices and compare their accuracy to the gold-standard tests that doctor's use.

They looked at two metrics: heart rate and calories burned. For heart rate, the fitness trackers were compared to findings from an electro-cardiogram, or EKG. It turned out the devices were "surprisingly accurate", says Ashley. "Most devices most of the time were 'off' by only about 5 percent."

However, when it came to measuring how many calories a person burned, the findings were way off, says Ashley, showing a degree of inaccuracy that ranged from 20 percent to 93 percent, meaning 93 percent of the time the worst-performing device was wrong. Researchers compared the findings of the wrist devices to a sophisticated system of calculating metabolism which measures oxygen and carbon dioxide in people's breath.

"This is a very well-designed and well-done study," says Dr. Tim Church, a professor of preventative medicine at Pennington Biomedical Research Center at Louisiana State University who was not involved in the study. Church routinely consults with companies about how to introduce wellness strategies into the workplace. Being wrong 93 percent of the time means the findings from the fitness tracker are more "fiction than fact," he says, which can actually undermine a healthy diet. "It's just human nature. People are checking these inaccurate counts and they think they've earned a muffin or earned some ice cream and they're sabotaging their weight-loss program."

Church points to a study last year which found participants in a weight-loss program who also wore fitness trackers actually lost less weight than participants who didn't wear the trackers. "It's an instance of no information is probably better than having bad information," he says.

The Stanford study was published in the Journal of Personalized Medicine. It was relatively small, with 29 men and 31 women. In addition to the primary results, there were some other interesting findings. In certain groups of people — for example, those with darker skin, higher BMIs and men — the error made by devices was actually greater than for Caucasian women with a more healthy weight.

Researcher Euan Ashley and his team in a testing lab at Stanford University School of Medicine.

"So, for those for whom it might matter the most, who are trying to lose weight, the error was actually greater," says Ashley, who doesn't know why this may be the case. He speculates that it could be that companies use a fairly narrow group of people for testing the equations they use to measure heart rate and calories burned.

The study didn't look at how well devices count steps or monitor sleep or stress. The take-home message, says Ashley, is to not rely on the devices to measure total calories burned. Instead focus on eating what we know is a healthy diet, which is low in sugar and high in fiber, and to "eat not until you're full but until you're no longer hungry."

And, of course, people should exercise, he says, adding, "we have no more important intervention than exercise for the prevention of any number of diseases."

Makers of two devices, Fitbit and PulseOn say they remain confident in the performance of the trackers both in measuring heart rate and calories burned. In a statement, PulseOn questioned the study's methodology, saying that the high errors for calorie measurements "suggest that the authors may not have properly set all the user parameters on the device."

Mark Gorelick, the chief science officer at the device-maker Mio Global, says, "we agree that more accurate calorie estimation is important for the industry as a whole, since most individuals are monitoring calorie deficits for weight loss." The other device makers did not immediately respond to requests for comment.