If you're a fan of your new, wearable fitness tracker, it must be reassuring to glance down at your wrist and see your heart rate measurement as you start your run. And as the numbers climb, there's gratification knowing that your heart is pumping – and your calorie burn is underway.
For purchasers of these cutting-edge products, real-time feedback on personal, physiological changes in your own body is thrilling to witness. That's a big reason why fitness trackers comprise a hot and growing market, with worldwide expected sales this year to reach 44.1 million units.
OK now, while having this newfound access to your personal health data is exciting and empowering, there's just one problem with these trackers ... some results are not close to being accurate – and in some cases the data are way wrong.
That's the key finding of new study from the Stanford University School of Medicine, which determined that while trackers can accurately monitor heart rate, "none of the seven devices measured energy expenditure accurately." Worse yet, according to the release announcing the findings, "the most accurate device was off by an average of 27 percent," while "the least accurate was off by 93 percent."
"We found that in most settings, heart rate measurements were within acceptable error range (5%)," wrote the authors in their paper titled, "Accuracy in Wrist-Worn, Sensor-Based Measurements of Heart Rate and Energy Expenditure in a Diverse Cohort," which was published today in the Journal of Personalized Medicine. "In contrast, none of the devices provided estimates of energy expenditure that were within an acceptable range in any setting."
Since consumers view the data being received as health related, which is occasionally shared with their doctors, researchers set out to confirm the accuracy of the statistics. Seven devices were tested – Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2 – by 60 participants (31 women and 29 men).
The volunteers were monitored while walking, running and cycling, at both low and high intensity. Of the seven devices, six achieved an error rate of 5 percent or less when tracking heart rate (HR), which was determined by comparing the results to those collected for each volunteer using a "medical-grade electrocardiograph."
"Across devices and modes of activities, the Apple Watch achieved the lowest error in HR, 2.0%," the authors wrote, "while the Samsung Gear S2 had the highest HR error, 6.8%." According to Euan Ashley, (foreground in photo, credit Paul Sakuma/Stanford University) professor of cardiovascular medicine, of genetics and of biomedical data science at Stanford, "[t]he heart rate measurements performed far better than we expected."
But the devices failed spectacularly when tested for energy expenditure, or EE.
"Error in estimation of EE was considerably higher than for HR for all devices," states the paper. "Median error rates across tasks varied from 27.4% for the Fitbit Surge to 92.6% for the PulseOn." Added Prof. Ashley, "the energy expenditure measures were way off the mark. The magnitude of just how bad they were surprised me."
The research team did not know precisely why the EE figures were so off, but it did have an explanation for the differences in accuracy between heart rate and energy expenditure.
Heart rate, they said, is measured directly from the body, so there's little to get in the way of a true reading. But EE is derived indirectly, as each device it outfitted with its own algorithm to estimate how many calories are being burned.
"It's very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone's fitness level, height and weight, etc.," stated Anna Shcherbina, a graduate student and the paper's co-lead author.
While it's enticing to adopt new technologies so that we can pay closer attention to our health, Stanford's team points out the value of first confirming the claims made by product manufacturers. And that's especially true when the information provided by these devices is used, as the authors state, to "increasingly inform clinical decision making. This makes transparency regarding benefits and limitations of paramount importance."
