Researchers at Stanford University tracking reliability from smartwatches and fitness trackers have found the Apple Watch to provide the most accurate heart rate monitoring, though all devices fell short in calorie counting.
The study locked the original Apple Watch, Samsung Gear S2, Fitbit Surge, Basis Peak, PulseOn, Mio Alpha 2 and Microsoft Band against FDA approved equipment, with 60 volunteers strapping on up to four devices and taking part in a total of 80 physical tests.
Read this: Heart rate monitors - chest strap vs wrist
So what exactly was found? Well, the Apple Watch was able to achieve the highest heart rate accuracy, seeing an error rate of just 2%, followed by the Basis Peak and Fitbit Surge. Samsung's Gear S2, meanwhile, offered up an error rate of 6.8%, the highest on test and just outside the study's acceptable buffer.
However, while there were some strong performances when monitoring heart rate, all devices were found to fare poorly when it came to tracking energy expenditure. Fitbit's Surge was the strongest in this area, but still only minimised the error rate to 27.2%. And while high intensity exercise can often throw wrist-based devices off, it was actually low impact activities that caused the most inaccuracies.
According to Stanford researcher Euan Ashley, this is due to the differences in how people exercise.
"The heart rate measurements performed far better than we expected, but the energy expenditure measures were way off the mark," Ashley said. "The magnitude of just how bad they were surprised me. People are so variable. Some people walk smoothly and others waddle along, and that has an impact."
Of course, while the findings suggest taking calorie data with a pinch of salt, it's also important to keep in mind that the majority of the devices used in this study aren't the freshest options available. Apple, Samsung and Fitbit have all released new iterations of these watches, while the Microsoft Band was joined by an updated sibling and is now discontinued.
The Stanford team's next study will evaluate the smartwatches and trackers as volunteers go about their usual walking and exercise routines, not on a treadmill in the lab.
Update: Fitbit has released the following statement in response to the Stanford study:
Fitbit trackers show an estimated total number of calories burned based on users' BMR (basal metabolic rate) and activity energy expenditure (AEE). Fitbit uses a scientifically validated estimate of BMR based on height, weight, age, and gender information that users provide when setting up their Fitbit account. When using a Fitbit device with PurePulse continuous wrist-based heart rate tracking, the calculated heart rate is used to help estimate AEE to provide a more accurate result than a step-only estimate.
Source: Stanford Medicine