(OT) Runners, don’t trust your Apple Watch Series 2 GPS routes!

We here at Precipice Labs are huge Apple fans.  Not only do we develop software targeted for their mobile platforms but we are former long-time employees.  Despite our love for the Apple universe, we know that sometimes they do drop the ball and we like to help set them straight.  In this case we are talking about the Apple Watch Series 2, at least with respect to it’s use as a training and racing device for serious athletes.  Admittedly, I am not close to the level of a professional athlete and perhaps on the fringe of what one would refer to as a “serious athlete.”   I’m serious in that, when running a race, I have a goal pace and time in mind.  I don’t expect to win or even place in my age group.  I think that about covers 90% of the people entering any given organized public race.  We put in lots of training hours and do our best to hit a goal finishing time.

For years I have been running with a Garmin Forerunner GPS watch.  It’s fine.  It always reported fairly accurate statistics and got the job done.  A few months back, after about 3 years of heavy use, the touchscreen stopped working and I was forced to look into buying a new GPS watch.  Apple had just released the Series 2 of their Apple Watch.  I hate carrying a phone with me on long runs so I was never interested in the prior version of the Apple Watch, which relied on a paired iPhone for GPS data.  Finally Apple added GPS directly to the watch, making it possible to leave the iPhone at home and have the watch track all of the important running data that I rely on.   When training for a race, knowing your pace and distance is vital information.   During the race itself, even more so.  For us non-elite athletes, pacing by feel is not so easy and having this little device tell me how far I’ve gone and how fast I’m running is awesome.

In addition to adding the GPS to the watch, Apple has cobranded it with Nike.  What’s the message there?   Apple is telling us that this watch is meant for serious athletes, not just office workers who want to track how many steps they’ve taken today.  Nike!  The company that makes shoes for Michael Jordan and Serena Williams and Mo Farah!   This watch is meant for professionals!  Surely it will work for me.  I am a serious runner!

So, I bought the Apple Watch.  It definitely has some quirks that I had to get used to, and the running apps that are available leave a lot to be desired.  I settled on the Strava app and used it several times as I was preparing to run in the LA Marathon in March of 2017.  Everything was working fine until the day of the race itself.

Unlike the Garmin watch, which takes minutes to lock onto satellites when you turn on the GPS, the Apple watch starts tracking instantly when you start a run.  At first I thought it was because Apple just lets you start without a good lock on the satellites and accuracy would suffer.   However, looking at my run maps I see that this is not the case.   The starting location for my runs is mapped fairly accurately.  However Apple does it, they seem to have solved the problem that Garmin users still suffer through.  So score one for Apple!  Unfortunately, this is where Apple’s advantage ends.

So, about the LA Marathon…Everything was working fine with the watch at the start of the race.  However, a few miles in I noticed that it is reporting paces that are WAY off what I’m actually running.  Also, I noticed that the mileage reported by the watch was also way off from the official mile markers that were posted for the race.   For example, as I passed the 3 mile marker the watch is telling me that I ran 3.7.   At the 4 mile mark, according to the watch I had run  over 5.   And it got worse from there.   At this point I stopped even checking the watch for anything other than the elapsed time so that I can do the mental math to figure out my pace myself.

As anyone who has used a GPS watch for running knows, the accuracy is never perfect.  It’s even less so during a large race like the LA Marathon.   Runners aren’t always able to take the ideal line through corners and there is a lot of weaving through other runners on the course so we expect a little bit of inaccuracy.  After a 26.2 mile race it isn’t uncommon for a decent GPS watch to report 27 miles or so.  However, the problems with the Apple watch were WAY beyond what I consider acceptable.  But why?  Why did a watch that performed acceptably during training fail so badly when it really mattered to me?  I decided to dig a little deeper to find out.

The first thing to note is that all of my normal training runs took place in mostly open areas where a good GPS signal was available.  The LA Marathon, on the other hand, traverses through areas of downtown Los Angeles where many large building can hinder the GPS  signal required for the device to get a good handle on your location.  This is precisely the situation where the Apple Watch fared so poorly.  With large buildings preventing a good location reading, wouldn’t all GPS watches have the same problem?   Not necessarily.

Let’s take a look at some real data to prove it.   The Strava.com website has an amazing feature called “Flybys.”   This lets you see all of the runners that ran the same route as you did at around the same time.  Additionally, it lets you look at each individual runner to see what specific device they are using to track their run.  Given that Strava is a very popular service and the fact that thousands of other Strava users were running in the LA Marathon, the public has access to literally thousands of data points to see how different GPS devices perform under similar conditions.  Examining this data was quite enlightening.   Admittedly, I did not do a scientific study and only spent a couple of hours looking at this information.   I did not exhaustively study every device used to reach the conclusions that I’m presenting here.  My hope is that Apple will do more thorough research into this and fix their problems.

First, take a look at the route that my Nike Apple Watch Series 2 recorded through a portion of downtown LA with a particularly bad GPS signal (represented by the black line):

AppleWatch

Wow, that is quite the circuitous route!  Note that the true marathon route was actually run on S. Main Street and 1st Street.  The route shown above makes it look like I had done some heavy drinking of something other than Gatorade before the race.  It’s no wonder that my watch was reporting back to me that I had run literally several miles farther than I did so early into the race.

Thinking that perhaps my specific watch has some hardware issues, I hunted through Strava to find another Apple Watch user.   Here we go:

AppleWatchAgain

We can see basically the same kind of jagged path.

Now let’s take a look at a Garmin device (Forerunner 630) :

GarminForerunner220

Yes, we can see the affects of the bad signal here as well but not nearly to the extent of the Apple hardware.  The path of the route is not drawn directly on the road, adding to the reported distance.  However, because the line is kept relatively smooth, the distance added to the run is a mere few percentage points as opposed to the Apple watch which added 20-30%, possibly more, to the actual distance for me.

Here’s another Garmin (Fenix) showing similar results:

Screen Shot 2017-04-06 at 7.37.54 PM

 

My takeaway from this is that the watch knows when it is getting a poor signal.  Or rather, the software that runs on the watch and manages the location services, knows that the signal is poor.  Adjustments can be made by the software to smooth out the route and attenuate the error.  It appears to me that Apple is not correcting for a poor GPS signal and we are left with a watch that is simply worthless under certain conditions.

Despite all of this, I am still a fan of the watch and will use it during training when in a familiar area.  However, I will be using a Garmin device when it matters most during races.  The Nike Apple Watch Series 2 is simply not ready for “prime time.”  Apple still has some work to do.