Map Yourself and Your Photos

I own a ten year old Garmin Nuvi car GPS unit. It's useful for directions when you're driving, but also produces "tracks" that you can retrieve later in the form of .gpx files. "GPX" is the "GPS Exchange Format," and it's an XML schema. The GPS has to be mounted as a USB device and it requires a bit of digging on its file system to find the file, but it's there on my GPS and probably on yours. That data (after the addition of some helpful newlines) looks like this:

<trkpt lat="49.097458" lon="-66.681009"><ele>-3.23</ele>
<time>2017-09-10T12:27:41Z</time>
<extensions><gpxtpx:TrackPointExtension><gpxtpx:speed>2.75</gpxtpx:speed>
<gpxtpx:course>24.00</gpxtpx:course>
</gpxtpx:TrackPointExtension>
</extensions>
</trkpt>
<trkpt lat="49.097532" lon="-66.680892"><ele>-2.75</ele>
<time>2017-09-10T12:27:44Z</time>
<extensions><gpxtpx:TrackPointExtension><gpxtpx:speed>5.49</gpxtpx:speed>
<gpxtpx:course>63.53</gpxtpx:course>
</gpxtpx:TrackPointExtension>
</extensions>
</trkpt>

The data format is very simple: (latitude, longitude), date stamp, speed, bearing. Even without an XML parser, just using standard text parsing, this is a surprisingly programmatically readable format. It takes measurements at frequent and consistent intervals regardless of what you're doing (because it has no idea what you're doing - that becomes an issue later).

I started using the GPX files from the GPS when I realized I had photos from trips that I had no idea where they were taken. I had a photo with a datestamp, and I knew what city I was in ... but no idea what building or sculpture or garden I had photographed. So I learned how to use an application to geo-tag my camera's photos based on matching GPX data. Rule one: make sure that the clocks on the GPS and the camera are in sync! The GPS should get its time (with considerable accuracy) from the satellites it also uses for location, but on most stand-alone cameras the time is set by hand. With that bit of technology figured out, I found that one minute out of sync means you can be as much as 50 metres out of place even at walking speed. And worse, GPS units claim perfect precision: do you see a "±" measure on those numbers? Do you believe they're 100% correct? There's a reason Google Maps marks your position on a map inside a big blue circle of varying size: the accuracy of a GPS signal varies from moment to moment based on your altitude, your distance from the equator, the weather, the density of the bag you're carrying the device in ... etc. GPS tags also don't tell you which way you're facing. So I discovered that GPS-tagging my photos didn't help me identify their location nearly as much as I thought they would. Imagine you're on a hike, and you take this great photo from a lookout. But the GPS is off by 50 metres for the reasons given, and Google Maps says you took that photo right inside a bush. Worse, where Google Maps and the photo claims you were - is right in between two lookouts. Do you remember which came first? Or what either of them looked like? And its even worse inside a city.

And then there's the weight: carrying a car GPS when you're hiking is a pain. It's a bit large, and if the hike is over two hours you're going to need an external USB battery - a big one is recommended, and that means more weight. Not only that, every time I reach into my bag for my camera I have to make my way around the USB cable connecting the battery to the GPS. You might think you could make it smaller by getting a hiking GPS, but those things are highly specialized, expensive, and some of them are quite inaccurate. It's not a path I've followed.

I've almost entirely solved the place-name-to-camera co-ordination problem by taking better notes when I'm travelling: some in writing, but mostly in the form of taking pictures of plaques, building names, and trailhead maps whenever they appear. In effect, the photos become their own identifiers.

So I thought I'd experiment with the Google Maps app. We all carry a phone 24x7 now, right? In Google Maps, you can do this to turn your "Location History" on (or off): Hamburger menu -> Your timeline -> "..." menu -> Settings -> "Location History is off" - tap to change. There are several other ways to turn your location history off, including in a browser because your history is associated with your Google account, not specifically your device. But this is the one I found on mobile. The availability of "Location History" may also be affected by "Location Services" being on or off - that's found on the same menu.

Once that's on, Google makes an intermittent recording of your location to its servers. On the phone, I could access the history with: Hamburger menu -> Your timeline - and then tapping the timeline at the bottom. But I find if you turn it off again, the old timeline becomes unavailable on the phone (although still available through Google's servers).

In the name of privacy, I highly recommend you turn your timeline off except when you have an explicit reason to have it on (like going hiking). Your phone company will still track you, but Google won't (or at least probably only when you use Maps).

To retrieve your timeline information:

https://www.google.com/maps/timeline

Click on the cog/gear in the lower right, and choose "Download a copy of all your data." You'll then see the absolutely incredible and horrific selection of stuff you have with Google that can be extracted, and so you sit there with your jaw open wondering that one company have so much data on you ... Turn off the extracts for all the non-Timeline stuff, and press the button. This takes a LONG time - but "don't worry - we'll send you an email link when we're done." (The email appears to have come about six hours after I made the request.) It's in JSON format: readily parsable to a computer, a horror show to a human (although as the first part suggests, easy to program for if that's your bent).

There's another method: go to the same URL ( https://www.google.com/maps/timeline ), select a particular day from your timeline, then click on the gear in the lower right and choose "Export this day to KML". Note the distinction: this only allows downloading KML data one day at a time, while the JSON download can do weeks, months, or years. But the KML is also immensely faster, returning its data immediately.

Here's a piece of the JSON:

{
  "timestampMs" : "1522529207466",
  "latitudeE7" : 440075474,
  "longitudeE7" : -769864452,
  "accuracy" : 50,
  "velocity" : 17,
  "heading" : 233,
  "altitude" : 80,
  "verticalAccuracy" : 48
}, {
  "timestampMs" : "1522529114538",
  "latitudeE7" : 440149490,
  "longitudeE7" : -769717300,
  "accuracy" : 43
}

The values in the JSON vary per entry, which is ... weird, although within the standard. The entries always included are:

  • timestampMs
  • latitudeE7
  • longitudeE7
  • accuracy

Lat and Long would seem fairly obvious but for that "E7" appended on the end. Which means that they've shifted the decimal place seven places to the right and you just have to shift it back again. The timestamp is in classic "Unix Time" (https://en.wikipedia.org/wiki/Unix_time) - loved only by programmers, it's the time since January 1, 1970 ... in seconds. EXCEPT it's 13 digits instead of 10 because it's using millisecond accuracy. As an intermittent but long-time programmer, I've come to understand that as insane as Unix Time sounds, it's actually incredibly useful.

Other entries - not always present - include:

  • altitude
  • heading
  • locations
  • velocity
  • verticalAccuracy

My first big question was, how frequently is this measured? The data I extracted was for a 24 hour period and included 92 entries. Here are the time gaps in minutes:

2, 1, 4, 0, 20, 3, 1, 6, 5, 2, 5, 2, 3, 5, 14, 1, 9, 6, 14, 5, 5, 3, 16, 10, 10, 11, 15, 8, 7, 5, 2, 8, 4, 5, 1, 50, 25, 2, 3, 14, 6, 14, 6, 4, 8, 2, 3, 3, 2, 6, 2, 0, 12, 5, 16, 14, 14, 7, 2, 0, 2, 2, 2, 8, 3, 4, 5, 10, 5, 15, 0, 6, 14, 14, 4, 2, 4, 2, 14, 1, 14, 10, 1, 11, 14, 6, 0, 3, 4, 14, 2

I'd like to claim this has something to do with them checking in and finding you're in motion and taking a new recording, but not bothering to record if you're not moving. But I was on a bus tour, and in motion a great deal. I'm stumped by the logic here. (This was done with a quick-and-dirty Bash script that rounds down 59 seconds to 0, but correcting for that wouldn't make this make any more sense.)

The KML data is based on the same data, but quite a bit more detailed - partly because XML is even more verbose, and also because Google has included a lot of speculation about where you are. As with their "Your places" data (such as your maps, which can also be exported as KML), the KML is exported as a single line all run together making it really hard for a human to parse. I would have preferred they didn't embed my email address 15 times (literally - what, they expect it to change between my stops for the day?) and instead had added the newlines to format the file ...

Here's a chunk of data reformatted for human readability. In this case, we were staying in one place:

<Placemark>
    <name>Waupoos Estates Winery &amp; Restaurant</name>
    <address>3016 County Road 8, Picton, ON K0K 2T0</address>
    <ExtendedData>
        <Data name="Email">
            <value>giles@example.com</value>
        </Data>
        <Data name="Category">
            <value>Winery</value>
        </Data>
        <Data name="Distance">
            <value>0</value>
        </Data>
    </ExtendedData>
    <description>Winery from 2018-03-31T19:29:28.348Z to 2018-03-31T20:45:14.538Z. Distance 0m </description>
    <Point>
        <coordinates>-76.97170609999999,44.0145718,0</coordinates>
    </Point>
    <TimeSpan>
        <begin>2018-03-31T19:29:28.348Z</begin>
        <end>2018-03-31T20:45:14.538Z</end>
    </TimeSpan>
</Placemark>

Here's the block from immediately after that, when we were in the bus and driving:

  <Placemark>
    <name>Driving</name>
    <address>
    </address>
    <ExtendedData>
        <Data name="Email">
            <value>giles@example.com</value>
        </Data>
        <Data name="Category">
            <value>Driving</value>
        </Data>
        <Data name="Distance">
            <value>80762</value>
        </Data>
    </ExtendedData>
    <description>Driving from 2018-03-31T20:45:14.538Z to 2018-03-31T21:56:50.001Z. Distance 80762m </description>
    <LineString>
        <altitudeMode>clampToGround</altitudeMode>
        <extrude>1</extrude>
        <tesselate>1</tesselate>
        <coordinates>-76.97170609999999,44.0145718,0 -76.97170609999999,44.0145718,0 -76.97173,44.014949,0 -76.9864452,44.0075474,0 -77.0355518,44.0102675,0 -77.1017751,44.0016979,0 -77.1807749,43.9947239,0 -77.2238329,44.0034283,0 -77.3173648,44.0155797,0 -77.4396911,44.0063735,0 -77.5478718,44.0264748,0 -77.6855269,44.1024685,0 -77.9058617,44.021827,0 -77.9059834,44.021992,0 -77.9059834,44.021992,0 </coordinates>
    </LineString>
    <TimeSpan>
        <begin>2018-03-31T20:45:14.538Z</begin>
        <end>2018-03-31T21:56:50.001Z</end>
    </TimeSpan>
</Placemark>

Google's guesses about where I was were disturbingly accurate - including its ability to determine that I took a TTC bus home at the end of the day (presumably because it compared my path, speed, and timing with that of a bus route it already has data for).

I haven't tried co-ordinating the output of Google's timeline with photos yet. There are many sites that allow you to upload GPX and KML files, and then plot that information on a graph, a couple of which I mention below. I think the detailed GPX data looks far better - if what you're looking for is tracks rather than photo tagging. For co-ordination with photos, Google's KML should be sufficient as it seems to take note when you stay in place for a while - even if it doesn't recognize the location. I would still prefer the simpler and more detailed GPX format ... if it weren't for the weight of the devices.

Bibliography