Reverse engineering Mapillary Tools

trekview-dgreenwood · January 1, 2025, 3:51pm

This content was originally posted to the Trek View blog on 2023-02-17. Some of the information found in it may now be outdated.

Unfortunately, Mapillary does not currently have a documented endpoint for uploading photos or videos directly. They do however have a command line tool to handle photo and video uploads, Mapillary Tools.

Install it as follows…

mkdir mapillary_tools
cd mapillary_tools
python3 -m venv mapillary_tools_venv
source mapillary_tools_venv/bin/activate
pip3 install mapillary_tools
mapillary_tools --help

Uploads

Uploading Photos using Mapillary tools

Upload a timelapse sequence as follows;

mapillary_tools process_and_upload <DIRECTORY OF TIMELAPSE PHOTOS> \
  --user_name trekviewhq \
  --desc_path tmp/mapillary_description_file_1.json

A few things:

--user_name I pass my Mapillary username using the flag. The command line will ask me for a username and password to actually authenticate before uploading any files.
--desc_path I want to store a copy of the description file Mapillary tools creates to help identify the actual sequence Mapillary creates on the server. Here’s the schema of the JSON structure for the description file.

Here’s an example of image processing:

mapillary_tools process_and_upload test_photos/tes-max-003g \
  --user_name trekviewhq \
  --desc_path tmp/mapillary_description_file_1.json

Extracting geotags from images: 100%|████| 418/418 [00:01<00:00, 290.19images/s]
Validating metadatas: 100%|███████████| 418/418 [00:00<00:00, 1226.21metadata/s]
2024-02-14 14:50:59,403 - INFO    - Checking upload status for 418 metadatas
2024-02-14 14:50:59,429 - INFO    - Check the description file for details: tmp/mapillary_description_file.json
2024-02-14 14:50:59,429 - INFO    -      418 image(s) read in total
2024-02-14 14:50:59,429 - INFO    -         418 image(s) are ready to be uploaded
Uploading IMAGE (1/2): 100%|███████████████| 1.11G/1.11G [01:24<00:00, 14.1MB/s]
Uploading IMAGE (2/2): 100%|█████████████████| 180M/180M [00:12<00:00, 15.0MB/s]
2024-02-14 14:52:43,150 - INFO    -        2  IMAGE sequences uploaded
2024-02-14 14:52:43,150 - INFO    -   1319.3M data in total
2024-02-14 14:52:43,151 - INFO    -   1319.3M data uploaded

Which creates a tmp/mapillary_description_file_1.json as follows (I’ve removed many of the track-points in the printed output below for brevity in this post);

[
  {
    "filename": "/Users/dgreenwood/Downloads/mapillary_tools/test_photos/tes-max-003g/GSAC5840.JPG",
    "md5sum": "655077ac078951d8bb52c38e0f41fb31",
    "filetype": "image",
    "MAPLatitude": 50.8966219,
    "MAPLongitude": -0.5591297,
    "MAPCaptureTime": "2021_08_28_10_06_42_000",
    "MAPAltitude": 77.42,
    "MAPCompassHeading": {
      "TrueHeading": 133.54,
      "MagneticHeading": 133.54
    },
    "MAPSequenceUUID": "0",
    "MAPDeviceMake": "GoPro",
    "MAPDeviceModel": "GoPro Max",
    "MAPOrientation": 1
  },
  {
    "filename": "/Users/dgreenwood/Downloads/mapillary_tools/test_photos/tes-max-003g/GSAC5841.JPG",
    "md5sum": "c1bb9c5f383cfc2e838f56b4e9e0a03f",
    "filetype": "image",
    "MAPLatitude": 50.8966029,
    "MAPLongitude": -0.559098,
    "MAPCaptureTime": "2021_08_28_10_06_44_000",
    "MAPAltitude": 77.231,
    "MAPCompassHeading": {
      "TrueHeading": 136.336,
      "MagneticHeading": 136.336
    },
    "MAPSequenceUUID": "0",
    "MAPDeviceMake": "GoPro",
    "MAPDeviceModel": "GoPro Max",
    "MAPOrientation": 1
  },
  {
    "filename": "/Users/dgreenwood/Downloads/mapillary_tools/test_photos/tes-max-003g/GSAC5842.JPG",
    "md5sum": "1039fb7c265bf0a73f89cab21765077d",
    "filetype": "image",
    "MAPLatitude": 50.896584,
    "MAPLongitude": -0.5590694,
    "MAPCaptureTime": "2021_08_28_10_06_46_000",
    "MAPAltitude": 76.932,
    "MAPCompassHeading": {
      "TrueHeading": 134.813,
      "MagneticHeading": 134.813
    },
    "MAPSequenceUUID": "0",
    "MAPDeviceMake": "GoPro",
    "MAPDeviceModel": "GoPro Max",
    "MAPOrientation": 1
  }
]

Most of it is fairly self-explanatory (again, schema here).

I do however want to point out the MAPSequenceUUID property inside each track-point object, as this is very useful for my use case.

From the schema;

"MAPSequenceUUID": {
    "type": "string",
    "description": "Arbitrary key for grouping images",
    "pattern": "[a-zA-Z0-9_-]+"
}

Based on previous experience uploading photos to Mapillary, I know a sequence can only contain 500 images. Thus, if 600 images for a single timelapse, it will probably result in at least two Mapillary sequence on the server.

Of course, if I upload 5 distinct captures in one upload that are very far apart in distance or time then the actual sequences generated might be much higher.

Back to the point, this is not defining the actual Sequence UUID Mapillary will generate on the server.

In my sequence above of 418 images, 361 are marked with "MAPSequenceUUID": "0" and 57 with "MAPSequenceUUID": "1". I’ll come back to this later when I look at the processed images, just keep it in mind for now.

Onto videos…

Uploading Videos using Mapillary tools

The upload process for videos is very similar to photos;

mapillary_tools process_and_upload <DIRECTORY OF VIDEOS / SINGLE VIDEO FILE> \
  --user_name trekviewhq \
  --desc_path tmp/mapillary_description_file.json

For example, to upload a single video:

mapillary_tools process_and_upload test_vids/ESAD001v205vid7-trek_view_full_nadir-2_with-nadir.mp4 \
  --user_name trekviewhq \
  --desc_path tmp/mapillary_description_file_2.json

Below is an example of that file (I’ve removed many MAPGPSTrack objects for brevity in this post);

[
  {
    "filename": "/Users/dgreenwood/Downloads/mapillary_tools/test_vids/ESAD001v205vid7-trek_view_full_nadir-2_with-nadir.mp4",
    "md5sum": "e67f95ffe0c10ed700506cadae70089a",
    "filetype": "gopro",
    "MAPGPSTrack": [
      [
        0,
        -13.942725,
        28.5107049,
        549.857,
        null
      ],
      [
        0,
        -13.9427281,
        28.5107056,
        550.076,
        null
      ],
      [
        200,
        -13.9427339,
        28.5107044,
        550.055,
        null
      ]
    ],
    "MAPDeviceMake": "GoPro",
    "MAPDeviceModel": "MAX"
  }
]

You’ll see the output is different to photos, but much of the data is similar.

The MAPGPSTrack list contains;

a time offset of the track-point relative to the start of the video
Longitude of the track point
Latitude of the track point
Altitude of the track point in meters
Camera angle of the track point, in degrees (no present in my track as it appears this is not extracted from GPMD by Mapillary tools)

For video uploads there is no specific mention of sequences in the description file.

How Uploads actually work in Mapillary tools

As I noted earlier, the Mapillary API docs doesn’t mention upload endpoints at all, so of course I was intrigued; what is Mapillary Tools using to send the photos and videos to the Mapillary servers?

As the code for Mapillary Tools is open, this wasn’t too hard to figure out. Here’s the part of the code responsible for uploads.

Whilst not documented the actual upload flow can be deduced from mapillary_tools, which performs an upload of videos or images like so;