Google Summer of Code/2015/Mapillary plugin for JOSM

From OpenStreetMap Wiki
Jump to navigation Jump to search

Mapillary plugin for JOSM

The developer/student will get access to documentation of a more complete API, which is currently being developed on the Mapillary side. The development blog can be found here

Features

  • Show icons of Mapillary pictures on the map. Including compass direction arrow.
  • Show tracks/picture sequences on the map.
  • Show one large picture. With additional information like coordinates, direction, extracted data (traffic signs, etc.)
  • Show traffic signs on the map. Link with picture they were extracted from, i.e. update large picture. future API should allow user to send corrections back
  • Offline cache of downloaded data.
  • Upload new pictures to Mapillary?
    • This could be for geo-located pictures, where the JOSM user processes them first, improves their location tags and orientation, then pushes them to Mapillary
    • It could also be for other pictures, for which the user knows where they were taken
    • It could be possible in this case to blur parts of the pictures, either as a request to do it on the Mapillary site or directly on the picture.

Background thread

  • When changing the viewport, make a request to Mapillary API for all pictures included in it
  • Retrieve GPX traces for them
  • Retrieve thumbnails for them and store in cache
  • Purge cache when it reaches treshold

Mapillary layer

Compare to Geotagged images layer in JOSM core

  • Layer showing small thumbnails or a drop symbol for all pictures available on the Mapillary server in current viewport
  • Draw lines between the pictures to show how they are related as sequences
  • When an orientation is in the EXIF, show an arrow pointing in the right direction.
    • When a picture is selected, this becomes a line in a different color than the one for the GPX connecting the images. This line has an extra virtual node which can be dragged to modify in what direction the picture was taken.
    • The position of the image can also be improved.

Mapillary Window Pane

There is also already such functionality in JOSM core

  • By default this is an independent pane, so it can be viewed easily on a second monitor
  • This shows the currently selected picture
  • On the Mapillary layer, miniatures of the thumbnails can be shown. In this window pane the size is bigger.
    • Mapillary stores pictures in 4 sizes/formats
    • As long as the user keeps the size of this pane small, thumbnails will be enough while flicking through them rapidly. When looking for more than a second, the full resolution picture can be retrieved and shown.
  • Add buttons for forward, backward (in the sequence), left, right (jump to other sequences).
  • A button to open a browser window with the image in edit mode on the Mapillary site
  • A button to add a mapillary tag to the currently selected object
  • A button to go into add/remove blurring mode
  • Possibility to validate recognition of traffic signs and possibility to add the tags which are a consequence of these signs to the ways they relate to and to add the proper tags on the signs themselves. Probably by having the RoadSigns plugin as a dependency.
  • Button to report bad pictures?

List of changes

It's possible to improve the positioning and compass heading of the pictures. When such changes are made, it's best to send them all in what Mapillary refers to as a changeset. In this list we keep track of these changes. So the user of the plugin can send them all at once.

  • It would be good if it were possible to send pictures taken with other cameras to Mapillary, preferably after improving or adding proper geotags to them. Upload scripts written in Python

Mapillary Settings

  • Ask for the user's credentials at Mapillary and optionally store them locally (oAuth 2.0)
  • Treshold size of cache
  • Automatic update of Mapillary layer if user moves the map or zooms, or on request? Looks like there is quite some data transmitted with every API request. I think that's crucial

References

Technical details

Classes for individual data types

Images Sequences Objects, with sub-classes for specific objects (I would suggest to call this objects or something more generic, we are going to detect more stuff) Traffic signs

Use case scenarios

User edits some bbox

  • The user wants to see all the images available at Mapillary, so thumbnails for them get downloaded to a cache folder. an icon or mini-thumbnail shows up at its position on the mapillary layer.
  • The image the user is working on, gets downloaded at full resolution for viewing in a separate Window pane.
  • If there are many pictures for an area, the user may want to filter them on some criteria (date range, only picture with traffic signs in them, ...)
  • If the user notices the position is not correct, it should be possible to update it
  • If the user notices the angle is not correct, it should be possible to change it by dragging a virtual 'helper' node
  • User can confirm or correct detected traffic signs, confirmation is only possible when completely zoomed out (viewing the whole picture)
  • User can zoom in and out on the picture using the mouse wheel, pan with right mouse button

User has taken pictures with the Mapillary App, or otherwise, and wants to review them before uploading

  • If the user notices the position is not correct, it should be possible to drag it to the correct position
  • If the user notices the angle is not correct, it should be possible to change it by dragging a virtual 'helper' node, which indicates where the camera was pointing to
  • User can delete pictures which aren't suitable for uploading
  • User can decide to move pictures to a 'private' folder, so they won't get uploaded, but still remain available locally
  • User can tag pictures with keywords to indicate what is visible in the picture (as far as I know there is no support for this on the Mapillary side (yet), but it's easy to add such tags to EXIF)

User has geotagged pictures and other pictures which were taken during the same session

  • the other pictures get geotagged automatically based on timestamps and geotags in the main sequence
  • user decides whether the angles are the same or always with the same offset

User wants to add detected traffic signs to OSM

  • Lower priority
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Google_Summer_of_Code/2015/Mapillary_plugin_for_JOSM&oldid=1167875"