Quick Reference

What we need

a) Have a dedicated import account.

b) Have JOSM Remote control enabled.

c) Have utilsplugin2, todo, conflation and buildings_tools JOSM plugins installed.

d) To avoid a timeout error while retrieving the data from the Tasking Manager, change the JOSM socket.timeout.read option to 90.

e) Have the following filters ready:

building
type: node AND untagged

Steps

1) Choose a task and load the OSM data and the ARU-TCRA Data into JOSM.

2) Load the Bing imagery from Imagery option in JOSM

3a) If the selected tile contains at least one existing building in the OSM layer, configure the Conflation plugin. Then generate matches between the existing OSM data and the ARU-TCRA Data data.

3b) If there is no existing building in OSM, simply merge the ARU-TCRA Data layer with the OSM one. Then, using filters, select all buildings in the data layer and add them to the Todo list, so we don't leave any building behind.

4) Then, deactivate all filters and go building after building through each of the 3 lists of the conflation plugin, or through the Todo list depending on the case:

If there is a match between an existing building and a new footprint, check if the new footprint is bringing any added value to the older one, otherwise remove it from the list. If the new footprint is best, click on Conflate (or combine) to replace the geometry of the former footprint by the most recent one. Solve any tagging conflict that could occur during the conflation.
Check if the building is visible on the imagery. Keep it as it is, improve it, correct it, complete it or delete it if you are sure it isn't there anymore. Remember that this is not a blind import, and it's you who will decide about what to do with each building. If in doubt, add a fixme=Please check existence tag.
Check its shape. It should normally be ok. If squared instead of round, correct the shape using Shift-O. If it needs to be squared, divided or disconnected from other objects, make any necessary adaptation.
Check any overlap, if you visualize any overlap between the new buildings and or with existing objects, correct them.

5) When finished, upload the data with the specific import OSM user account and the specific changeset tags:

where NUMBEROFPROJECT is the Tasking Manager project number.

You are ready to start a new task. If you want to leave it here, don't forget to switch back to your REGULAR OSM ACCOUNT now, to avoid using it for purposes other than importing data into OSM!

6) If appropriate, manually trace missing buildings. If you are doing this simultaneously with Step 4, consider using a separate layer for the buildings which you are manually tracing.

7) Then upload any manually traced buildings into OSM with your regular OSM account using the following changesets tags :

Before you actually start with any task

a) You need to create one specific import OSM user account for this import (if you don't have one yet). For example, if your regular OSM account is KAWAMALA, you can create one named KAWAMALA_imports

You will need a different email account than the one you used to create your regular user account. Hint: If you have a Google Mail account for your regular OSM account, you can open an import account putting dots in the email user. For example: antidykawamala@gmail.com is equivalent to antidy.kawamala@gmail.com or a.kawamala@gmail.com.

b) Open JOSM and check that you have enabled the Enable remote control option in the JOSM preferences (F12

c) Install the following JOSM plugins (if you don't have them installed yet):

c.1) Utilsplugin2

c.2) Todo (list)

c.3) Conflation

c.4) Buildings tool (recommanded)

d) To avoid a timeout error while retrieving the data from the Tasking Manager, change the JOSM socket.timeout.read option to 90. To do that:

Go to the JOSM preferences (F12),
Make sure the Expert Mode is enabled,
Go to Advanced Preferences,
Search for time to filter the options, and
Change the value of the socket.timeout.read to 90.

socket.time.out

e) We will have the following useful filters ready to work with the data:

building
type:node AND untagged

Workflow steps

Getting OSM data and ARU-TCRA building footprints

1) Open JOSM editor.

2) Select one unfinished project from the following list of HOT Tasking Manager projects for this import:


Type	Dates	Project	Created By	Status	Done By	Comments
TM Projects	14 Jul 2021	BUILDINGS IMPORT CENTRAL WARD - TANGA DISTRICT	Geoffrey Katerega/ HOTOSM	Available	OSM Community	For Testing purposes only

*This task is currently available solely for testing the import workflow, no data should be uploaded to OSM until discussions on mailing lists are not finalized.

3) Click on Contribute:

4) Select one task on the map and click on Map selected task:

... and you will download both the OSM data for that task area into one layer of the JOSM editor called OSM Data (or similar) and the custom imagery layer that is Bing Imagery:

5) We will now download the ARU-TCRA building footprints for that same area. Go to Extra Instructions and click in the CLICK HERE link to load the ARU-TCRA data into JOSM:

The ARU-TCRA building footprints will be loaded in a new data layer. As TZA_Tanga_buildings_imports XXX (WHERE XXX = Project Number)

Preparing the data

If there is at least one existing building in the OSM layer

Configuration of the conflation plugin

1) Select all the objects in the TZA_Tanga_buildings_imports layer using the following filters, so that only the edge of each building footprint is selected.

2) While your TZA_Tanga_buildings_imports layer is active, start configuring the Conflation plugin, clicking on Configure in the plugin panel.

3) Click on Freeze on the Reference line, the number of frozen objects appear.

4) Activate the OSM layer and select all the existing buildings in that layer using the same 2 filters as earlier.

5) In the conflation window, then Freeze the object layer (while the OSM layer is still the active one), the number of existing buildings selected will appear.

6) Check the other configuration parameters in the conflation window, the process can work with the default values. However, to avoid building "jumps" matching buildings which should not be matched, especially in areas where the existing buildings carry more information than building=yes, use Hausdorff distance calculation and reduce the value to 15. Make sure the box in front of Replace geometry is checked as in the image below.

Generating matches

7) Click on Generate matches to close the conflation configuration window. The conflation panel now shows 3 tabs:

Matches
Reference only
Subject only

Each building from either theTZA_Tanga_buildings_imports or the OSM layer is now listed under one of these 3 tabs.

If there is no existing building in the OSM layer

Merging TZA_Tanga_buildings_imports layer with the OSM one

To merge the TZA_Tanga_buildings_imports layer with the existing OSM data: right-click on the TZA_Tanga_buildings_imports layer and click on Merge.

Adding the buildings to the Todo list

So to make sure that we don't leave any building behind, we are putting all the buildings inside the Todo list plugin window. In order to do this, we will make sure the JOSM filters are activated as follows:

And we select all buildings (you can do that with the Ctrl+A shortcut):

In this case, we can see that we have 366 buildings selected, and not their nodes:

Now we can add the ways to the Todo list. Just go to the Todo list window, click on the Add button:

and you will see that the 366 buildings are added to the list:

Now we are ready to work with each building.

Working with the data

VERY IMPORTANT: Before continuing, disable all JOSM filters.

All the buildings that are going to be added or updated through this import will therefore be reviewed at one step or another of this workflow.

Depending on the existence or absence of buildings in OSM for your tile, you are now finding yourself with either 3 lists of buildings within the Conflation plugin panel or one single list in the Todo list panel. You are now going to review each of them individually.

In the case of the todo list plugin, the aspects to be considered are listed under Adding a new object (Conflation/Reference only & To do list) subsection.

If the case of the conflation plugin lists, we will have the ability to choose between a conflation and a removal of each object in the list, various potential cases are detailed under the 3 following subsections.

Updating an existing object (Conflation/Matches)

The matches are buildings identified in both OSM and TZA_Tanga_buildings_imports layers. If there are any matches listed, you can review and visualize them on the map. We can check if the matches identified by the conflation plugin are consistent with our observations on the imagery and if their geometry got improved. When a conflation is made based on matches, each of the existing objects will keep its history in the OSM database while receiving updated geometry and tags from the AI layer, in a similar way as if we would have been using the Replace geometry tool, except that this is done here for multiple buildings at a time.

Double-click on one of the listed matches to have the map zoom and centered on this match.
Compare the existing building footprint from the OSM layer with the TZA_Tanga_buildings_imports one while tentatively visualizing the imagery used.

The TZA_Tanga_buildings_imports footprint doesn't add any value

If the TZA_Tanga_buildings_imports version of the building doesn't bring any significant improvement, click on the Remove button to remove this footprint from the list. Alternatively, the object can be deleted from the TZA_Tanga_buildings_imports layer.

The TZA_Tanga_buildings_imports footprint improves the geometry

If the TZA_Tanga_buildings_imports version of the building has a better geometry, merge the 2 buildings by clicking on the Conflate button while the building is selected in the list. The former geometry of the object is therefore replaced by the TZA_Tanga_buildings_imports one in the OSM layer.

If a tag related conflict is detected in the matches list, it will pop up during the conflation and you will need to arbitrate between tags to solve the conflict. If it is not solved, the conflation of the involved buildings won't happen. In most cases, keeping the pre-existing tag will be preferred. However, if a building previously tagged as construction based on imagery now appears completed, its tag can for instance be switched to building=yes.

Adding a new object (Conflation/Reference only & To do list)

The buildings listed under Reference only or in the todo list are new buildings coming from TZA_Tanga_buildings_imports and for which there is no equivalent object in OSM. We are going to review each of them to check if each footprint can be identified on the imagery and if its shape has been correctly delineated. If it's effectively the case, the object will be added to OSM. If not, the footprint will be respectively deleted or manually corrected.

The TZA_Tanga_buildings_imports footprint isn't matching a building

An object has been incorrectly interpreted as a building by the ARU-TCRA methodology. It is a "false positive", they are relatively scarce. We delete it from our layer.

In the case of the todo list, the building has already been added to the OSM layer, you will therefore need to select it clicking on its edge, before deleting it clicking on Delete.

In the case of the Conflation plugin, since the object hasn't been added yet to the OSM layer, you can simply remove it from the "Reference only" list, by clicking on the Remove button, inside the Conflation panel. Alternatively, the object can be deleted from the TZA_Tanga_buildings_imports data layer.

The building shape is incorrect

Wrong shape interpretation

In some cases, we may see that the TZA_Tanga_buildings_imports building footprint was correctly detected but that its geometry had been wrongly interpreted as a square instead of a circle. We may correct it. One example of TZA_Tanga_buildings_imports building interpreted with a wrong shape.

1) To correct it you can delete extra nodes, keeping only 2 to 3 nodes located on the edge of the circular building.

2) Select the way and click Shift-O and the way will be given a new round shape while its tagging stays unchanged.

In other cases, a building partially covered by some vegetation can come up with a trapezoidal shape instead of a rectangle. In this case, it will be necessary to square it.

1) Move an existing node to the position where you expect it should be (where the building ends, under the vegetation).

2) Select the edge of the building, then click on Q and the building will be squared without affecting any of its other properties.

Shape including several buildings

Sometimes, especially in Tanga CBD areas, several close buildings can be interpreted as a single big building. They might need to be separated or retraced separately.

Building footprints are connected to each other

1) Separate the buildings connected to each other using the shortcut G after having selected on the building you wanted to disconnect from the others.

2) Then reposition their nodes in order to avoid any overlap and square any building which would require it (with Q shortcut).

The TZA_Tanga_buildings_imports footprint is correct

The TZA_Tanga_buildings_imports footprint effectively matches a building on the imagery and its shape had been correctly interpreted or corrected. This one can be added to OSM.

In the case of the todo list, we will simply mark the object as completed clicking on the Mark button an the plugin will automatically bring us to the next object in the list.

In the case of the Conflation plugin, we will click on the Conflate button to merge the verified object with the OSM layer and we can then select the next object in the list.

Checking an existing building (Conflation/Subject only)

All the existing objects in OSM for which the Conflation plugin hasn't identified a match in the TZA_Tanga_buildings_imports layer can be found in the Subject only list. Several reasons can explain this. We will go through each of the buildings listed under the Subject only tab individually. Double-clicking on a building in the list will get JOSM zooming on the selected one. After a visual check, decide if the building should be deleted or kept.

Imagery can't be used

If the imagery associated to the project is cloudy in the area, it is possible that the cloud cover hides buildings. In this case, we will ignore the objects involved without modifying them, keeping them as they are in OSM.

Building doesn't exist anymore

A building which previously existed in OSM doesn't appear anymore on the imagery. We check the object history (Ctrl-H) to make sure it was created from an older source than the project imagery. If this is not obvious, or that clouds or tree cover affect the visibility, don't delete the building but add a fixme tag on it for further check. If it is obvious that the building was created from an older source (for instance Bing imagery), that it doesn't exist anymore and that it doesn't include any additional valuable tags or reference points, delete it.

In the below example, the 2 buildings on the right side of the imagery were digitized based on an older satellite image. As a more recent source shows that these buildings have disappeared, they will both be deleted from the database.

The building exists

In the case the TZA_Tanga_buildings_imports layer does not contain any match for an object existing in OSM and visible on the imagery layer associated to the project, maintain the original object in OSM.

Correcting visible overlaps

New buildings may have been built on former roads or paths. Using the imagery correct the visible objects to avoid having overlaps. JOSM validation tool can help you detect such type of situation, classifying them as warnings.

Optional: Tracing additional missing buildings

Some buildings might have been missed by the digitization process. In most cases, they will be scarce and can be manually traced using the Buildings plugin and the imagery associated to the project.

In case of cloud cover on the imagery associated to the project, in an area which hasn't been mapped yet, it is possible to use another, even older, satellite imagery to digitize the missing buildings.

ATTENTION, in all cases, this digitization should take place in an editing session separate from the import one.

Either in a successive manner: after the upload of your import results to OSM, you start to only digitize buildings.
Or simultaneously with the import process: digitizing these buildings in a separate layer from the one used to do the import.

This distinction is important because this type of edit doesn't require using an import account, it will need to be uploaded using your regular OSM account (instead of the one dedicated to imports), because the involved objects are not coming from an external source. The characteristics of the upload are detailed at the end of this page.

Upload new data to OSM

Upload import data

Once we are done with the review of all the building footprints contained in our list(s), we will upload the data to OSM.

1) As we will import these data with the dedicated OSM import account, we need to make sure we changed the user in the preferences (F12):

2) We click on the Upload button. We correct any errors/warnings we may have, and then we only have to set the following changeset tags:

Tag automatically added by the Tasking Manager:

comment=ARU-TCRA #hotosm-project-NUMBEROFPROJECT #MissingMaps #HOTOSM #HOTanzania
where NUMBEROFPROJECT is the Tasking Manager project number.

Tags to be updated or added manually:

Example:

And now we (finally!) upload the data.

You may now proceed with another task. In case you don't want to continue for now, DON'T FORGET TO GO TO JOSM PREFERENCES (F12) AND SWITCH BACK TO YOUR REGULAR OSM USER ACCOUNT. If it's not immediately disconnected, it is quite common to leave the import user account active and go mapping other things that you should map with your regular OSM user account.

Optional: Upload additional objects

If you have manually traced a significant number of additional buildings, you will have to upload them to OSM with your regular OSM account.

Make sure your OSM account dedicated to imports has been disconnected. Then upload your manually traced buildings to OSM with your regular OSM account using the following changeset tags:

Tanga Building Footprints Import/Workflow