DakotaCounty/Contributor Guide

Most of the content on this page is copied from the very excellent Louisville,_Kentucky/Building_Outlines_Import/Contributor_Guide - we are adopting the basic outline of their import process, but there are a few minor differences. If you notice any errors, please report them or even better, just to jump in and fix them.

So you want to help out with the Dakota County building import? Great! Here's how.

Overview

Thank you for your interest in helping with the Dakota County building import! Here is a guide for how to contribute.

Install and Configure What You Need

The How to start section contains steps that only need to be completed one time at the beginning. This data import process requires a bit more software setup than normal OSM editing, but we'll try to make it fast and painless.

See our Software Setup Guide.

1) Select a task and initialize JOSM

Now that those preparations are complete, we're ready to start working with the building and address data.

The OSM Tasking Manager is our central tool for coordinating this buildings import project.

Take a task

City divided into census block groups in OSM task manager

One task selected and active.

In order to make sure two people don't work on the same block group at the same time, when you "take" a task the task tracker will mark it as "currently worked on" and not give it to anybody else. Here's how:

Start with the task manager here: http://tasks.openstreetmap.us/project/196 (Note: you should log in with your username_dcmnbuildings account)
Now you should be able to browse the map of the county and see what areas are complete and what areas need work. You "take" a task by clicking on a block group on the map.
- Grey areas are still available to be imported. Click on one of these grey areas to see the task for that area.
- Green areas have been uploaded *and* verified (checked by a second set of eyes)
- Yellow areas have been uploaded but not yet verified
Start with areas that you are familiar with - neighborhoods where you have lived or worked, for example
- If you don't care what block group you will work on, you can click on "Contribute" and then "Take a task at random".
Click the green "Start Mapping" button to lock the task. Only one person can lock a task at a time, so this signals to other mappers that you are working on this task. The lock on the task will automatically expire in 2 hours.

Open in JOSM

3 layers in JOSM: Austin data, OSM data, Bing Imagery

Before proceeding farther, make sure JOSM is running (with remote control enabled, as described above)

Now you should see a dropdown to start mapping. Select Edit with JOSM. This will load the pre-existing data from OSM into JOSM in a new layer.
Turn on satellite imagery layer to give you a better frame of reference. Select Imagery -> Bing from the menu bar. The Mapbox Satellite imagery is also good. If imagery is badly misaligned you can adjust it by right clicking the layer and choosing New Offset - this won't affect the building data, only your visuals in JOSM. The best imagery is from Dakota County - https://gis2.co.dakota.mn.us/arcgis/services/AerialPhotography/2016AirPhotoLeafOff6Inch/ImageServer/WMSServer?request=GetCapabilities&service=WMS

2) Work on Data in JOSM

A side note: JOSM can be a bit daunting to use at first. This is OSM's advanced offline editor, which is quite powerful and allows us to integrate nicely with the task manager and import data. So it is the best fit for this import process.

This step is the bulk of our work. Now we will use JOSM to review and make any changes or merges to the data before uploading to OSM's database. Until the upload step, no changes are made to the public map, so you can still bail out if you have problems.

General JOSM editing info

If you haven't used JOSM before, there is a nice guide on this wiki that may be helpful. For our purposes JOSM/Basic_editing will contain most of what's needed to get up to speed. And there's another guide here from the folks at learnosm.org. (Of course you can also just learn by trying, we're here to help)

Tips for Editing

Use a mouse, not a trackpad if you can.
Control + Click will drag the map (or right click).
Two finger slide up or down will zoom the map (or mouse wheel).
Command + Shift + V will paste all copied tags into selected item.
Go to preferences and remap the Delete key to something else (like Eject) if you are on a keyboard without a number pad.
Add commonly used menu items to the top quick bar.
Shift Click selects multiple items at once, Control Click deselects.
Shift Click to select multiple tags at once, right click a tag for more options (copy selected, copy all, etc).

You can watch/listen to a demo of this step from the NYC buildings folks: click here for youtube video.

Importing and Editing Data

The Dakota County buildings and addresses have been combined into one file for import, found in the "import links:" comment of the task (under the Contribute tab). To open a file in JOSM, just click on its link.

areaName.osm

These are the building footprints that are ready to directly import into JOSM. They contain building and address information that you may have to trust or may be able to visually verify by looking at the imagery. These building polygons were digitized in a stereo environment that reduces building lean. The reference imagery you may use to proof this data before load may contain building lean and will appear as if there is a shift in the data. The data is positionally correct.

Use the Edit -> Search tool to select all the buildings .

The Configure conflation settings dialog with address nodes and buildings locked in.

The Conflation Plugin window after running Generate Matches.

The conflict dialog where you can merge address nodes to buildings or fix any conflicts.

Areas where buildings exist may need to be conflated to their respective OSM buildings using the JOSM Conflation plugin (JOSM/Plugins/Conflation). In some cases, you may just replace them with the new buildings as they were created in a stereo digitizing environment.

For our purposes we'll be using address points as reference layer in the conflation plugin and the existing buildings will be the subject layer.

Steps to do the conflation:

information forthcoming

Copy imported data to OSM layer and validate. You can watch/listen to a demo of this step from the NYC buildings folks: click here for youtube video.

3) Run Validator JOSM tool

JOSM Validation errors. All errors should be fixed, and most warnings.

JOSM has something called a Validator (click for details) that attempts to automatically find common errors and alert us to fixes before we finish.

Run JOSM validator and review any errors/warnings it presents. Typically these only come up in very dense areas with lots of buildings. Most will be about duplicate nodes or ways, which are minor and should be automatically fixed.
- May find these warnings: Crossing buildings; Self-intersecting ways; Building inside building

JOSM Fix Validation errors.

Now fix the Errors automatically with JOSM. Click the Errors folder in the bottom right, and click the Fix button (see image to the right). Then re-validate and there should be no more errors! If there are fix them manually.

If there are still Warnings related to buildings or ways, please fix those manually (right click the warning and click Zoom to see it):

If a way intersects a building, you can modify the road so it goes around the new building. Reference the satellite imagery.
If the interior of a building should be cut out (eg, for a courtyard), the problem is usually that there are 2 interior shapes. Left click and drag to select the entire interior shape, the double click the 'building' selection.
If you have serious doubts about the data's accuracy or suspect a technical glitch, contact the rest of the import team for assistance or a second opinion.

Some errors and warnings are from existing data problems with boundaries, roads, tagging etc. If they were already there and not part of buildings or addresses or roads overlapping buildings, you can ignore them.

4) Upload Data to OSM

You can watch/listen to a demo of this step from the NYC buildings folks: click here for youtube video.

JOSM Upload dialog

Note: If you changed the existing OSM user data (ie, removed buildings or addresses), upload the changes for this layer first! Then upload the data layer from task manager.

Click the upload button which looks like this: on the top button toolbar. This will only upload the active layer, so make sure it's selected!

JOSM Upload progress bar

If you get a Conflict notice, click the far right button and try again.

You'll be faced with a dialog listing the modified objects (likely a lot, in the case of our import). It will also ask you for a brief comment and source. We're going to follow a consistent format for these two fields throughout the import, which is important as it allows us to track our work within this rather large chunk of data. It also clearly documents our work so that others in the OSM community to understand how this data got here.

JOSM Upload Changeset Comment and Tags

Make sure you include the following info in changesets to make them more easy to track.

Comment

If anything special or out of the ordinary happened, include that in the changeset comment. At the very least, each changset should include this line:

This changeset is part of an import of buildings and addresses for the Dakota County. Full details on the import can be found at https://wiki.openstreetmap.org/wiki/Minnesota/DakotaCounty/Buildings_Import

import

Set the import=yes tag on every changeset.

Source

Source should always be set to:

Dakota County

Mark complete in task manager

Back in your web browser, there is a third box with a field for comment. This is a good place to put qualitative thoughts on how the data seemed to you, which parts you are more or less confident of, or any other notes you think the next person looking over this section should see first.

Then click Mark task as done. Now the area will show as done on the task tracker (colored in yellow).

Congratulations!

Validation

After areas are marked as done on the task manager, a second mapper will check it over to see if everything looks right. This is our final QA/QC check for the import, so be sure to only mark a task as valid if you are completely sure that it is done and error-free. Don't validate your own tasks. This will allow some cross-over so at least two people will need to have signed off on a task. Things to look for before marking a task as valid:

Did everything import?

Kind of obvious, but buildings and addresses for the task should have made it into OSM.

Validation errors and warnings to watch out for

crossing ways OR crossing buildings

meaning: This happens whenever two ways intersect. With imported buildings, this can happen when buildings were imported on top of pre-existing ways or if there was funky import data for whatever reason. If one of the ways involved is not a building we imported, then don't worry about it (you can definitely fix if you want, but it's not a blocker).

how to fix:

Check imagery to see what's going on.
Sometimes the features are just off and can be fixed by moving them as appropriate.
If a way actually passes underneath a building then set layer=* or covered=* tags as needed.

examples:

A toll booth building over a highway. The correct way to map this is to split the highway into segments with nodes joined to the building way where it passes under the building. The segment under the building should have the covered=yes tag set. The building should also have the building=roof tag set.
A creek crossing a building. This often indicates that the creek runs through a culvert underground here. If that is the case, then the creek should be split into segments where it passes underground, and both layer=-1 and tunnel=culvert tags should be set on that segment.

building node within a building way

meaning: There are many variants on this, but happens if a node already existed for a building before the building was imported and now there is a building way and node in the same area.

how to fix: Either merge the tags from the building way and node into the building way, or just leave the node and remove the building tag. If the tag is more specific than building=yes, then set that building value on the building way.

duplicate nodes

meaning: This might indicates that addresses nodes were have been imported twice by accident.

how to fix: You can just run auto-fix on this.

duplicate ways

meaning: This might indicate that buildings were imported twice by accident. If this happened from a duplicate import, there will usually be a ton of them (as many as there are buildings, so at least a few hundred). If the duplicate ways are not buildings, then you can ignore this.

how to fix: Use the JOSM reverter plugin roll back all but one of the duplicate changesets. See the reverter plugin page at JOSM/Plugins/Reverter for instructions on how to do that.

abbreviated street name

meaning: This is a pre-existing error in OSM but we should fix it now since it affects the address we are importing and is important for routing. This occurs when an OSM street name is abbreviated (for example: Lighthouse Landing Dr should be Lighthouse Landing Drive)