New Orleans, Louisiana/Building Outlines Import

From OpenStreetMap Wiki
Jump to navigation Jump to search

Work In Progress

Data import in progress! Please check the import progress at our task tracker: http://tasks.openstreetmap.us/job/41

After discussion on the OSM mailing lists (imports-us@ and imports@), we began importing in late December, 2014.

This process will be in accordance with OSM's Import Guidelines.

How to get involved

Everything that mappers need to know to get involved should be found in our Contributor Guide.

Everything else found below on this page is background/technical information on the import which was prepared while this project was under consideration. It is not necessary to read on unless you want to learn more about how this project began.

About the data

The City of New Orleans provides us with a public domain shapefile with high-quality building outlines (153492 buildings, based on imagery from January 2013) and a shapefile of addresses (198568 points, commercial and residential).

License

According to data.nola.gov, the data is licensed CC0. (Public domain)

cc0 license screenshot

Data files

Tagging

  • building = yes
  • nolagis:id = OBJECTID (useful for comparing against future upstream data updates) may not be useful after all

If building only contains one address, the following will be added to the area. In the case with multiple addresses in different locations within the same building footprint, these tags will go on separate nodes:

  • addr:housenumber = HOUSE_NUMB
  • addr:street = DIR (expanded: N->North) + STREET + TYPE (expanded: ST->Street)

Reconciliation with existing buildings in OSM

There are not a lot of buildings already in OSM for New Orleans. As of October 2014 there are 512. Most of these fall into the following categories:

  • 10 blocks of the French Quarter were mapped by hand by user wegavision in March of 2013. This accounts for 178 of the 512 buildings (35% of the total). This data is superior to the data.nola.gov data and should be kept. Merging in additional address data can be done by hand in JOSM or Potlatch 2.
  • Buildings at local colleges and universities (Tulane, Loyola, UNO, Dillard, and Delgado) are mostly included already. (About 115 buildings, or about 22%) These should be kept. Additional buildings not already mapped can be manually pulled out of the nola.gov data. Most of these buildings were added by user Matt Toups, who is helping with this import and can help merge his edits into the imported data.
  • Several blocks of the Oak Street commercial area were mapped by user ceseifert (who is parcipating in the import) in December 2013. (This accounts for about 40 buildings) These should be kept, they are tagged nicely with shop and amenity tags. Addresses can be added to these buildings manually.
  • Buildings in City Park (32 buildings) should be kept.
  • Several buildings in New Orleans East were added by Minh Nguyen. (About 16 buildlings) This user is assisting with the import and can help merge his edits into the imported data.

Excluding these groups of buildings from the import will not be difficult to do manually.

This leaves only 131 buildings scattered around the city to review manually. This will be built into the review process. The majority of these were originally added by user Matt Toups who is also importing.

Conflation with existing addresses in OSM

Similar to buildings, there are not a great deal of addresses to conflate. There are 119 ways and 990 nodes with addresses today. The ways are all buildings which are addressed above.

The nodes are POIs that should be kept. We will remove duplicate nodes from the imported data when the existing POI contains address data. (Or in some cases, merge the POI tags into the building outline)

Of those 119 ways and 990 nodes, 45% were created by wegavision (all in the French Quarter, which we plan to keep). 21% were created by Matt Toups. 5% were created by ELadner. Outside of a few areas, there are very few remaining addresses to conflate.

Participants

  • Matt Toups: longtime local mapper and lifelong resident. Maintaining scripts for import and coordinating effort.
  • Carl Seifert: local mapper, will review resulting data
  • Eric Jiang: reviewing import code
  • Minh Nguyễn [1]: expert mapper, will help review results for New Orleans East (an area he knows well)
  • Editor B: local mapper, can review buildings at Xavier University
  • NOLA_Chris: local mapper
  • Eric Ladner: expert mapper based nearby in coastal Mississippi (with experience importing MGC data)
  • Katie Pickett: local mapper also active in humanitarian mapping in Sierra Leone

Process

Note: this is still in development at this time.

User accounts

In accordance with Import Guidelines, each uploader will use a separate user account specifically for this import. The user name will follow the pattern: <regularusername>_nolaimport

Task Manager

City divided into precincts in OSM task manager

We will track who is doing what here: http://tasks.openstreetmap.us/job/41 (using the OSM_Tasking_Manager)

Workflow

The city is divided up into 17 wards, each of which is divided into some number of precincts. Each region (ward/precinct) will be reviewed, uploaded, and verified.

Address nodes may be slightly offset from building footprints, but should be ordered properly. Small adjustments like these can be made by local importers during our workflow.
  • User checks out a region of the city (on the task manager) and downloads .osm file to local computer.
  • General sanity check: does the data look reasonable compared to aerial photos? (Mapbox at zoom levels 12–17 appears to be the most up-to-date.)
  • For the region, compare with the current data in OSM:
    • Are there existing building outlines? If so, we must decide if we want to replace the existing one with the import or keep the existing building. Usually comparing geometries shows that one or the other is of superior quality, when compared with aerial photography.
      • If we keep the existing building, we can transfer the addr:housenumber and addr:street tags from the import building (if they exist) and then remove the duplicate entry from the import data. If there are addresses in the import data in point form, they should stay in the data for upload as long as they're adding useful information to the map.
      • If we replace the existing building, check it for any useful tags (either addressing, name, or other indication of function) which can be transferred to the new building outline.
    • Are there POIs that should be merged with our building outlines?
      • If a small building contains, for example, a restaurant and nothing else, then it could be appropriate to transfer the restaurant POI tags to the building.
      • However a larger building with multiple tenants should keep POIs as separate nodes which indicate which part of the building they occupy.
  • Is the addressing correct?
    • Make sure an address is attached to the "main" building and not a shed or outbuilding.
    • If there are two (or more) address points within a building, are they placed in a reasonable location within the building footprint? (see 2910/2908 St Ann Street in the example to the right)
  • Run JOSM validator and review any errors/warnings it presents. (Typically these only come up in very dense areas like the French Quarter)
    • May find these warnings: Crossing buildings; Self-intersecting ways; Building inside building
  • If you have any doubts about the data's accuracy or suspect a technical glitch, stop and contact the rest of the import team for assistance or a second opinion.
  • Once the region has been reviewed, upload data to OSM using dedicated import user account.
  • Afterwards: peer review.

Resources

We're starting to collect info from previous imports that may be helpful to our volunteers:

The folks who ran the NYC building import used a very similar process, and have a nice step-by-step video guide for getting started:

QA tools:

Source Code

Managed in git: https://github.com/mtoupsUNO/nola-buildings

The results can be downloaded as a .zip file from here: https://github.com/mtoupsUNO/nola-buildings/releases/tag/v1.0

Individual .osm files are available here: https://github.com/mtoupsUNO/nola-buildings/releases/tag/v1.1

This is based on work from the DC and NYC building imports, thanks to the folks who made that possible.

How it works

Makefile will:

  • Download the following:
    • Building Outlines shapefile
    • Addresses shapefile
    • Voting Precincts shapefile (used for chunking)
  • extract each zip file into its own directory
  • convert to EPSG:4236 using ogr2ogr
  • make target directories for the processed data
  • break the building outlines and addresses into chunks using chunks.py (thanks to dcbuildings / nycbuildings folks for this)
  • run convert.py which yields osm XML for each voting precinct (more detail below)

convert.py is based on the work of dcbuildings but has been modified to work properly on the New Orleans shapefiles. It does the following:

  • Exclude all addresses with STATUS=RETIRED
  • For all addresses that fall within a building outline, add that address to a list of addresses for that building.
  • For each building:
    • If it has exactly 1 address, add address tags to the building area.
    • If it has more than 1 address:
      • Multiple addresses in different positions: add each address as a node (which gives us the correct behavior for duplexes, the most common case)
      • Multiple addresses in the same location: only one of these will be "Primary" according to the GIS data. keep that one only. (This is a pathological case but unfortunately it does happen in this data set.)
  • add tags for housenumber
  • add tags for street
    • Add direction to street (from "DIR" field in shapefile)
      • Expand N, E, S, and W to full "North" etc
    • Add street type (from "TYPE" field)
      • Expand "St" to "Street" etc

Quick start guide

This is how to use the scripts to generate .osm files: 

git clone https://github.com/mtoupsUNO/nola-buildings
make

Resulting OSM XML files will be written into the osm/ directory. You can view these in JOSM, Merkaartor, or Potlatch 2.

/Contributor Guide

Feedback

Please send feedback on the import scripts to Matt Toups.

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=New_Orleans,_Louisiana/Building_Outlines_Import&oldid=1869291"