Denver Planimetrics Import

About

The Denver Regional Council of Governments (DRCOG), in partnership with local governments and public entities, has contracted for the collection of detailed infrastructure data for the Denver Region and wishes to make it available to the OSM community.

DRCOG is following the Import Guidelines and is collaborating with OSM-Colorado to obtain community buy-in, support, and volunteers to perform import(s). This effort began in May 2016 and is currently in planning.

This project was funded, in part, by the FHWA and/or FTA, DRCOG, RTD, Denver Water, Arvada, Aurora, Bennett, Centennial, Commerce City, Denver, Englewood, Frederick, Glendale, Golden, Greenwood Village, Lafayette, Lone Tree, Lakewood, Louisville, Thornton, Wheat Ridge, and Northglenn. DRCOG owns the rights to the data and has the authority to put it in the public domain.

Note: There are several datasets available, which may necessitate additional wiki pages with more detailed information - please stay tuned.

If you have questions about this project, please contact DRCOG at geospatial@drcog.org.

Import Plan Outline

Phase 1 - DRCOG would like to complete a pilot study in winter 2018 using the building roofprints data in a small area to determine import workflow (utilizing OSM Tasking Manager).

Phase 2 - If Phase 1 is successful, DRCOG will make the remainder of the building roofprint data available for volunteers to import at their own discretion.

Phase 3 - DRCOG will then ask the community if they want to proceed with other datasets, and if so, DRCOG will work with the community to rank them according to priority. DRCOG suggests working on centerline sidewalks next but has no preference after that on sequence.

Goals

To determine what, if any, planimetric datasets from DRCOG can and should be imported into OSM.

To develop a workflow for volunteers to use DRCOG’s planimetric data to update OSM.

Schedule

• Announcement to local community January 2017.
• Bi-weekly planning calls with DRCOG and OSM-Colorado January - October 2017.
• Announcement to global community October 2017.
• Final comments and initial workflow updates Fall/Winter 2017-2018.
• Talk at State of the Map - Boulder October 2017.
• Pilot project, reevaluate workflow, make improvements Fall 2017 - Summer 2018.
• Perform import via mapathons and individuals, Summer 2018 - Finish (TBD).

Import Data

Background

The planimetric data proposed for import was digitized from spring 2014 imagery (4-band RGBIR color orthoimagery; 6in and 3in resolution) obtained through the Denver Regional Aerial Photography Project.

Imagery was collected with the Leica ADS40 and ADS80 digital sensors and processed with Leica XPro software. Imagery is projected in State Plane Coordinate System, Colorado central zone using the Lambert Conformal Conic map projection parameters. Horizontal and vertical datums are NAD83(11) and NAVD88(GEOID12A) respectively.

DRCOG’s stated accuracy requirement for the datasets was National Map Accuracy Standard (NMAS) for 1"=100' scale mapping. NMAS accuracy for map scales larger than 1:20000 horizontal tolerance is 1/30" for 90% of well defined points or, 3.0' horizontal accuracy. RMSE of the control is X = 0.108' and Y = 0.082'.The Horizontal_Positional_Accuracy_Value was tested in accordance with Geospatial Positioning Accuracy Standards Part 3: National Standard for Spatial Data Accuracy (NSSDA), FGDC-STD-007.3-1998. NSSDA horizontal accuracy is 1.7308*RMSE_r or 1.7308*0.095 = 0.16'. NSSDA Horizontal accuracy of the aerotriangulated aerial imagery meets or exceeds the NMAS requirement of 1/30" for 1"100' scale mapping.

Note that the extents for the datasets are different.

Data source site: DRCOG Data Site 2014 building roofprints(Data we are currently looking to import)

Data license: License Unspecified at time of import planning, Usage Permitted by Owner (see letter)

ODbL Compliance verified: OSM Usage Permission Letter

Dataset details

• Building Roofprints (poly) - Stereo-compiled (3D). Only permanent structures are captured. Collected buildings have an area >= 48 sq. ft. and are over 5 ft. in height. Arms and indentations are 2.5 ft or greater. Canopies, carport covers, and sheds are included. For multi-level commercial/industrial buildings, each level is a unique record IF the building is >= 1000 sq. ft. and the multi-level roof levels are >= 400 sq. ft. in area and over 5.0 ft in height. Each polygon is a separate record. Polygons that are part of the same building have a matching building ID. For pitched roofs, the highest point is collected to get the building height. The lowest point is the common lowest point if multi-level.
• Parking Lots (poly) - Digitized (2D). Outlines of paved, impervious surface parking lots with an area >=400 sqft. Includes the entrance into the lot. Connects to road features. Pervious features within parking lots (islands) that are >= 50 sqft are captured. This layer does not include on-street parking or parking garages (the latter are captured by the building roofprints layer). When sidewalk polygons meet parking aprons, the sidewalk breaks and parking polygon continues – unless you can visibly see the sidewalk crossing the parking lot. Lots within apartment complexes are included in this layer. Dumpster areas in the parking lot are included.
• Edge of Pavement (poly) - Digitized (2D). Outline of paved and unpaved streets/roads with width >= 9ft. Unpaved trail is not included in this layer. Edge is the flowline (lowest point in curb). A road/street is defined as having consistent material from edge to edge. Includes paved and unpaved features intended for street-legal vehicle use (corridors) and medians. Medians are delineated by a curb or raised surface, not a paint line. Paved road polygons take precedence over sidewalk polygons, but unpaved roads do not.
• Edge of Pavement (line) - Stereo-compiled (3D). Edge of paved and unpaved streets/roads with width >= 9ft. Unpaved trail is not included in this layer. Includes paved and unpaved features intended for street-legal vehicle use (corridors). A road/street is defined as having consistent material from edge to edge. Breaks occur at intersections and where surface type or curb type changes. Edge is the flowline (lowest point in curb). Paved roads take precedence over sidewalks, but unpaved roads do not. For overpass/underpass situations, features are continuous on overpasses, and break features underneath by the bridge deck. For drive-through or building overhangs (like entrance of hotels), the structures supersede and break parking polygons.
• Sidewalks (poly) - Digitized (2D). Polygon sidewalks and trails do not cross road intersections. Connectivity is not maintained as in the centerline sidewalk dataset. For sidewalk polygons meeting driveway apron, sidewalk always stays continuous and breaks driveway polygons. For sidewalk polygons meeting parking apron, sidewalk always breaks and parking polygon continuous- unless you can visibly see the sidewalk crossing the parking lot. Sidewalks are not delineated in areas where they don't provide connectivity (i.e. in a parking lot median). This layer does not include private sidewalks/trails. This layer includes all public sidewalks (e.g. adjacent to a public building, schools, or roads, adjacent to commercial buildings, maintained by a public entity) and trails (e.g. in public parks, maintained by a public entity). Polygon sidewalks extend to include areas with the tree plantings, but only exclude the pervious part if it is over 50 square feet (this threshold is also used in the parking layer).
• Sidewalks (line) - Paved sidewalks and paved trails with width > 5ft as centerlines. Sidewalks are paved walks for pedestrians, "Point A to Point B" use. Trails in this layer are public paved walkways intended for recreational purposes. If the sidewalk widens and the centerline shifts, a "jig" is created to keep the centerline connected. The sidewalk does not stop at intersections or at parking aprons or at driveways, BUT these areas are separate segments within the line feature and attributed differently. When connecting "sidewalk"segments with an "other crossing" segment, a connection is only made when there is visible access between the two sidewalk segments. Access means that a wheelchair can traverse it (so stairs do not constitute a connection). For sidewalk centerlines meeting parking apron, the portion of sidewalk centerline within the apron always breaks into a different line segment, and has a type of "other crossing" unless the sidewalk crossing the parking lot is readily visually apparent, in which case the sidewalk centerline simply continues unbroken. For sidewalk centerlines meeting driveway apron, the sidewalk continues as with the polygon sidewalks. This layer includes all public sidewalks (e.g. adjacent to a public building, schools, or roads, adjacent to commercial buildings, maintained by a public entity) and trails (e.g. in public parks, maintained by a public entity). The sidewalks layer does not include private sidewalks (i.e. serving individual residences, contained within campuses, mails or commercial complexes) EXCEPT where the sidewalk maintains connectivity with the public sidewalk network. In these instances, if the private sidewalk is too complex (i.e. multiple, indirect paths, or with many branches), a best-fit line may be drawn to maintain connectivity with the public sidewalks.
• Trails (line) - Digitized (2D). Centerlines for unpaved trails with a width greater than 5ft and less than 9ft. The primary use of "trails" is for recreation purposes. If the trail widens and the centerline shifts, a "jig" is created to keep the centerline connected (just as with centerline sidewalks). A goal of this feature dataset is connectivity. Connect to the sidewalk centerline layer when appropriate. This layer will only include public, but not private trails. Break all trail centerlines at intersections.
• Driveways (poly) - Digitized (2D). Paved polygon driveways. Includes public and private.
• Ramps (point) - Digitized (2D). The location of sidewalk ramps. Ramps do not have to connect to other layers (e.g. sidewalk centerlines).

OSM Data Files

Converted OSM Data Files can be found here.

Import Type

At present, this is a one-time import opportunity for all datasets. DRCOG would like to update these layers every two years, but that is contingent on funding.

Data Preparation

Data Reduction & Simplification

• Re-project data to Geographic WGS-84 (EPSG:4326).
• Separate building types shed and garages in the source data. The planimetric data combined these two categories, whereas they are distinct building types in OSM. If a building square footage was below 240 sq ft (a typical size of a one-car garage), then the building was reclassified as shed. All others classified as garage/shed were reclassified to simply garage.
• Convert other building types to OSM building tags.

Planimetric Building Type	OSM Building Tag	Count	Comment
Commercial	commercial	39,465
Garage/Shed	garage	254,960
Garage/Shed	shed	153,773
Foundation/Ruins	ruins	1,622
Industrial	industrial	9,575
Medical	hospital	922
Misc	yes	11,432	"yes" is the default building tag for an unknown building
Parking Structure	garage	427	other tag: amenity=parking
Public	public	9,269
Residential	residential	724,868
Tank	yes	5,673	other tag: man_made=storage_tank

• Merge Colorado Office of Information Technology (OIT) state address layer (which is public and governed by these terms of use) with roofprints to obtain address information for buildings. There are 797,222 addressable features in the dataset (non-addressable features include garages, sheds, tanks, and parking structures). Of the addressable features, 75% (598,838 records) have an address. The remainder were not addressed either because the address data was not shareable or because there was not an address match. To achieve a match, two spatial joins were performed with the OIT address points and roofprint polygons.  First using “point falling inside polygon”, then using “closest point” for the remaining records.  A QA/QC process overlaying the addressed footprints with parcel data was performed.  Roughly 50,000 records did not match, and were dropped.   The combination of these dropped records and those that were not shareable resulted in the remaining 598,838 records with address data. 

Tagging Plans

The data schema was modified for the import.

Shapefile Field Name	OSM Tag	Description
DRCOGID		Unique building identifier
HOUSENUMBR	addr:housenumber=*	Address house number
STREET	addr:street=*	Address street
CITY	addr:city=*	Address city
COUNTY	addr:county=*	Address county
STATE	addr:state=*	Address state
BLDG_TYPE	building=*	Building type
BLDG_HT_M	height=*	Building height in meters (converted from feet)
OTHER_TAG		Other tags further describing building type

Changeset Tags

#DRCOGPlanimetrics + #[City][Feature] (i.e. #DenverBuildings)

Data Transformation

Single Polygon Buildings

After completion of the garage/shed reclassification and the address data merge the resulting shapefiles were imported into a PostGIS database for further processing. A 1000 meter x 1000 meter grid covering the import area provided classification for organizing the individual buildings into discreet groups attributed based on the arbitrary X/Y designation of the grid cell containing the building. The grid cell designation follows the format "rowNum_colNum". The Python script that produces the OSM data files used for the import exports buildings based on their grid assignment. This allows the OSM data file names to include the corresponding grid id value. Therefore the file named "drcog_building_grid_ref_77_22.osm" contains all of the buildings from the grid cell intersecting row 77 and column 22. The OSM Tasking Manager tasks are broken up using the same grid so that the building data download URL can easily be associated with the Tasking Manager grid cell.

A wrapper around the OGR2OSM Python script converts the PostGIS data into individual OSM files each tagged with the corresponding grid from the tasks GeoJSON file.

Multipart Buildings

The DRCOG data contains one "building" for each part of a physical building which has a different roof height. We wanted to group the parts that belonged to a physical building together in an OSM building relation. Each "building" in the DRCOG data contains a building id (DRCOGID), whose value is shared by the various "buildings" that comprise a physical building. We created a Python program (make_building_relations.py) to make use of this information to create building relations.

The following applies if a DRCOG "building" shares an ID with one or more other "buildings." Where this is not the case, the program simply copies the building information as is to the output.

• Building relations are created and tagged as:
  • type=building
• Each DRCOG building is added to a building relation with role=part, and is tagged with:

  Shapefile Field Name	OSM Tag	Description
  BLDG_TYPE	building:part=*	Building type
  BLDG_HT_M	height=*	height in meters

• Outline - All of the parts that make up a given building are unioned together
  • Added to relation and with role=outline
  • Tagging
    • Any tags from the original buildings (now considered parts) other than height=* are applied to the outline. In case the parts have different values for the tags to be transferred, we arbitrarily use the last one encountered.

      Shapefile Field Name	OSM Tag	Description
      DRCOGID	drcog:bldg_id=*	Unique building identifier
      HOUSENUMBR	addr:housenumber=*	Address house number
      STREET	addr:street=*	Address street
      CITY	addr:city=*	Address city
      STATE	addr:state=*	Address state
      	addr:postcode=*	Address postcode
      BLDG_TYPE	building=*	Building type
      	source=DRCOG PLANIMETRICS DATA

Data Transformation Results

• The data transformation results in 4,668 OSM files ready for human editor review
• These 4,668 files correlate to the 4,668 1000 meter by 1000 meter grid cells that cover the import area
• Each file name will contain the row and column reference that correlates the file to the corresponding project grid cell, for example "80_52"

Data Merge Workflow

Team Approach

This is a team effort led by Ashley Summers of DRCOG with Russell Deffner, Chad Hammond, Jim McAndrew and eventually any and all in OSM-Colorado, OSM-US, or generally on OSM imports and other mailing lists.

References

The team referred to Los Angeles, California/Buildings Import as a reference and guide to organize this import.

Import Workflow

• After completion of the above Data Transformation steps the resulting OSM xml files will be made available for download
• A project-specific instance of the OSM Tasking Manager will be setup to house the project or projects related to the import
• Remember to create an import-specific account for any work done as part of this project; do NOT use your main OSM ID
• Select a task in the Tasking Manager by clicking on the project area map or using the "Take a task at Random" button
• Once in the task click the "Start Mapping" button
• In the "Extra Instructions" section of the task screen click the link to download the OSM data file for the task
• Open the OSM file in JOSM and follow the instructions in the detailed workflow instructions locate here (add link)
• Conflation will be handled in the detailed workflow and by the JOSM validator before upload.
• QA will be undertaken by editors in JOSM when working with the OSM data file for an individual Tasking Manager task.
• Detailed QA steps are outlined in the detailed workflow document