Santa Clara County, California/San Jose Sidewalk Import

As a first step toward mapping the pedestrian network for Santa Clara County, this import would significantly improve OpenStreetMap's coverage of sidewalks in San Jose, California. We are using data derived from a shapefile released into the public domain by the City of San José.

This import will be performed piecemeal using the OSMUS Tasking Manager. Contributors to the tasking manager project will import small portions of the city at a time, taking care to adhere to best practices around sidewalk and crosswalk mapping. There will not be a large-scale, automated import of any kind. We are following the example set by successful imports of sidewalk data in Seattle and Southern Ohio.

The data we are importing contains only sidewalks and not crossings or curb cuts. The thrust of this project is to verify the accuracy of the sidewalk data and insert it on the map to form the backbone of the pedestrian network. Since we are only importing sidewalks, we will use commonly accepted, well supported tags, supplemented where necessary by the sidewalk schema proposed by the OpenSidewalks project.

After the sidewalks are uploaded, we will do a crossings import. Crossings candidates are auto-generated through an algorithm, reviewed by a person who deletes the bad candidates and uploads only the good ones.

Afterwards we will go over the network again to finish adding all the crossings and make sure the sidewalks are connected.

Both imports were completed by September 2018. The OSMUS tasking manager went offline so the final part of the import was coordinated with Google Sheets. Any more work on the pedestrian network will be done manually and outside of imports.

Import Plan Outline

We created a Tasking Manager for this project http://tasks2.openstreetmap.us/project/127. We broke San Jose into about 640 sections using VTA TAZ census blocks. Some zones are bigger than others.

To an individual helping with this import it will consist of:

1. Visiting the project
2. Selecting a (very small, ~a few blocks) region to import
3. Downloading the relevant chunk of data for that region via the Tasking Manager and opening as a layer on JOSM.
4. Reviewing the data for accuracy based on ESRI or DigitalGlobe Premium imagery. Mapillary may be excessive at this stage. Bing is okay but is old.
5. Adding the local data and checking for existing sidewalks (there are very few of these)
6. Adding nodes for relevant intersecting features such as parking isles but not buildings or areas
7. Upload their region to OpenStreetMap, pre-filled with a changeset tag referencing the San Jose Sidewalk Import Task
8. Getting their submission validated by another local user.

The crossings import will only be done by advanced users. For the crossings import:

1. Import the set of crossings into JOSM (Should be around 200 crossings per group, 100 groups total)
2. View every single of the auto-generated crossings and delete any that don't appear to correspond to actual crossings (based on the presence of pavement markings or curb cuts).
3. Download the streets using the JOSM advanced mode overpass api
4. Create intersecting nodes using intersecting way command, tag with highway=crossing (the ways are already tagged)
5. Merge created crossing nodes to any existing crossing nodes
6. Download the sidewalks using the JOSM advanced mode overpass api
7. Connect sidewalks to crossings, fix duplication errors.
8. Upload the region to OpenStreetMap, pre-filled with a changeset tag referencing the San Jose Sidewalk / Crossing Import Task
9. Get the submission validated by another local user

After the generated crossings are imported for an area users should go in and manually fill in any missing crossings using this MapRoulette challenge.

Goals

The primary goal of this import is to create a backbone of data for the pedestrian network within San Jose and Santa Clara County by importing open data on sidewalks from the San Jose City Public Works GIS Data page. This original source data is created from orthoimagery and is in polygons. This data is available under the public domain and has no licensing restrictions.

For the most part, sidewalks are not annotated in OpenStreetMap in the San Jose region, so this import would address a fundamental lack of information about the pedestrian right of way for this region. To explore the relevance of pedestrian network data to the question of accessibility, see the related accessmap.app project. However, sidewalk data is relevant to most individuals as it represents fundamental infrastructure supporting most other modes of transport in urbanized areas: public transit, automobile traffic, and cyclists.

Schedule

• September 2017: Getting Review Approval
• September 2017: Start Sidewalk Importing
• January 2018: Sidewalk import completed

• January 2018: Started Crossings import
• September 2018: Completed Crossings import

Import Data

San Jose Pubic Works GIS Data:

• Sidewalks: http://www.sanjoseca.gov/index.aspx?NID=3308 Basemap_2 sidewalk areas.
• Copyright status: Public domain under the California Public Records Act, though we will be acknowledging the City of San José as the source.
• ODbL compliance verified: yes

Derived sidewalk data:

• Sidewalks GeoJSON before fixing duplicates: https://drive.google.com/open?id=0B098fXDVjQOhd1pGWW5Dc25UeFE
• Sidewalks OSM after run through osmizer fixing duplicates: https://drive.google.com/file/d/0B098fXDVjQOhUDZMQ3o4NENzOWM/view?usp=sharing

Versions of the generated crossings data. Remember the importer is expected to delete any bad candidates. This will change with every sidewalk import or if people manually add crossings.

• ""Crossings GeoJSON Folder"" https://drive.google.com/open?id=11R5tYJNXsdYI4tpseoGJ-3D5y430QTx9

OSM Data Files

Generated on-the-fly based on neighborhood + block regions.

Import Type

The import will not be automated. We'll ask experienced mappers to create a separate import username such as username_sjsidewalks_import.

Data Preparation

Full script and details located here for prepping the sidewalk data: https://github.com/codeforsanjose/OSM-SouthBay/blob/master/SJ_Sidewalks/SJ%20Sidewalks%20Processing%20Steps.ipynb

The software for crossify is here: https://github.com/3vivekb/crossify/tree/crossify_adjustments-%233

Tagging Plans

We'll just copy the reasoning of the Seattle Imports. Some work is being done with tagging streets but ultimately we think sidewalks as separate ways will be better.

We recognize we could be creating sidewalk islands since we aren't dealing with crossings or connecting the sidewalks to streets but most routing software doesn't use sidewalks and typically uses left/right tags. Regardless, adding sidewalks will eventually make things better.

There are attributes connected to the original sidewalks but they get lost during the data preparation process. None of the attributes available in the source data are typically used in OSM.

For the sidewalk data we will use source=San José Public Works Department

For the crossings import, the crossing ways will simply be tagged with highway=footway and footway=crossing. The node on the major street will be tagged highway=crossing. The changeset will be tagged source=Bing or source=DigitalGlobe Standard, depending on the aerial imagery used to verify each crossing.

Inspirations

We are taking inspiration from the Seattle import: Seattle Washington Sidewalk Import

Workflow

Our workflow is as follows:

• Convert the city data to acceptable sidewalks in geojson format and split into tiny zones
• Convert the tiny geojson zones of sidewalks into OSM XML using osmizer
• Create an OSM Tasking Manager project using that set of polygons.

Our changeset size policy is smaller than census tract, and can be easily reviewed without making major map changes.

We take responsibility to revert any changes that cause breakages.

Conflation

Our conflation strategy is we ask the importer to make a judgement call on which sidewalk is better and use that one accordingly. For the most part, we do not expect many conflation issues, due to the lack of existing sidewalk data.

Expected issues related to conflation:

• Shared nodes with other pedestrian ways.
  • The sidewalk data will need to be combined with existing pedestrian ways intended to describe non-sidewalk pedestrian paths, such as plazas and trails.
  • In all such instances, a shared node will be added between the existing footpath and the new sidewalk data.
• Existing pedestrian ways connected to streets:
  • Many dedicated pedestrian ways, such as paths through parks, are currently connected directly to streets.
  • These will not be modified in any way, outside of adding a node (described above) where they meet sidewalks. This helps prevent our import from breaking any existing routing software.
• Existing crossings at intersections:
  • Where we encounter existing crossings at intersections we will make sure they are connected with the imported sidewalks.