Import/South Australian Roads

About

The South Australian government has provided a dataset "Roads" in shapefile format, along with explicit permission to use it in OpenStreetMap. The description of the data is: Statewide Road Network including sealed and unsealed roads. The dataset represents navigable roads, including public and private access roads and tracks

As well as covering South Australia, the data also extends approximately 100km into Victoria.

This page will describe a plan to import some of this data to OpenStreetMap.

Import Plan Outline

The source data will be converted to .osm format. The data will be then filtered, to find roads which are missing from OSM or named differently in OSM. This filtered data will be made available to the OSM community. It will help mappers identify roads in OSM which need adding or correcting. Using an editor such as JOSM, mappers will be able to copy and paste entire missing roads into OSM, or copy and paste missing or incorrect tags

Background

Data source site: http://data.sa.gov.au/dataset/roads

Data license: http://creativecommons.org/licenses/by/4.0/

Link to permission: Attribution/sa.data.gov.au_explicit_permission

Resulting OSM Data File

The file(s) are available for download from the following Google Drive shared folder:

https://drive.google.com/folderview?id=0B1JwNHL1bER0VlZLQkRZRUUySlk&usp=sharing

It contained up to 19475 ways. Amongst those, there are 12543 unique names.

Probably it would be better if I broke this up into more managable pieces, say 100 separate .osm files each containing a chunk of approximately 200 nearby ways, so people could volunteer to take on a whole chunk.

Progress can be checked via http://qa.poole.ch/

Import Type

This is a manual import, after scripted data processing and transformation.

Data Preparation

The file "Roads_shp.zip" was downloaded from data.sa.gov.au on 8 February 2015. The file is dated 7 October 2014 on the download page.

The program QGIS was used to covert the shapefile "Roads.shp" to another shapefile "Roads WGS84.shp" having the WGS84 coordinate system. JOSM was used to convert that file to osm format, "Roads WGS84.osm".

JOSM was used to download every way tagged as "highway=*" in South Australia (plus margins around SA) that already exist in OSM. This data was saved as "existing highways 9 feb 2015.osm". The following Overpass API script was used to do the download:

(way[highway];>;
 relation[highway];>;
);
out meta;

The matlab scripts "readnodes.m", "find_missing_roads.m", and "get_existing_info.m" were used to analyse the files "Roads WGS84.osm" and "existing highways 9 feb 2015.osm", and produce the final output file "missing_v1.osm". The steps performed by these scripts are described below.

Data Processing

Simply speaking, the scripts find all ways in "Roads WGS84.osm" for which there is not an existing highway in OSM that is nearby with the same name. Those ways are then given the correct OSM tags (as described in a section below) and where appropriate are joined together to form longer ways; then saved in the output .osm file.

Two ways are considered "nearby" if their two bounding boxes are no further apart than approximately 100 meters. "Same name" ignores case, whitespace, and punctuation.

Tagging

I use the following tags from the source data to generate OSM-compatible tags: CLASS, SURFACE, NAME, ONE_WAY, ROUTENUM, ROADTYPE, ONTYPE, TYPESUFFIX.

The translation of tags is summarised in the following table. It does a fairly good job at translating ways that exist in both datasets, therefore it should also do a fairly good job for ways that are missing from OSM.

Source tags	Generated OSM tags	note
NAME=x, ROADTYPE=y, TYPESUFFIX=z	name=x y z
ONE_WAY=FT	oneway=yes
ONE_WAY=TF	oneway=yes	The script will reverse the direction of the way in this case
SURFACE=UNSE	surface=unpaved
SURFACE=SEAL	surface=paved
ONTYPE=BRDG	bridge=yes, layer=1
ONTYPE=CAUS	embankment=yes	Causeway
ONTYPE=TUNL	tunnel=yes, layer=-1
ROUTENUM=x	ref=x
CLASS=FREE	highway=motorway	Freeway
CLASS=HWY	highway=trunk	Highway
CLASS=ART	highway=primary	Arterial
CLASS=SUBA	highway=secondary	Subarterial
CLASS=COLL	highway=tertiary	Collector
CLASS=LOCL	highway=unclassified --or-- highway=residential	Local
CLASS=TRK2 or CLASS=TRK4	highway=track	Track

Notice the second-last row of the table. The source dataset does not seem to distinguish between what OSM labels "unclassified" and "residential" highways. The script makes its best guess for which is appropriate based on how many of each type exist already in OSM in the kilometer or so around the way in question.

Errors in the source data

If the source data is incorrect then you will need to contact the relevant local council. They will then notify the Roads unit within DPTI who will update their database and then this will, eventually make the data.sa.gov.au consistent with openstreetmap.

Errors under investigation

See https://github.com/q-bits/osm-scripts/issues

• Between Dora Street and Howard Street, First Street/Avenue (on the ground) Sefton Park - Prospect Council (rates officer) - Council advised DPTI 24/2/2015, Correction visible in PLB 3 Mar 2015
• Between Barton Terrace and Park Terrace, Jeffcott Road (on the ground) /Street, North Adelaide - Adelaide City Council - Council advised 26/2/2015, Council advised DPTI 27 Feb 2015, Correction visible in PLB 3 Mar 2015

Suggested Email Text

Once you've found the right person, an email wouldn't hurt.

I am an editor of OpenStreetMap www.openstreetmap.org and am processing errors found when conflating current OpenStreetMap data with DPTI road name data.

http://wiki.openstreetmap.org/wiki/Import/South_Australian_Roads Page on OpenStreetMap describing the process.
https://data.sa.gov.au/dataset/roads Data set source
http://dpti.sa.gov.au/open_data_portal Alternate link to data set (Roads)

This has found mis-spellings and typos on OpenStreetMap but has also highlighted an error in the roads database that DPTI maintains. The Roads unit require notification from the relevant LGA to correct the error in the DPTI Roads database.

https://www.sa.gov.au/topics/housing-property-and-land/land-services-industry/land-services-group-contacts#title7 Roads Unit

The DPTI roads database can be readily viewed at on the Property Location Browser and if you go to <<Give location>>, zoom in and then turn on the “Road Labels” layer and select satellite view to turn off the Google maps layer you can see their record.

http://maps.sa.gov.au/plb/ Property Location Browser

As you can see they have << Give nature of error>>

As discussed can you please advise the Roads Unit so the database can be corrected?

Can you let me know as this progresses so I can watch the error get flushed away through various systems?