HSL bus stop import

From OpenStreetMap Wiki
Jump to navigation Jump to search



HSL bus stop import is an import of Helsinki Regional Public Transportation Authority (HSL) public transportation stop data covering HSL operating area. The area consists of cities and municipalities around Greater Helsinki area of the capital city of Helsinki in Finland. The import implementation is (as of May 12th 2021) done.

Goals

  • Update the ref=*-tag of local public transportation stops (bus, commuter train, tram and subway stops) in Helsinki to include the "H" letter in front of the numeral identifier.
  • Import attribute data for public transportation stops from HSL data for the whole HSL area. For example if a stop is sheltered or if a name is missing in either Finnish or Swedish.

In addition to a name the physical HSL public transportation stop signs have alphanumeric reference, "a short code" that consist of a letter prefix with numerical part. For example V6110 of stop "Orvokkitie" where "V" stand for city of Vantaa. In OSM the "short code" is stored in ref-tag of the stop. For historical reasons the public transportation stops of Helsinki did not previously have a letter prefix on the stop sign of the physical stop. These days the prefix of "H" for Helsinki is included on the stop sign but many of the old OSM stops in Helsinki do not have the updated value.

Schedule

By the spring of 2021.

Import Data

Background

Data source site: HSL stops
Data license: https://creativecommons.org/licenses/by/4.0/
Type of license (if applicable): CC BY 4.0 (waiver required, see link to permission below)
Link to permission (if required): https://wiki.openstreetmap.org/w/images/7/75/HSL_permission.pdf
OSM attribution (if required): https://wiki.openstreetmap.org/wiki/Contributors#Helsinki_Regional_Transport_Authority_.2F_Helsingin_Seudun_Liikenne_.28HSL.29
ODbL Compliance verified: yes

Attributes to be imported

Full description of the source data can be found in the reference document (only in Finnish). Rough translation can be found in the table below. The imported attributes determine if a stop is sheltered or not and add possible missing stop names in the two official languages of Finnish and Swedish.

Source attribute Description Example source value Targeted OSM-tag
LYHYTTUNNU Stop reference ID in physical stop sign H2007 ref=*
NIMI1 Name of the stop in Finnish Ritarihuone name=* and name:fi=*
NAMN1 Name of the stop in Swedish Riddarhuset name:sv=*
PYSAKKITYY Stop type (rough translation from Finnish)

01 = "glass shelter"

02 = "steel shelter"

03 = "terminal"

04 = "a post"

05 = "urban shelter"

06 = "concrete shelter"

07 = "wood shelter"

08 = "stop position" or "virtual stop"

09 = "trunk line"

04 shelter=* Values "04" and "08" stand for "post" and "virtual stop", thus resulting in shelter=no.

Value "99" is unknown or blank stop type and does not result in a new tag.

Rest of the PYSAKKITYY values stand for different types of sheltered stops, thus resulting in shelter=yes.

OSM Data Files

hsl-osm-stops-2020-08-11-EXAMPLE.osm - file containing public transport stops in HSL area downloaded from Overpass API via jOSM using following query. Overpass Turbo

try it yourself in overpass-turbo
// define HSL area filter with participant municipalities (OSM municipality)
(
  area[name="Helsinki"];
  area[name="Espoo"];
  area[name="Vantaa"];
  area[name="Kirkkonummi"];
  area[name="Kerava"];
  area[name="Kauniainen"];
  area[name="Sipoo"];
  area[name="Tuusula"];
  area[name="Siuntio"];
)->.hsl;

// different public transportation stop types in HSL area

(
  // tram stops
  node["railway"="tram_stop"]["ref"](area.hsl);
  way["railway"="tram_stop"]["ref"](area.hsl);
  relation["railway"="tram_stop"]["ref"](area.hsl);

  // bus stops
  node["highway"="bus_stop"]["ref"](area.hsl);
  way["highway"="bus_stop"]["ref"](area.hsl);
  relation["highway"="bus_stop"]["ref"](area.hsl);

  // train and subway stations
  node["railway"="station"]["public_transport"="station"]["ref"](area.hsl);
  way["railway"="station"]["public_transport"="station"]["ref"](area.hsl);
  relation["railway"="station"]["public_transport"="station"]["ref"](area.hsl);

  // include stop area relations
  relation["public_transport"="stop_area"]["ref"](area.hsl);

  // include stop position
  node["public_transport"="stop_position"]["ref"](area.hsl);
)->.hsl_stops;

// nodes part of subway relation in HSL area.
(
  rel(area.hsl)[type=route][route=subway];
)->.metro;

// subway station platforms and stop positions (each subway station has two platforms with unique ref)
(
  way(r.metro)[ref](area.hsl);
  node(r.metro)[ref](area.hsl);
)->.metro_stops;

// output union
(
  .hsl_stops;
  .metro_stops;
);
out body;
>;
out meta;

Import Type

This import is a one-time import.

jOSM will be used for entering the updated data into the OSM database.

Data Preparation

Data Reduction & Simplification

The import will target public transportation stops within the cities and municipalities of HSL operating area by updating and adding new tags to existing nodes. The import will not add, modify or remove any geometry. Only tags of stops with a match in source data are added / updated. OSM data is considered "master data". Attribute data will not be imported in conflicting cases. Additionally matching stops are validated based on their geographic location. OSM-stop not within 100m distance of the HSL stop will not be updated (the matched stop not within the 100m is logged for manual surveying).

The import deals only with attributes because missing stops have already been imported previously in the national Finland:Digiroad/Digiroad bus stop import.

Tagging Plans

Only stops with a matching ref=*-tag will be updated. Following stop types are targeted:

Stop type OSM-tags Example
Bus stops highway=bus_stop https://www.openstreetmap.org/node/4938329537
Tram stops railway=tram_stop https://www.openstreetmap.org/node/313760579
Commuter train and metro stations railway=station with public_transport=station https://www.openstreetmap.org/node/1516867775
https://www.openstreetmap.org/node/146450892
Metro station platforms public_transport=platform with railway=platform that are parts of route=subway-relation https://www.openstreetmap.org/way/489577759
Metro station platform stop position public_transport=stop_position with railway=stop that are part of route=subway-relation https://www.openstreetmap.org/node/5320327937

1. Updating the ref-tag

All OSM public transportation stops are first validated against the source dataset. HSL source data stop reference "LYHYTTUNNU" is matched with the OSM stop ref-tag value. Some OSM stops already have the "H"-prefix in the ref-tag value.

The "H"-prefix is added to the ref-tag value of a matching stop if it exists within Helsinki and the OSM stop is missing the "H"-prefix.

2. Creating new tags for the stops

Attribute data is added to each matching stop after initial matching between the source data set and the existing OSM stops. Following tags are added only if the target OSM stop does not already have them:

  • shelter=no if the stop is not protected from the rain by a shelter
  • shelter=yes if the stop is protected from the rain by a shelter
  • name=*, name:fi=* and name:sv=* for Finnish and Swedish name for the stop.

Shelter info is not added for OSM relations (for example public_transport=stop_area) and public_transport=stop_position.

Changeset Tags

Data Transformation

A Python script update-tags.py is used in the transformation. The script takes inputs of HSL public transport stop data in GeoJSON-format and .osm data. OSM Data is downloaded via the Overpass API in jOSM. The script outputs a modified .osm-file, some statistics of the modifications and a log file. Latest version of the script can be found in the https://github.com/HSLdevcom/osm-stop-import repository.

  • Script updates OSM ref=*-value with "H"-prefix if the municipality of the matched stop is Helsinki and the "H"-prefix is missing in ref-tag.
  • Script adds tag shelter=yes or shelter=no if the OSM stop does not already have shelter=*-value *and* HSL data has shelter info for the stop.
  • Script adds name=*, name:fi=* and name:sv=* tags if the OSM stop does not already have them.

Data Transformation Results

[1] - modified .osm file.

Data Merge Workflow

Team Approach

References

List all factors that will be evaluated in the import.

OSM data is considered "master" data and overrules in conflicting cases and even with matching stops. Conflicts will be logged and later on manually investigated, in order to get possible existing errors fixed.

The lists of conflicting values will be generated for shelter and names. Additionally there will be a list of OSM stops that have a ref-tag but not a matching value in HSL stop data.


The import will be done in 4 geographically separate batches. The areas are based on the HSL municipalities.

Batches by municipality Status
Kirkkonummi and Siuntio Submitted on 2021-05-03 - Changeset & log files
Tuusula, Kerava, Sipoo Submitted on 2021-05-05 - Changeset & log files
Espoo, Vantaa, Kauniainen Submitted on 2021-05-10 - Changeset & log files
Helsinki Submitted on 2021-05-12 - Changeset & log files
HSL bus stop import - Fix stops missed due mistake reading source data Submitted on 2021-05-15 - Changeset & log files

Step by step

  1. Download HSL stop data to a .geojson file from HSL AGOL Portal.
  2. Query public_transportation stops with ref=*-tags from HSL area via Overpass API in jOSM and save the file to a .OSM XML-file. Overpass Turbo
  3. Run update-tags.py, a Python script that manipulates the .OSM file and matches OSM-stops to HSL-data using ref=* tag and "PYSAKKITUNNU" values. Matched stops are validated based on their location (100m margin).
  4. Upload the modified .OSM file to with OSM database with jOSM with a import dedicated user System-users-3.svghsl-stop-import (on osm, edits, contrib, heatmap, chngset com.) .
Changeset size policy

The import will modify approximately 6 700 objects. The import will be uploaded in 4 geographically separed batches each resulting in a different changeset. Addiotionally depending on the changeset size, the upload to OSM database is done in chunks of 1000 changes per chunk via jOSM.

Revert plans

If any reverting issues arise, jOSM Reverter plugin will be used to revert the changeset. The import will be done with a dedicated user account System-users-3.svghsl-stop-import (on osm, edits, contrib, heatmap, chngset com.).

Conflation

Conflation is rather straight-forward as the import does not add, remove or modify any features. The import only modifies ref=* tag in Helsinki area stops by adding "H" prefix if a OSM stop with a matching source data stop is missing the prefix. Script creates new tags name=*, name:fi=*, name:sv=* and shelter=* only for existing features in case the tags are missing in OSM. In case of conflicting tag values OSM is considered master data and no tag values are modified.

QA

The import is run in four, geographically separate batches. The results are reviewed manually and by overpass-turbo queries between each batch. Additionally the script produces logs files for conflicting cases that can be reviewed manually and used as a basis for field work. The script creates lists of: OSM-stops having conflicting shelter value and osm-stops having a match with HSL stop data but the location difference between the two stops is more than the max limit.

See also

The email to the Imports mailing list was sent on 2020-08-25 and can be found in the archives of the mailing list at [2].

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=HSL_bus_stop_import&oldid=2614179"