Import/Catalogue/Kindergarten import Norway

From OpenStreetMap Wiki
Jump to navigation Jump to search

Import Plan Outline

The Norwegian Directorate for Education and Training (utdanningsdirektoratet) has released a dataset about all kindergartens in Norway, available at data.udir.no/baf. Kindergartens which are manually associated with a Key:no-barnehage:nsrid will be kept automatically updated with changes in the data.udir dataset. This automatic update does not overwrite any modified openstreetmap values, a key-value is only modified if the previous opensteetmap data agrees with the previous data.udir value. See below for more details. A (norwegian) website has been setup to track progress and to view and/or download extracted data, please see: barnehagefakta_osm_data

Goals

The goals are

  • Provide mappers with a simple view of the NBR data which is relevant to OSM.
  • Keep any added tags up-to-date with changes from NBR.
  • To provide a consistent set of tags for kindergarten data in Norway

Schedule

This will be an ongoing community conflation/import. As an initial step, kindergartens allready in OSM must be associated with the current NBR tags, after which an automatic script will keep the tags up-to-date. After that, kindergartens missing from OSM should be added, based on local knowledge.

Import Data

Background

Links to sources.

Data source site: http://data.udir.no/baf/
Data license: http://data.norge.no/nlod
Type of license: NLOD, Norwegian Licence for Open Government Data.
Link to permission: Approval from Utdanningsdirektoratet (in Norwegian) - http://lists.nuug.no/pipermail/kart/2015-October/005780.html
OSM attribution: https://wiki.openstreetmap.org/wiki/Contributors#Norway
ODbL Compliance verified: yes

OSM Data Files

The prepared OSM data can be found under http://obtitus.github.io/barnehagefakta_osm_data

.osm files are made available for each municipalities, e.g. http://obtitus.github.io/barnehagefakta_osm_data/data/0101/barnehagefakta.osm where 0101 is the municipalities id.

Import Type

The inital conflation/importing will be done manually. See Manual Workflow below.

An automated scripts has been written to keep kindergartens which are manually associated with a Key:no-barnehage:nsrid automatically updated with changes in the data.udir dataset. This automatic update does not overwrite any modified openstreetmap values, a key-value is only modified if the previous opensteetmap data agrees with the previous data.udir value.

The automated script relies on https://github.com/xificurk/osmapis for the upload to the OSM database. The manual part can be done in JOSM with the provided .osm files, or any other OSM editor by copying from the webpage http://obtitus.github.io/barnehagefakta_osm_data

Data Preparation

Data Reduction & Simplification

The dataset contains meta-data which are not relevant to OSM and some data which the local community has decided not to include. The original data contains nodes (single lat/lon), but these can be merged to existing closed ways (areas) or relations as appropriate.

Tagging Plans

See Tag:amenity=kindergarten for how to map a kindergarten, as an example of tags from data.udir

Either changesets or each object should contain source=data.udir.no, preferably combined with survey and/or knowledge, e.g. source=data.udir.no;survey.

The id can be used in the following urls (using "888839542" as an example)

Not all kindergartens contain all of the keys.

Changeset Tags

Changesets should contain source=data.udir.no.

Data Transformation

The github code can be found here: https://github.com/obtitus/barnehagefakta_osm the main 'transformation' is in barnehagefakta_osm.py.create_osmtags()

Data Transformation Results

Example .osm file http://obtitus.github.io/barnehagefakta_osm_data/data/0101/barnehagefakta.osm

Data Merge Workflow

Team Approach

This will be a community import. The scripts, automated edits and webpage is maintained by OSM User:Øystein Bjørndal.

References

Manual Workflow

Based on local knowledge or previous OSM data, add all missing tags from data.udir. There are a couple of starting points for contributing:

Option A: Using the webpage

  1. Find a missing or incorrect kindergarten here: http://obtitus.github.io/barnehagefakta_osm_data.
  2. Using your favorite OSM editor, copy tags from the "Tags fra NBR" column into OSM. This is particularly simple in JOSM, as it allows you to paste all of the key=value pairs directly.

Option B: Using JOSM

  1. Find the barnehagefakta.osm for the municipality you are interested in: http://obtitus.github.io/barnehagefakta_osm_data.
  2. Download and open in JOSM
  3. Copy nodes over to the appropriate node, area or relation.

Option C: POI importer

  1. Open JOSM and ensure "Enable remote control" is enabled by going to "Preferences" -> "Settings for the remote control feature.".
  2. Using the POI_Importer zoom in on the area of interest: http://poi-importer.github.io/#map=5/65.3577/18.7646&datasets=NOkg
  3. After nodes are no longer grey, click a flag and press either the "Import Data" or "OSM Data" buttons on the popup. Please note the following warnings: The tool looks for amenity=kindergarten in a small radius around the NBR data point, it will therefore miss kindergartens already in OSM:

Option D: Using Python

If you want to conflate a large number of kindergartens, the following python script might be useful: https://github.com/obtitus/barnehagefakta_osm/blob/master/conflate_osm.py feel free to open an issue if stumble on some bugs.

Automated Workflow

  • For each modification in data.udir
  • Ignore if the no-barnehage:nsrid value has not been added to OSM
  • Auto update if previous OSM data matches exactly previous data.udir
  • Mark as conflicting if the OSM data does not match previous data.udir.

Example:

  • The initial data.udir data says that contact:phone=1
  • This is modified in OSM to the (correct) contact:phone=123456789. The automated script will not notice, as the data.udir data has not been modified.
  • data.udir updates the phone number to contact:phone=987654321. The script compares the previous value ('1') with the OSM value ('123456789'), as these do not match, the conflict must be manually handled.

Conflation

Conflation will be manually handled, with the exception of the trivial case of the OSM data matching the previous data.udir data as described above. Any dump of the data.udir nodes into OSM is highly discurraged.

QA

Feel free to add to the QA!

Do you need the id no-barnehage:nsrid=*?

The id is necessary for keeping the information up-to-date. This is especially important for keeping the contact:* tags updated. (Kindergartens have a tendency to stay stationary and keep their names, but we expect the contact, operator and capacity tags to change over time).

None of the other tags can be realistically used for a one-to-one mapping, as the name in OSM does not need to match the name in the dataset and the name might also be used on other objects (like bus-stops).

The id can also be used to look up the raw API response: http://barnehagefakta.no/api/barnehage/888839542 or the slightly prettier: http://barnehagefakta.no/barnehage/888839542

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Import/Catalogue/Kindergarten_import_Norway&oldid=2622020"