Import/New Zealand Street Addresses (2021)

From OpenStreetMap Wiki
Jump to navigation Jump to search
LINZ Main Page LINZ Building Import LINZ Address Import Missing Streets LINZ Place Name Import
LINZ Address Update
LINZ Address Import Screenshot.png
Author: Kylenz
License: MIT License
Platform: Web
Status: Active
Version: 2.0.0 (2023-12-29)
Language: multiple languages
Website: https://osm-nz.github.io/RapiD
Source code: osm-nz/linz-address-import GitHub
Programming language: TypeScript

Modification of RapiD to compare and update OSM data based on LINZ data

Background

This page documents a plan to continue the 2017 import of street address data from Toitū Te Whenua Land Information New Zealand (LINZ) into OSM. Since the initial import in 2017, the imported data has become out of date.

This project aims to update the address data and set up a system to regularly update addresses in OSM, by conflating them with the data from LINZ.

A screenshot of the import tool, showing 42 addresses to be added, 1 to be deleted, and 1 building requiring an edit
A screenshot of the import tool showing an address that needs to be moved

Goals

  • To import the 161,000 missing addresses for the New Zealand mainland (and the outer islands, where data exists)
  • To fix typos and other issues with 17,000 existing addresses in OSM that were added in 2017.
  • To add the ref:linz:address_id=* tag to 40,000 addresses where the tag was not added during the initial import, or where it was removed when address points were manually merged with buildings
  • To delete 11,000 addresses that were imported but since deleted by LINZ. See the #Address Deletion section below
  • To set up a system that is simple and efficient, which allows OSM street address data to be regularly conflated with LINZ's data

Data source & Import code

The data is sourced from the NZ Street Address layer from LINZ.

The source code is available from osm-nz/linz-address-import GitHub.

How it works

A script runs regularly which compares the latest LINZ Street Address data with the latest OSM data for the New Zealand mainland. This script requires at least 16 GB of RAM and takes approximately 5 minutes to execute. This script automatically runs weekly using GitHub Actions (a free service), and the generated files are stored in a free  CDN.

This script conflates the street address data from LINZ with OSM and categorizes each address into one of the categories in the table below, depending on what changes are required.

This data is made available for anyone to add/update/move/delete, using a fork of the RapiD editor designed for this particular import.

How do I contribute?

The tool is available from the a link on the homepage.

Categorization

Each address in the LINZ data is processed, and categorized as follows:

ID Status Status Description Action
1 PERFECT An address node already exists in OSM with the same ref:linz:address_id=*, and the data is correct none
2 EXISTS_BUT_WRONG_DATA An address node exists in OSM with the same ref:linz:address_id=*, BUT the data in the OSM node doesn't match LINZ's data Anyone can use the fork of RapiD to edit this data
3 EXISTS_BUT_NO_LINZ_REF There is no OSM address node with a matching ref:linz:address_id=*, BUT there is an OSM address node with the same data, just no ref:linz:address_id=*
4 MULTIPLE_EXIST_BUT_NO_LINZ_REF There are multiple address nodes for this address, none of which have a ref:linz:address_id=*
5 MULTIPLE_EXIST There are two different nodes in OSM with the same ref:linz:address_id=* Requires manual action for complex situations
6 EXISTS_BUT_LOCATION_WRONG The address exists in OSM, with the ref:linz:address_id=* tag but the coördinates are totally wrong (>500metres away from the gazetted location) Anyone can use the fork of RapiD to add, move, or delete this data
7 TOTALLY_MISSING Address doesn't exist in OSM at all
8 NEEDS_DELETE The address was imported from LINZ, but since then LINZ has deleted the address (see #Address Deletion)
9 NEEDS_DELETE_NON_TRIVIAL The address was imported from LINZ, but since then LINZ has deleted the address. However, an OSM mapper has added business/POI information to the address node (e.g. name=*, shop=*, craft=*). See #Address Deletion. Requires manual action
10 CORRUPT The address in OSM contains multiple ref:linz:address_id=*, separated by a semicolon. Anyone can use the fork of RapiD to edit this data
11 LINZ_REF_CHANGED LINZ has changed the value of ref:linz:address_id=*. Effectively a delete and a re-create with a new ID.
12 obsolete
13 COULD_BE_STACKED Addresses in OSM that are PERFECT, but the flats/units could be stacked if this is desired. Stacked means the many individual nodes for each flat in an apartment complex are replaced with a single node for the whole apartment complex. More Info. none, just for statistics
14 NEEDS_DELETE_ON_BUILDING Similar to #9. The address was imported from LINZ, but since then LINZ has deleted the address. However, an OSM mapper has merged the address node into a building. We won't delete the building, but we will remove the address tags from it. See #Address Deletion. Anyone can use the fork of RapiD to edit this data
15 REPLACED_BY_BUILDING The address exists twice in OSM: once on an imported node, and once on a building. We will merge the node into the building. This is usually caused by StreetComplete users.

Tagging

The tagging scheme used is the same as the original 2017 import:

LINZ Field Name OSM Tag
address_id ref:linz:address_id=*
full_address_number addr:housenumber=*
and alt_addr:housenumber=* if the property has an 'Alternate Address'
full_road_name addr:street=*
suburb_locality addr:suburb=* if urban, addr:hamlet=* if rural[1]
town_city addr:city=* if urban, does not exist for rural addresses
water_name If present, addr:type=water will be added.

Other tags:

  • building:flats=* - added if multiple flats are stacked[2] together.
  • check_date=* - added by the editor if an importer ignores the suggestions from the conflation service. Addresses with a check_date will be ignored by the conflation process if the date is less than a year ago.
  • alt_addr:housenumber=* - added to addresses that have an "alternative address", for example if 123A and 1/123 are both valid house numbers for the same place.
  • linz:stack=no - mappers can temporarily add this tag to a stacked[2] address to tell the system to delete the stacked node and replace it with individual nodes for each appartment. This will happen next time the sync runs.

Potential issues

This table will be updated as the project progresses.

Issue Mitigation
Duplicate data being imported The fork of RapiD has an added feature to prevent duplicate addresses being imported based on the ref:linz:address_id=* tag. This conflation happens in real-time, in the browser
Multiple people editing the same dataset at the same time Users will be presented with a warning if someone else is/was editing that dataset in the last hour

Address Deletion

There are cases when an address was imported into OSM from LINZ, and then deleted by LINZ. Part of this proposal is to delete previously imported addresses in OSM that have been deleted by LINZ. This is possible using the fork of RapiD.

Example: In the following image, address 14 Subdivision Road has been deleted because it no longer exists. Five new properties now occupy the former  lot including a new road.

An example of a property that was subdivided, and the original address (No. 14) was deleted.

Out of scope

  • Dealing with duplicate addresses in OSM. If one address has a ref:linz:address_id=*, this project will normally ignore any other identical addresses. There is one exception where this is automatically fixed, see REPLACED_BY_BUILDING above.

Discussion with the community

This import was first proposed via an email to the talk-nz mailing list and another email to the imports mailing list. The feedback was entirely positive.

During testing, node a single node was uploaded to OSM and then deleted.

See Also

Process Diagram

LINZ Address Conflation Flowchart.png

  1. Discussed recently on the community forum
  2. 2.0 2.1 See here for information about stacked addresses