Import/New Zealand Street Addresses (2021)
LINZ Address Update | |
---|---|
Author: | Kylenz |
License: | MIT License |
Platform: | Web |
Status: | Active |
Version: | 2.0.0 (2023-12-29) |
Language: | multiple languages |
Website: | https://osm-nz.github.io/RapiD |
Source code: | osm-nz/linz-address-import GitHub |
Programming language: | TypeScript |
Modification of RapiD to compare and update OSM data based on LINZ data |
Background
This page documents a plan to continue the 2017 import of street address data from Toitū Te Whenua Land Information New Zealand (LINZ) into OSM. Since the initial import in 2017, the imported data has become out of date.
This project aims to update the address data and set up a system to regularly update addresses in OSM, by conflating them with the data from LINZ.
Goals
- To import the 161,000 missing addresses for the New Zealand mainland (and the outer islands, where data exists)
- To fix typos and other issues with 17,000 existing addresses in OSM that were added in 2017.
- To add the ref:linz:address_id=* tag to 40,000 addresses where the tag was not added during the initial import, or where it was removed when address points were manually merged with buildings
- To delete 11,000 addresses that were imported but since deleted by LINZ. See the #Address Deletion section below
- To set up a system that is simple and efficient, which allows OSM street address data to be regularly conflated with LINZ's data
Data source & Import code
The data is sourced from the NZ Street Address layer from LINZ.
The source code is available from osm-nz/linz-address-import GitHub.
How it works
A script runs regularly which compares the latest LINZ Street Address data with the latest OSM data for the New Zealand mainland. This script requires at least 16 GB of RAM and takes approximately 5 minutes to execute. This script automatically runs weekly using GitHub Actions (a free service), and the generated files are stored in a free CDN.
This script conflates the street address data from LINZ with OSM and categorizes each address into one of the categories in the table below, depending on what changes are required.
This data is made available for anyone to add/update/move/delete, using a fork of the RapiD editor designed for this particular import.
How do I contribute?
The tool is available from the a link on the homepage.
Categorization
Each address in the LINZ data is processed, and categorized as follows:
ID | Status | Status Description | Action |
---|---|---|---|
1 | PERFECT
|
An address node already exists in OSM with the same ref:linz:address_id=*, and the data is correct | none |
2 | EXISTS_BUT_WRONG_DATA
|
An address node exists in OSM with the same ref:linz:address_id=*, BUT the data in the OSM node doesn't match LINZ's data | Anyone can use the fork of RapiD to edit this data |
3 | EXISTS_BUT_NO_LINZ_REF
|
There is no OSM address node with a matching ref:linz:address_id=*, BUT there is an OSM address node with the same data, just no ref:linz:address_id=* | |
4 | MULTIPLE_EXIST_BUT_NO_LINZ_REF
|
There are multiple address nodes for this address, none of which have a ref:linz:address_id=* | |
5 | MULTIPLE_EXIST
|
There are two different nodes in OSM with the same ref:linz:address_id=* | Requires manual action for complex situations |
6 | EXISTS_BUT_LOCATION_WRONG
|
The address exists in OSM, with the ref:linz:address_id=* tag but the coördinates are totally wrong (>500metres away from the gazetted location) | Anyone can use the fork of RapiD to add, move, or delete this data |
7 | TOTALLY_MISSING
|
Address doesn't exist in OSM at all | |
8 | NEEDS_DELETE
|
The address was imported from LINZ, but since then LINZ has deleted the address (see #Address Deletion) | |
9 | NEEDS_DELETE_NON_TRIVIAL
|
The address was imported from LINZ, but since then LINZ has deleted the address. However, an OSM mapper has added business/POI information to the address node (e.g. name=*, shop=*, craft=*). See #Address Deletion. | Requires manual action |
10 | CORRUPT
|
The address in OSM contains multiple ref:linz:address_id=*, separated by a semicolon. | Anyone can use the fork of RapiD to edit this data |
11 | LINZ_REF_CHANGED
|
LINZ has changed the value of ref:linz:address_id=*. Effectively a delete and a re-create with a new ID. | |
12 | obsolete | ||
13 | COULD_BE_STACKED
|
Addresses in OSM that are PERFECT , but the flats/units could be stacked if this is desired. Stacked means the many individual nodes for each flat in an apartment complex are replaced with a single node for the whole apartment complex. More Info.
|
none, just for statistics |
14 | NEEDS_DELETE_ON_BUILDING
|
Similar to #9. The address was imported from LINZ, but since then LINZ has deleted the address. However, an OSM mapper has merged the address node into a building. We won't delete the building, but we will remove the address tags from it. See #Address Deletion. | Anyone can use the fork of RapiD to edit this data |
15 | REPLACED_BY_BUILDING
|
The address exists twice in OSM: once on an imported node, and once on a building. We will merge the node into the building. This is usually caused by StreetComplete users. |
Tagging
The tagging scheme used is the same as the original 2017 import:
LINZ Field Name | OSM Tag |
---|---|
address_id
|
ref:linz:address_id=* |
full_address_number
|
addr:housenumber=* and alt_addr:housenumber=* if the property has an 'Alternate Address' |
full_road_name
|
addr:street=* |
suburb_locality
|
addr:suburb=* if urban, addr:hamlet=* if rural[1] |
town_city
|
addr:city=* if urban, does not exist for rural addresses |
water_name
|
If present, addr:type=water will be added. |
Other tags:
- building:flats=* - added if multiple flats are stacked[2] together.
- check_date=* - added by the editor if an importer ignores the suggestions from the conflation service. Addresses with a
check_date
will be ignored by the conflation process if the date is less than a year ago. - alt_addr:housenumber=* - added to addresses that have an "alternative address", for example if
123A
and1/123
are both valid house numbers for the same place. - linz:stack=no - mappers can temporarily add this tag to a stacked[2] address to tell the system to delete the stacked node and replace it with individual nodes for each appartment. This will happen next time the sync runs.
Potential issues
This table will be updated as the project progresses.
Issue | Mitigation |
---|---|
Duplicate data being imported | The fork of RapiD has an added feature to prevent duplicate addresses being imported based on the ref:linz:address_id=* tag. This conflation happens in real-time, in the browser |
Multiple people editing the same dataset at the same time | Users will be presented with a warning if someone else is/was editing that dataset in the last hour |
Address Deletion
There are cases when an address was imported into OSM from LINZ, and then deleted by LINZ. Part of this proposal is to delete previously imported addresses in OSM that have been deleted by LINZ. This is possible using the fork of RapiD.
Example: In the following image, address 14 Subdivision Road has been deleted because it no longer exists. Five new properties now occupy the former lot including a new road.
Out of scope
- Dealing with duplicate addresses in OSM. If one address has a ref:linz:address_id=*, this project will normally ignore any other identical addresses. There is one exception where this is automatically fixed, see
REPLACED_BY_BUILDING
above.
Discussion with the community
This import was first proposed via an email to the talk-nz mailing list and another email to the imports mailing list. The feedback was entirely positive.
During testing, a single node a single node was uploaded to OSM and then deleted.
See Also
- Main wiki page for imports from LINZ
- Background information about NZ addresses:
- Previous address import: Wiki Page and Code (link no longer works)
Process Diagram
- ↑ Discussed recently on the community forum
- ↑ 2.0 2.1 See here for information about stacked addresses