Central Virginia Import
Import In Progress
Import cleared with imports-us mailing list and individual local contributors.
First Locality to Import: Hanover County, VA
About
This page is intended to document plans for an upcoming import of data from Virginia's central GIS clearinghouse. This ODbL Compliance verified data source has been used in numerous imports in the Southeast part of Virginia. This import is designed to build on these foundations and push into central Virginia This is intended to be a craft-style import of addresses and buildings. This import is manually done and will be completed with an eye toward on-the-ground accuracy.
We plan to follow & update the plan found at import checklist, but our general high level plan is this:
- Identify data to import
- Translate, tag, and otherwise prepare the data
- Make sure the community are all on board with the plan.
- Do the work: Import/merge the data
The intent is to begin this effort in earnest in mid 2019.
Goals
The goal of this effort is to radically improve the quality of central Virginia address and building information in OpenStreetMap.
Schedule
- Planning: Mid 2019
- Import: Late 2019 Onward
- QA: During import
Import Data
Background
Address data will be derived from the Virginia Geographic Information Network. ODbL compliance has been confirmed during numerous imports in Southeast VA.
Data source site: https://vgin.maps.arcgis.com/home/index.html
ODbL Compliance verified: Yes
"As of January 1, 2015, the VGIN database is public domain. Best efforts were undertaken to ensure the correctness of Address Point data throughout the Commonwealth of Virginia, however, all warranties regarding the accuracy of the map data and any representation or inferences derived there from are hereby expressly disclaimed."
Data Files
Virginia provides its information as shapefiles.
Visual accuracy can be seen below:
Import Type
This is an one-user, one-time import.
There are currently no plans to script or automate this import.
Data Preparation
Data has been prepared using a visual data preparation tool called Alteryx (https://www.alteryx.com/) using its geospatial capabilities.
Building and address shapefiles were loaded into the tool and filtered down to a single locality.
Building data had no useful tags, thus only a building=yes was applied.
Address data includes street, housenumber, and zip code. Road names were expanded fully using a table of USPS abbreviations (nearly 600), locality custom abbreviations, and logic to ensure no errors in expansion. The case was adjusted from capitals to proper case.
The two files were then conflated using an iterative nearest neighbor approach with a single match. Maximum distances of ~0, then 25, then 50 feet were applied, which yielded the best results. The combined data was then checked against both the address file and the building file to add any unmatched items back in using indexes. This methodology will ensure easy reruns when looking at a different locality. Data was then exported as shapefiles for import into JOSM.
Tagging Plans
Tagging will be straightforward. As an example:
addr:housenumber=6004
addr:postcode=23111
addr:street=Dugout Trail
building=yes
Changeset Tags
Tag in import-related changesets:import=VGIN
References
- Local knowledge
- Bing/ESRI/DigitalGlobe aerial layer/Virginia Imagery Service (depending on the age of the data)
- Existing OSM data
- Address data import
Workflow
- A section of conflated building data will be obtained from the Bing dataset roughly equaling 1000 buildings (or 10,000 objects).
- Run validation on the existing OSM data layer that now contains new data, resolve all issues emanating from collisions between existing and new data (see merge rules).
- Do a sanity check on data: Do addresses correspond to adjacent roads? (more on imagery below)
- Merge with existing address data. Use the JOSM "m" key to merge nodes. Check for addresses in POI and merge to new address data.
- To merge with a building outline, use the Replace Geometry (Ctrl+Sift+G) by selection the existing building outline and the new address node. Note: Only merge building outline with an address node if the building outline contains just one address node.
- Upload data to OSM
Known Dataset Conflation Issues
Local knowledge
Where known, additional data will be added during the import
Dedicated Import Account
A dedicated account will be used for the import, 
Omnific_Import (on osm, edits, contrib, heatmap, chngset com.)
QA
Due to the non-automated nature of the import, QA will occur during the process. All data will be visually checked for out-of-date outlines, excess address nodes, poor quality outlines, etc.