Vermont Town Boundary Import 2018

Status

Status: Automated import canceled. Manual process completed.

2018-11-05 Update: After some more testing of the import process I've found it difficult to properly conflate lines and ensure that no connections are broken. Rather than a mass-import I am now going through and adding/realigning boundaries town-by town with manual connection to existing ways and manual creation of the boundary relations. So far, copy-pasting the boundary lines town-by-town is proving to make more progress than an all-at-once import. Adamfranco (talk)

2018-12-06 Update: I've now completed realignment or addition using a manual process for Franklin, Lamoille, Chittenden, Washington, Addison, Rutland, Bennington, Windham, and Windsor counties. Each boundary way is individually conflated with the VCGI's data and its geometry replaced where appropriate. In cases where there are boundaries from neighboring states/provinces intersecting, the source line gets split appropriately so that the non-Vermont boundaries do not change their position. Each Vermont town and all intersecting town, county, state and country relations are then verified to avoid gaps that would break these adjoining relations. In most cases I'm detaching the boundaries from existing waterways and other features, though I have left a few ways shared with the Green Mountain National forest and a stream as the boundary between South Burlington and Williston. Still remaining to do are Orange, Caledonia, Essex, Orleans, and Grand Isle counties. Adamfranco (talk) 16:18, 6 December 2018 (UTC)

2018-12-14 Update: I've completed adding Town/City/Village boundaries for the remaining counties. This project is now complete. Adamfranco (talk) 20:18, 14 December 2018 (UTC)

Background

This page documents a proposed import of Vermont town boundaries. As can be seen from this OSM screen-shot, only a portion of the state of Vermont has Town boundaries:

Boundary data is available from the State of Vermont with a usage grant for import into OSM in VCGI's VT Boundaries - all lines data set.

As described in United_States_admin_level Vermont Towns, Cities, and Gores encompass the entire area of the state should have admin-level 8. Additionally, incorporated villages are embedded within Towns with admin-level 9. Village polygons are available in the VT Boundaries - village polygons data-set.

Object and Tagging details

Care will be taken to ensure that each town, village, and county exists as a a relation, with the following tags:

type=boundary
boundary=administrative
admin_level=[6,8,9] (county, town/city/gore, incorporated village)
name=*

The member ways will each be given the following tags:

boundary=administrative
admin_level=[2,4,6,8,9] - (national, state, county, town/city/gore, incorporated village) The way will have the lowest-number admin_level of the relations it is a part of.

People/Accounts

This import is being prepared by Adam Franco, a mapper from Middlebury, VT. A separate AdamFranco-VT-Towns account will be used for this import.

Process

0. Advance clean-up

Prior to this import, clean-up of CDPs mis-tagged as administrative boundaries was done to remove invalid boundaries.

0.1. Search for Town and Incorporated village boundaries via OSM Turbo: http://overpass-turbo.eu/s/AC1

Query:

/*
This has been generated by the overpass-turbo wizard.
The original search was:
“place=village and boundary=administrative and admin_level=8 and is_in:state_code=VT”
*/
[out:json][timeout:25];
// gather results
(
  area[admin_level=4]["name"="Vermont"][boundary=administrative]->.boundaryarea;
  // query part for: “place=village and boundary=administrative and admin_level=8 and "is_in:state_code"=VT”
  way["boundary"="administrative"]["admin_level"="8"](area.boundaryarea);
  way["boundary"="administrative"]["admin_level"="9"](area.boundaryarea);
  relation["boundary"="administrative"]["admin_level"="8"](area.boundaryarea);
  relation["boundary"="administrative"]["admin_level"="9"](area.boundaryarea);
  
);
// print results
out meta;
>;
out meta qt;

0.2. Validate towns

Visually compare the extent of towns to the VT Boundaries - all lines shapefile as well as the list of incorporated villages from the VT Secretary of State's office. For boundaries around villages check if these are truly incorporated villages or are maybe Census Designated Places (CDPs). The term "CDP" may be in the tag data or the boundary just might not match an actual town or incorporated villages.

0.3. Change to boundary=census where needed

If boundary is actually a CDP boundary rather than a town boundary, change the boundary=administrative tag to boundary=census and remove the admin_level=* tag. Example: Middlebury CDP, changeset

1. Data sources

Loading OSM Data

The OSM data is first loaded into JOSM via an OverPass Query to select all current boundary ways and relations for Vermont, (as well as a few relations joining its boarders:

/*
This has been generated by the overpass-turbo wizard.
The original search was:
“place=village and boundary=administrative and admin_level=8 and is_in:state_code=VT”
*/
[out:xml][timeout:25];
// gather results
(
  area[admin_level=4]["name"="Vermont"][boundary=administrative]->.boundaryarea;
  // query part for: “place=village and boundary=administrative and admin_level=8 and "is_in:state_code"=VT”
  way["boundary"="administrative"]["admin_level"="6"](area.boundaryarea);
  way["boundary"="administrative"]["admin_level"="8"](area.boundaryarea);
  way["boundary"="administrative"]["admin_level"="9"](area.boundaryarea);
  relation["boundary"="administrative"]["admin_level"="6"](area.boundaryarea);
  relation["boundary"="administrative"]["admin_level"="8"](area.boundaryarea);
  relation["boundary"="administrative"]["admin_level"="9"](area.boundaryarea);
  
);
// print results
out meta;
>;
out meta qt;

Once opened in JOSM, all relations are selected and "Download members" is performed to ensure that ways not directly tagged with boundary=administrative are also loaded.

Loading Reference data

The VT Boundaries - all lines shape

2. Data cleanup

OSM data tweaks

Split existing county boundaries

Many of the existing boundary ways are not split on town-borders, but rather include multiple town borders that make up a previously imported county boundary. By overlaying both data-sets in JOSM, it is easy to see these overly-long boundaries and split them at the town boundary:

Disconnect waterways

There are several cases where town boundaries follow waterways and the way also has a water=stream or water=river tag. In these cases, the existing ways will be left as only water features and new ways will be created for the boundary.

Reference data tweaks

Joining segments

The ways in the shape-file mostly run the entire length of town boundaries, however some town-boundaries are made up of separate segments, each with different tags with things like differing data-sources for each segment. These will be joined into a single way so that its geometry can replace the one in OSM.

Splitting for external/other boundaries

Along the state boarder, the ways that make up the shared state-boundary, county-boundary, and town-boundary also serve as town/county boundaries for neighboring states. In these locations, individual nodes are added at the adjoining boundary position and the reference way is split.

Here is an example on the Vermont/New York border:

3. Conflation

Currently exploring the use of the JOSM Conflation Plugin and the JOSM utils2 Replace Geometry features for updating/replacing ways....

State and national boundaries

As the goal of this import is focused on the internal boundaries of Vermont, where possible existing national and state boundaries will be left with their geometry in place. In general the VCGI data is within 10 meters of the OSM data on the state borders, so there is little advantage to changing the border geometry.

Exceptions:

1. Near Poultney, VT and Hampton, NY, the Vermont data has the state border following the course of the Poultney River until east of Poutlney, wOhile OSM has the border leaving he river north of Poultney and beginning a straight cross-country line.

OSM (orange) vs VCGI data (black)

The state border in this location will need to be verified via additional sources.

2. The western bank of the Connecticut river seems to define the border between Vermont and New Hampshire. The VCGI data set generally follows this shoreline more closely than the current OSM border. This border will be updated to reflect the VCGI data.

4. Final data preparation

Manually creating relations for missing towns...

admin_level tweaks to new and existing boundary ways...

Validation....

...

5. Upload

...

Previous efforts by others

A previous import was discussed in 2012 at http://lists.openstreetmap.org/pipermail/imports/2012-August/001473.html .