Talk:CanVec

notes on technical work 2010

This is archived notes dumped here by User:Acrosscanadatrails 01:55, 24 April 2010‎

Needs to be developed further, list of items User:Fsteggink is working on

• canvec-to-osm Python script (will be available directly from NRCan ftp server as .osm files)
• tool to split large features in shapefile, see [1]
• tool to merge multiple OSM files TODO
• tool to deduplicate nodes in progress (JOSM is too slow)
• part of water bodies need to be converted to coastlines manually
• Tool to split large OSM files into managable 'chunks' (or working units) Further than the NTS Grid 999xx99 into 25 smaller areas (coded "a" to "y"). (use OSM splitter)

-- User:Acrosscanadatrails 01:55, 24 April 2010‎

Please note this is supposed to be a wiki talk page, not a place to keep notes. If these notes hold any meaning they should be moved to a main page, however they date back to 2010. Perhaps we could just delete this section? -- Harry Wood (talk) 09:03, 1 July 2016 (UTC)

Some old content

This is some more old content which seems to have been put here (incorrectly) by User:Acrosscanadatrails 01:55, 24 April 2010‎

Status [old]

Import Progress

old As of a week ago, the canvec data is available to use, however, the script is still at BETA 0.49 so its gonna still change :) But my manually going through it, its easy to fix up errors.

See CanVec Import Status for the progress chart.

Import Script

old See page canvec2osm for the current version (which is still in BETA phase) please comment on the talk-ca list, as questions will be answered much faster that way.

Bug Reports

old Please see canvec2osm and list errors you find as another bug# or discuss on the talk page rather than using the talk-ca list (as it's likely to get lost in the archives quickly) :)

Sample Features List

Here's the chart for the CanVec OSM Map Features showing the sample features uploaded list.

Older Status of CanVec Map Features Upload Samples

(The the wiki-charts might be out-dated and should be removed) --acrosscanadatrails 04:05, 20 November 2009 (UTC)

Voting

old, chart missing, is in Google Docs Included in the chart below, are 2 extra columns. Please sign your name on the notes, and explain why you approved/disproved with it. Also indicate indicate approved or disapproved in the column.

The purpose of voting AND explaining your position, is to make this import the 'most right'. If you have further comments, please use the talk page. Once similar answered are revielved, we can break it down to defult answers for others to choose from.

Map Features

CanVec - Feature Catalogue, Edition 1.0.2 (PDF version, 1,123 KB) This document describes all of the CanVec's features and their attributes. More ifnormation can be found at the [Canvec/OSM Map Features|OSM Map Features] page.

Garmin IMG tiles

I have converted 4 of the 6 sample canvec.osm.zip files into a routable Garmin IMG tile.

011d12 http://www.mediafire.com/file/4myjw4oinmd/50110412.img

022c07 http://www.mediafire.com/file/vnyweihfgnm/50220307.img

031d08 http://www.mediafire.com/file/ymdzjot5lyz/50310408.img

082h04 http://www.mediafire.com/file/kztvllnjwf3/50820804.img

All of the files (for all data) is available in the "NTS Grid (all data)" folder http://www.mediafire.com/acrosscanadatrails

http://www.mediafire.com/?sharekey=3b30da6df5072902ab1eab3e9fa335ca23d9631e5a05b367

-- User:Acrosscanadatrails 01:55, 24 April 2010‎

Please note this is supposed to be a wiki talk page, not a place to keep notes. If these notes hold any meaning they should be moved to a main page, however they date back to 2010. Perhaps we can just blow away this old content -- Harry Wood (talk) 09:03, 1 July 2016 (UTC)

too much wood?

The natural=wood areas in Canada look weird. Take for example this map. The first thing that strikes me is that there are *squares* of forest, which is obviously an abberation. The second thing of course is that some forest is missing. I would be quite surprised if the forest stops abrubtly like that north of Ottawa. The third thing is that there are lines crossing the forest which also seem to be an abberation. I assume that is related to the way the data is formatted, but it should be fixed.

Lastly and more importantly: the data makes national parks and such important features mostly impossible to distinguish (if they exist at all). The comparison with US data (vermont, in this case) is striking.

Oh and another thing, not sure it's related: there's this weird triangle here, and only at that zoom level...

-- TheAnarcat 22:42, 4 January 2012 (UTC)

I find this to be a serious problem as well. The good news is that it seems that the layer containing this data is separate from the data for things like parks, at least it seems that way when I watch the image draw up in the JS editor. Perhaps the results would be improved by simply not importing this data? Maury Markowitz 13:28, 16 October 2012 (BST)

Yes zoom level 8 does look like a total dog's dinner across much of Canada because of the very messy squared off areas of forest coverage. That's my impression every time I look at the map of Canada, but I was noticing it recently because someone was suggesting this image as one of the Featured image proposals

Potentially a nice topical news story for OSM, but we don't want to feature an image which shows such messy inconsistent forest data. This case also begs the question, how would anyone sell the idea of using OpenStreetMap during a wildfire emergency response (or indeed for any other purpose), when the map of forested areas (before wildfire) bares so little relation to reality?

Is that how the forest data is within the canvec data? or just how it was imported? What was the plan for data clean up after this import?

-- Harry Wood (talk) 12:11, 1 July 2016 (UTC)

Harry — there's some ‘spirited discussion’ going on about this right now on the talk-ca list after a mapper from away decided he didn't like the way Canada looked and started deleting stuff. To your questions: 1) This is how forests appear within CanVec, though some users make an effort to clean it up on import. 2) Some of the data are very old, and the forest and lake surveys don't match in age. You might have forest surveys from the 1970s mixed in with lake surveys twenty years newer. The boundaries often don't quite match. In the CanVec metadata, you can see these dates, along with an accuracy estimate (often not very; I've seen 150m applied as standard tolerance over large areas). 3) Plan for data cleanup? We'd like to have one, but there are so few mappers in Canada, and even fewer with the kind of computational power (or time, or interest) needed to clean up multi-thousand km² forest polygons each with a few thousand lakes, that we may never get there. --Scruss (talk) 12:56, 2 September 2016 (UTC)

Link for other readers: the mailing list archive of Sept 2016: https://lists.openstreetmap.org/pipermail/talk-ca/2016-September/thread.html (look for the "forest" or "canvec" subjects). --Aseerel4c26 (talk) 03:54, 20 January 2017 (UTC)

Algorithm to determine NTS grid designator from latitude and longitude

Moved to Talk:Canada NTS Tile System. -- DENelson83 (talk) 10:13, 10 February 2013 (UTC)

Fixing continuous vectors

Several helpful people have pointed me to this page, so here goes...

I have noticed that the CanVec data is split up across tiles, as described on the associated page. Unfortunately this causes problems for any features that cross the grids, as they get split up into multiple unrelated parts. I first noticed this for Mackenzie Lake, and when I expanded my view I noticed this through the entire map.

It appears that "fixing" this is a manual process of reconnecting dots. Is this really the only solution? Does the CanVec database not include some sort of ID number that we could use to automate the re-linking?

Maury Markowitz 16:57, 16 October 2012 (BST)

Yes, this is a huge problem with the existing imports, and given that the map ‘looks okay’, may never be fixed. Unless we can find a team of incredibly picky people with near infinite computing power and lots of free time … --Scruss (talk) 23:09, 15 November 2014 (UTC)

It looks less okay for these natural=glacier relations, just because in this standard style, these are rendered with an edge on the polygon, so the straight lines edges show up pretty prominently.

I'm not sure if joining together the squares is the best thing to do anyway, because they have quite a lot of intricate complexity (too much detail for any normal mapper to be maintaining actually), and so the multi-polygons will start to use a very unwieldy amount of data if they were all joined together.

The best solution (given lots of time of volunteer fixup mappers, or a very clever automated algorithm) is to join them together to eliminate the straight lines, but then find ....more logical ...places to chop up the polygons e.g. following some river course or other natural division between the chunks.

There's some examples of this in northern india actually. Some really crappy slabs of woodland (VMAP0 data I think) were dumped on the map. From the import, they are split into irregular polygon chunks (not squares, but still long crude straight lines representing nothing in reality). In many places that's still very visible. In some places mappers have worked, firstly to add much more detail to the data, but also to rationalise by re-dividing the chunks of woodland along valley bottoms (See this note for some more discussion of that)

It's a huge post-import fixup challenge you have in Canada!

-- Harry Wood (talk) 14:41, 29 September 2016 (UTC)

Too much detail!?

I'm the only one who thinks that the CanVec import sometimes contains too much details? I mean some highways have a node every 10-12 meters even if it's straight. E.g Countryview Drive http://www.openstreetmap.org/?mlat=44.70125&mlon=-63.56487#map=19/44.70125/-63.56487

Shouldn't this be simplified on a large scale? --Tylla (talk) 01:35, 9 September 2014 (UTC)

Absolutely. There is no reason to have an untagged node in the middle of a straight line. —Michael Z. 2014-09-16 22:19 z

CanVec+ vs CanVec

Seems that CanVec+, and the updated product is renamed back to CanVec … wha? Source here: http://open.canada.ca/data/en/dataset/e2c70a40-e4dc-11e3-a555-90b11c67bbbf -

   “This collection is a legacy product that is no longer supported. Users of CanVec+ (release of October 29, 2015) should plan to make the transition towards the new CanVec product.”

--Scruss (talk) 22:01, 7 May 2016 (UTC)

When I spoke with someone at nrcan(2yrs ago), they said CanVec+ was always a transintionary format and would be discontinued. That's why it was never offered in osm format and only in shapefiles James2432 (talk) 04:04, 8 May 2016 (UTC)

Tagging values

natural=landform says little

The tag natural=landform does not appear to add any information. The use of landform=* by itself should be enough for information. For rendering some equivalent might be found in the values presently rendered in the key:natural#Landform_related. These tags can be used together on the one feature. Warin61 (talk) 01:24, 8 September 2019 (UTC)

Significant inaccuracies in roads

I've been on a quest to get the roads in the area of my cottage at 45.375,-78.000 updated so that people end up within 500 m of the place instead of 15000.

There appear to be two separate sets of maps in widespread use. OSM and the Ontario government topo mapping site agree with each other closely. I assume both are using CanVec? Apple and Google agree with each other too. Those two sets do not agree with each other. Does anyone have any idea where Apple and Google get their mapping from?

In any event, the two sets are different and both wrong. Wrong to the point that we had to pull someones Jeep out of Moore Creek because Google plotted a course where there is a trail at 45.3725,-77.9954, which I'm happy to see OSM does not have. But looking around the current map in OSM, I can point out some of the issues immediately:

1. 45.3655,-78.0004 - OSM shows a looping road structure here with two branches. None of this exists.
2. 45.3817,-78.0071 - private driveway
3. 45.3644,-78.0490 - the name of this road is "South McKenzie Lake Road" NOT "McKenzie Lake Road South"
4. 45.3910,-78.0438 - the name of this road is "North McKenzie Lake Road" NOT "McKenzie Lake Road"
5. 45.3470,-78.0557 - private driveway
6. 45.3922,-78.0716 - doesn't seem to exist, but if it does its a track
7. 45.3921,-78.0784 - the name of this road is "McKenzie Lake Road" NOT "McKenzie North Road"

What do I do about this? I assume the problem is in the underlying CanVec data, assuming this IS CanVec data. Do I fix it here? Can I get CanVec fixed. Any advice? Maury Markowitz (talk) 18:00, 17 August 2020 (UTC)

First - fix it in OSM as this is fast.

1. Not found...
2. Way: 128469526 128469526 If it is private then tag access=private, driveway then tag highway=service, service=driveway ... but I'd add a house somewhere along the road? Remove the source tag, or modify it to reflect the new source.

Similarly for the others. Contact CanVic to see if they can or you can fix it there. Warin61 (talk) 01:14, 18 August 2020 (UTC)

I did contact CanVec and they got back last night. It seems they have handed off the road naming to Stats Canada for some odd reason. I have contacted them as well.

I have made several edits in this location. I simply selected the Feature Type and changed it to Driveway, which seems to fix things? Would you mind reviewing it? Maury Markowitz (talk) 14:55, 18 August 2020 (UTC)

No changes to way 128469526 since 27 jan 2013. Not spending time looking for others. Warin61 (talk) 04:22, 20 August 2020 (UTC)

How do I find 128469526? I made many changes and when I click the History button I see them all along with their checkin notes. Why do you not see these? Maury Markowitz (talk) 19:59, 20 August 2020 (UTC)

https://www.openstreetmap.org/way/128469526/history Similar can be done for any OSM object, use that web map and click on the lower right "Query Features" button, the one with the pointer and the question mark, to identify it. Then bottom left of the identified feature click on "History" Warin61 (talk) 03:56, 21 August 2020 (UTC)

I see... but I am not sure what you are saying, that one was already set to Unmaintained Track Road, so I did not change it to Driveway. I have made a number of other edits though, does this list not work for anyone other than the person that made the edits? Maury Markowitz (talk) 12:34, 21 August 2020 (UTC)

Another oddities, too many vectors?

CanVec led me to StatsCan, and from there I was able to find KML files for the national road data. Yay!

Interestingly, there seem to be a lot less vectors in that data than what I see here on OpenStreetMap. For instance, South McKenzie Lake Road in OSM is made of two parts, broken at a seemingly arbitrary point. But the original KML data has only one vector set for the entire road. Another example is the unnamed road running along the north central portion of the lake, both of which show two vector sets for a single road, but "break" at different points.

Can anyone suggest how this might be? Is this a problem that occurs between StatsCan and CanVec, or is CanVec importing incorrect? Maury Markowitz (talk) 18:02, 18 August 2020 (UTC)

Most of the CanVec import into OSM happened 10 years ago. So probably OSM reflects a state of CanVec which is 10 years more broken.--Vapor (talk) 01:20, 21 August 2020 (UTC)

@Vapor: If a new update were to occur, is there a merge process that ensures the OSM community's fixes get applied? I suspect the number of fixes in OSM is larger and better than those in Canvec. Maury Markowitz (talk) 13:13, 1 September 2020 (UTC)

Do you mean what the OSM community has improved at its CanVec? Without stable IDs at every way an automated exchange between the datasets is impossible.

And when we are unable to review more than a few percent of CanVec in 10 years, "manual backfeeding" is IMHO impossible, too.

Clearly here lies potential, currently wasted. Best wishes, --Vapor (talk) 18:58, 1 September 2020 (UTC)

Further Information on CanVec road info

So I talked to the local Ministry of Forestry (et all) and found that the road data for CanVec is supplied by them. Each province supplies their own data which is then merged into CanVec. So they have asked me to send in the change list, and hopefully, we can get all of these updated in time for the next CanVec release! Maury Markowitz (talk) 20:00, 20 August 2020 (UTC)

The data has been updated and they say it will begin to appear in various datasets within about a year. I notice that my similar edits in OSM have already appeared on 3rd party sites like CellMapper. I have also sent updates to Apple, Google and TomTom, the later being the original source of most of the errors. Maury Markowitz (talk) 13:11, 1 September 2020 (UTC)