Talk:Importing Government Data
Discuss Importing Government Data:
Rename / tidy up
I'm trying to work out where this page fits in with anything. We have some documentation on Import and Import/Guidelines. I don't see that there are many points worth making about Importing *Government* Data as opposed to importing any other data, and indeed this page makes many points which apply equally to any large data import. So the title seems wrong.
But what should the title be? In general this has the feel of a messy brainstorming page, with lots of text which doesn't really go anywhere
-- Harry Wood 20:07, 19 September 2009 (UTC)
- I wouldn't call it 'messy' but 'brainstorming' is better. :) It can all be moved to myhttp://wiki.openstreetmap.org/wiki/User:Acrosscanadatrails
As this was the thought process which led me to working on the canvec2osm script, which became a 'conversion script' rather than an 'import script' is it does the former. Where on the other hand, geobase2osm, is an 'import script' as it is designed to import the data directly to the OSM database, where canvec2osm just makes the .osm files available so that local area users can copy the features. This theme seems to be the best way to handle government data that is available.
Similarly, the Ontario Dataset (which will be imported (Erm.. converted) soon), from Land Information Ontario, has done the extra step for me, where the updates are given in .diff files where 1st there is the file which contains what features need to be removed. This can be made to an .osm file, and checked using AutoMatch, and then an add-on DIFF SHP file is made available, so that can be converted.
I think the point of that page creation was to explain how we need to treat the government data as a one-off copy and just use it to 'help improve' the OSM database, rather than trying to always keep up with the government database, as that can never happen (until the government starts using OSM as their INTERNAL basemap, and works as another user themselves)
--acrosscanadatrails 00:13, 20 September 2009 (UTC)
--acrosscanadatrails 00:21, 20 September 2009 (UTC) Its now available in my archives to it can be removed if you like :) http://docs.google.com/View?id=d2d8mrd_245fvg5jkfg
--acrosscanadatrails 18:03, 27 March 2010 (UTC) I decided to leave the 'bulk_implopping' essay for a blog. here's the version from the discussion lists
re: [Talk-GB] Announce: OpenOS (archived at) http://lists.openstreetmap.org/pipermail/talk-ca/2010-March/002440.html
... Which is an expansion of OpenOS to include the world, so this would be a place to host the entire Import catelogue (which BTW, that top header should be BOLDED a little more) IMO. http://wiki.openstreetmap.org/wiki/Import/Catalogue
From my example, I would need to be careful and only put in the data that is intended to be used. ie. if a road network is available from 3 different sources (national canvec nts size & geobase province size & provincial Ontario separate dataset) we would want to make sure that the best version is available.
Ideally, this is also a good home to store the entire CanVec/ TIGER / LINZ / OS etc..dataset (of all features). ... So then it can: 1 - be updated all the time with the latest data (as new datasets become available, that precious (or precarious) UUID is kept nice and clean) 2 - it can be converted to a WMS (of the complete set & available to anyone at any time) 3 - source shp files could be kept? (or just converted to .osm as soon as its available from the source, if not available in real-time... but ogr2osm could be programmed with a script to be run to check the source for changes & convert as changes happen from the data source (checking source file update-date & file size) 3 - and it can be all converted to .gpx (for if it isn't available as a WMS) so people can trace over the geometry (in the regular OSM environment) 4 - and have it as an (ugly) mapnik/osmarender/special_mix map tiles can be rendered and shown. .. not pritty because we dont want to encourage editing of the this database. 5 - ... and doesn't interfere with the community work. 6 - ... and doesn't require people to 'claim' areas they are working on
eg1 (as in Canada we are 'claiming NTS tiles on a Google docs chart, eg2 in Haiti the Hospital POIs were organized on a GoogleDocs chart with access by invite (which was frustrating for mappers, as it became a separate project, that always has to keep up) (ketchup and muster) ... keeping the playground separate is requested many times over. eg3 and other countries 'claim areas' using a wikitable. (and requires users to 'claim' that part of the file/file slice or geographic region) eg4 or some other like Coraine with a dedicated website hosting it all (like OpenStreetBugs). ...[1]
These these methods currently in use are fine for an overview, but not for the details as it cannot keep up, and actually destroys communities (yes destroys). Since the entire community is not on the talk@ lists, or IRC's they dont know what's going on & cant help in real-time, where as a 'OpenImportsMap' WMS Layer[2] or pulling data from a different API[3] or pulling GPX from a different API would solve this and keep the community)
.... i could continue on for days but wont. :0)
(I cc'd the imports@ list and talk-us@ list and talk-ca@ list and the New Zealand list (as they are currently in the planning stage of LINZ data. Sorry, i just cant handle another discussion list :-)
Thanks to that user for sharing the wiki http://wiki.openstreetmap.org/wiki/Importing_Government_Data page on that journal entry yesterday :-) ... i'll edit it and talk about canvec a bit there. (instead of making a new essay, i'll do that on a blog instead)
Cheers, Sam
[1] the OpenImportsMap idea is a derivative / spin-off from the Corrine import process (but on a much larger planet size scale)
[2] http://wms.openstreetmap.de/ already exists and is a holding house for the WMS layers to be used. the all could technically be combined in to 1 large WMS feed (as a few countries are now listed)
[3] the dev server is like these ones http://apis.dev.openstreetmap.org/ ... where is says 'as a data sandbox', the idea here is to designate a 'sandbox' for all of the data that is available to import. (or has already been imported & the latest data can be 'imported' into this new sandbox and replace the old sand.' ... and the 'sand' is the entire world (made up of little nodes).