User:EdLoach

About Me

I support OSM by being a member of the OpenStreetMap Foundation. Are you? EdLoach (on osm, edits, contrib, heatmap, chngset com.) This user submits data to OpenStreetMap under the name EdLoach. This user hails from Europe This user hails from the United Kingdom This user hails from England This user is from Essex This user prefers Opera. EdLoach prefers to go outside and map. EdLoach uses a Locosys GT-31 and is willing to answer questions on it from OSM users. EdLoach uses a Garmin eTrex 30 and is willing to answer questions on it from OSM users. EdLoach uses a Blackvue DR550GW-2CH and is willing to answer questions on it from OSM users. EdLoach uses a computer running Windows. This user joined in with Big baseball project 2011! EdLoach joined in Operation Cowboy 23-25.11.2012 EdLoach commits code to OpenStreetMap SVN under the name EdLoach.

About me

Based in Clacton, I am using a Locosys Genie NaviGPS GT-31 and JOSM to try and add a lot more detail to the Tendring Peninsula (and Clacton in particular). My first edit (on 7th August 2008) was before getting a GPS device and involved correcting the spelling of the name of a road I knew well. The second was when I realised each section of the way needed the spelling correcting.

In my first year of mapping I helped map almost all of the roads in the Tendring district of Essex and made a start on walking all of the public footpaths, which may take many many months as they take that much longer than driving the same distance.

Having got a new phone which runs Windows Mobile towards the end of my first mapping year I downloaded OSMTracker and having mastered Custom Waypoints for adding custom text as a waypoint, have now started using that to note house numbers while out walking, although I still try and track with the GT-31 at the same time as the reception seems better. Also makes noting bus stops easy (though I often also add a custom text waypoint for any name or route references, or just may take a photo).

In addition to the Tendring area, I also try and map a bit more of Wolverhampton each time I visit relatives there.

I also mapped quite a bit of east Playa Blanca on foot and added detail to the town centre when on holiday there in July 2009. There is a bit of the town centre I realised I missed after processing all my gpx files on the return to the UK - zoom to the town centre and just west of the beach there are steps going up to nowhere and a couple of ways that are marked heading towards the beach which just end. There is another pedestrian way or two in the gap between them.

Bus Routes

This page may be relevant.#

I developed an application that helped me compare the TNDS opendata from their ftp site with what is in Openstreetmap. Initially it ignored timetable data until I realised it was including route variants which had finished, or routes where every possible data was listed as an exclusion, so it made some effort to try and evaluate which route variants run at least once within the next 30 days. Development on that github version has now stopped. Before it stopped I had been maintaining all bus routes in Essex and had updated the ones in Wolverhampton (where I used to live).

I am now using a new version which originally was to help teach me MVVM development which we were trying at work. Unfortunately it uses a base class library developed at work which I can't put on github, so am using an internal git server for the source. Currently it still uses the TNDS opendata from the ftp site, but is using the GTFS data from this site. The ftp site allows me to only download the data if anything has changed which is why I've not yet changed to getting the transxchange format from the same site as the GTFS.

In recent weekends I've been trying to compare the data from both sources. The first weekend I'd downloaded the data a couple of days apart so discounted the differences as possible real changes. The next weekend both sets were missing all the First Essex routes (so I didn't check any further). The next tests had a few differences, similar to those shown at Bus Routes Opendata Comparison, and the following weekend the differences were similar.

It currently looks like I need to carry on importing both sets of data to get all bus routes. There is more work for me to do to work out why the number of route variants differs so often - I may need to do a "running at least once in the next 30 days" check using calendar_dates for gtfs.

Mapping Parties

I have attended:

• Milton Keynes
• Basildon
• Lugradio 2009 (sort of - I was mapping in the area on the Saturday, and called in at the Connaught Hotel before mapping more on Sunday)

I organised:

• Witham

Tendring Lidar Tiles

In October 2015 I made some Lidar tiles based on the Environment Agency OpenData. The notes are here.

Brewery Mapping

Not something I've done yet, but the question pops up occasionally on how to map them. So thoughts on how at User:EdLoach/Brewery.

Superceded by WikiProject UK Breweries

Armchair mapping

Firstly, I have to say I am not a great fan of armchair mapping in many cases. There is no real substitute for a survey. So many details are still lacking when only say Bing, OS OpenData StreetView and OS OpenData Locator are available. There are also arguments for and against mapping an area currently bereft of mappers; will they be drawn more by a blank canvas, or will they perhaps be drawn by improving what is already there, perhaps initially via reporting issues in MapDust/Skobbler for example. Who knows? It is probably different for everyone. Having said all that I have done a bit of armchair mapping, usually of areas I have at least visited at some point in the past, and details are below.

County of Herefordshire, July 2011

After working on North Shropshire (below), Herefordshire was looking like it needed some work (and wasn't getting any as at 18/07/2011 - details from OSM Analysis):

Rank                        404
Area                      County Of Herefordshire	
Roads in OS Locator       2,685
Road missing from OSM     1,497
Percentage Complete          43.76
Missing Change Last Day       0
Missing Change Last Week      0	
Missing Change Last 30 Days -60

The same ground rules were used as for North Shropshire, with the following added during the task.

• If a road has been mapped already with a source of OS_OpenData_StreetView, and a different name, if StreetView shows no name use the OS_OpenData_Locator one. The existing name is usually that of a nearby way, so either a copy and paste error, or a guess.

It was noted that a number of ways looked to be phonetically similar to that in Locator, but without a survey it isn't clear whether the mapped spelling or the Locator spelling is correct, so no changes were made. It is possible a mapper was using voice annotation of their trace, so had no note of the actual spellings, but it is equally possible that any transcription error was at the OS.

Once the area had been covered that leaves us with the following to survey to "complete" the county:

• Roads that have a name in OSM different from OS Locator
• Roads with different names in OS Locator from OS StreetView
• Roads which could do with a proper survey to confirm their names tagged source:name=OS_OpenData_StreetView

There are a lot of ways in Herefordshire that have no names in OS Locator. Many were added while adding nearby missing named roads. Some do have names in StreetView funnily enough. A further pass of the county was made using MapDust which (amongst other items) located an entire unnamed business park which was still missing after all the OS Locator boxes on the ITO layer had been reviewed. This could lead to a category of:

• Unnamed roads to see if they are actually named.

OSM Analysis Summary after the above exercise:

Rank                           132
Area                         County Of Herefordshire	
Roads in OS Locator          2,685
Road missing from OSM          141
Percentage Complete             94.26
Missing Change Last Day        -37
Missing Change Last Week      -384	
Missing Change Last 30 Days -1,395

Having now armchair mapped two areas it seems that named road coverage can generally be taken to about 90% using this method, and road coverage itself will be even higher. Clearing the existing MapDust reports (by checking, resolving and closing them) will hopefully over time reveal reports of where names are incorrect or missing (if they don't get surveyed by an OSMer sooner). MapDust reports I couldn't close include things like missing speed limits (a future major mapping task) and missing postcodes (when someone presumably was searching for a postcode as destination).

North Shropshire, July 2011

North Shropshire was languishing low down the tables on ITO's OSM Analysis, and as a regular visitor to the county in my youth I thought I'd see what I could do, and ended up adding over 500 road names from ITO's OSM Locator tiles. I set myself some ground rules:

• If a road is already named, don't touch it unless it is an obvious typo (e.g. Cresent for Crescent).
• If a name is added to any roads, add source:name=OS_OpenData_Locator
• It is OK to realign npe traced roads (of which there were many) based on Bing and Streetview which generally corresponded.
• If a road is added add source=bing or source=OS_OpenData_StreetView as appropriate. (In a few cases I added source=bing;OS_OpenData_StreetView
• If a road name is in the latest OS Locator information but not in the currently available Streetview background tiles or visible in Bing, don't add the road. Guessing the course of the way can't be guaranteed to be correct, or even fairly close.

This leaves about 100 ways which need a survey; to find the course of any way that isn't in Bing/StreetView, or ways that need checking as they are already named with something different from the OS Locator name. It also allows names added from OS Locator to be easily highlighted using the source:name tag (key) in much the same way as noname ways have been in the past, allowing surveys once there are keen mappers in the area. It also leaves North Shropshire with many missing ways now added, and a number of nearly joined ways correctly linked so available for routing. OSM Analysis name completion has gone up by over 50% (although survey's are still really required to confirm whether OS Locator names are accurate).

Operation Cowboy 2012

I took part in Operation Cowboy and worked mainly on slice 132 of this cake. As background imagery in JOSM I had the WMS link to show the cake slice boundaries, Bing imagery, and OSM US TIGER 2012 Roads Overlay. I did download GPX data for the area but there was very little. I chose this area as my sister used to live there (still rents out the house, hoping to move back at some point), and we visited for a week a couple of years ago now. I had already added her street and the local schools where my nephews went.

The USGS data for this area seems to be complete rubbish. None of the areas remotely lined up with what was visible in Bing. In many cases the landuse indicated one thing but it was clear it had since been developed into housing (or industrial use, etc). So I deleted quite a bit, so it was easier to start tracing back *real* landuse without getting confused by what was already there. This is an example of an import that was clearly well intentioned, and even makes the map look pretty at first glance, but has so obviously been done without careful review to realise just how wrong much of the data was. And when I say wrong, it even made the imported TIGER data for this area look not quite so bad, and that is saying something.

Anyway, the weekend was clearly insufficient time to do this tiny cake slice, and I will continue with it as time permits until I've TIGER reviewed the whole area. I'm not guaranteeing to get the landuse all correct; that really needs local surveys to determine what is farmland and what is just big gardens, for example. So in many cases I won't even take a guess.

Tendring District and Bing Imagery, November 2010-

As I type this, the news has only just been announced. Below are what I think I might be able to use the imagery for best locally.

Buildings

If resolution is sufficiently good, improve building outlines over those available in OS Opendata StreetView

Landuse

This area currently has very poor Yahoo imagery coverage. It seems likely that Bing imagery should be good enough to determine farmland and possibly even distinguish field boundaries hedge vs fence vs ditch vs whatever. Field boundaries, combined with the existing 90%+ complete footpath coverage should bring the level of detail up to something ramblers can use (along with existing complete road data and other existing data).

Power Lines

Again, dependent on resolution available, it looks like power lines and pylons can be confirmed. Some of the minor lines not available in the OS opendata might also be visible (at least in combination with road/footpath surveys and extrapolation).

Tendring District and OS OpenData, April 2010-

Buildings

I have started tracing building outlines in the district, but try and restrict the tracing to areas where I hope to be able to go and survey the house numbers. Occasionally I find house numbers not visible from the road (due to parked vehicles, failing light and small numbers, or whatever) and use addr:housenumber=? to allow a later XAPI query to find where needs resurveying.

To date most of the surveying has been done in north Clacton, Little Clacton and Aingers Green, with some tracing in Bradfield and Brightlingsea for visits where I didn't get the hoped for mapping time.

Coastlines

I have converted the vector map district information for coastlines to WGS84 projection using a corrected .prj file and an import/export via Merkaartor. I have used this to enhance the coastline of the district. All coastlines for the Tendring district have now been updated in this way. In combination with the StreetView slippymap background the Mean High Water is being used as the coastline, and the Mean Low Water ways are used to help define tidal wetland multipolygons.

Power lines

These were extracted from the Settlement_Line VectorMapDistrict information. A pylon was assumed at each gap in the cables. No assumptions were made at the ends so these should be surveyed to check. The ones at the nearest substation to me go from the last pylons to the substation.

Water features

Lakes, reservoirs and the like were added from the information in the NaturalFeature_Line data. Where there was an existing stream at lower resolution, usually NPE traced, already in OSM I realigned that along the middle of the areas of water that were added.

Woodland

The NPE woods I added ages ago I removed and added the woods that are available in the NaturalFeature_Area data.

Administrative Boundaries

I added these for Tendring to parish level some time ago now, based on NPE parish boundaries and estimates of changes based on road signs. This was pretty close to the boundaries in the AdministrativeBoundary VectorMapDistrict stuff, but sufficiently wrong and lacking in detail for me to systematically replace all the boundaries with the more accurate information. The VectorMap stuff doesn't have a coast side boundary for parishes/counties but having discussed this with other mappers it is usually the case that the boundary follows the low water mark, so I have now used this rather than have a separate boundary way. Changes have been made from where the East of England boundary leaves Mersea to where the Suffolk border rounds the tip of land at the end of the Stour estuary. My biggest surprise was that Elmstead parish has a coastline; I thought Alresford shared a boundary with Wivenhoe.

Duplicate node clearing February/March 2010

After help clearing the UK of duplicate nodes thanks to Matt's excellent tool here:

http://matt.dev.openstreetmap.org/dupe_nodes/

I decided to just have a quick look at where my sister used to live in Effingham County, Georgia, USA. The whole state was red with duplicates. Very red. At first I was examining and merging them individually, but after some discussion on the #osm irc channel I got into a more systematic approach at clearing the main cuplrit in the state, the USGS-LULC land use import.

Adjacent landuse areas were using duplicate nodes on top of each other. If there was a hole in an area then a multipolygon existed with no tags on the inner way, but then two or three (usually) landuses inside that hole.

So the solution? JOSM with the validator plugin. At first I was using the remote control plugin to download areas, but more efficiently it was suggested by SK53 (who has some tips on cleaning up areas that have been subject to an NHD upload here User:SK53/NHD_Upload) that the JOSM tile downloader would be more efficient. So I worked out roughly that at z12 the range of tiles I want to cover to cover the state is 1074-1129 x and 1622-1687 y. I download an area of 4x3 tiles (e.g. 1074-1077x1676-1678) and use select to select the nodes that are likely to need validating. That query is made up as follows:

• Nodes that are part of the landuse import: child(source=USGS-LULC)
• Nodes that are part of the inner role untagged ways: child(type:way tags:0 child(type:relation))

That latter one, finds ways with no tags that are in a relation, and then selects the nodes in those ways.

Both can be joined into a single query using | (in build 3037 OR seems to be faulty):

• child(type:way tags:0 child(type:relation))|child(source=USGS-LULC)

Another useful search in some areas is for orphaned nodes:

• type:node tags:0 -child

Or nodes with no tags that aren't in ways

A variant on that one were *old* orphaned nodes:

• type:node tags:1 created_by=JOSM -child

I think you can work that one out...

Update

Things slowed down once I started hitting areas of Georgia which also had been subject to NHD or LandPro08 imports; this meant downloading and checking smaller areas, as well as using a couple more queries. The four I'm running are:

(type:node tags:0 -child)|(type:node tags:1 created_by=* -child)
child(type:way tags:0 child(type:relation))|child(type:way (attribution=NHD|source=NHD) -man_made=* -waterway=canal)
child(type:way tags:0 child(type:relation))|child(source=USGS-LULC)
child(type:way tags:0 child(type:relation))|child(source=LandPro08)

Haiti Boundaries 2010

I got a bit involved with the Haiti mapping in early 2010. Initially I was tracing detail from the public domain CIA map tiles (rivers/streams/peaks/place names sort of thing) and later got involved with trying to tidy up the administrative boundaries and assigning the correct pcode to each relation.

• Wiki page(s) documenting boundaries - http://wiki.openstreetmap.org/wiki/WikiProject_Haiti/Status/Boundaries
• Flickr Set containing OSM miscellany including Kosmos renderings of the boundaries - http://www.flickr.com/photos/edloach/sets/72157623182538309/

The latest image was created with my Kosmos boundary rules (link lower down this page), and the Geofabrik Haiti extract. Previous images had used a XAPI request to get all relations with boundary=administrative (or perhaps type=boundary, I forget now) as a tag.

GB boundaries validation

Update: I now only run the "New version" - see below

I downloaded the Great Britain pbf download from Geofabrik and used osmosis to convert to .osm as I couldn't get the bzip2 support working in Boundaries.pl.

osmosis --read-pbf file=great_britain.osm.pbf --write-xml file=gb.osm

I then ran gb.osm through Boundaries.pl using the command

perl boundaries.pl -in=gb.osm -html=gb.htm -csv=gb.csv

This failed as gb.osm seemed to exclude ways shared with an adjacent extract, in particular some that form part of the Scilly Isles relation, and some along the border of Great Britain (extract, technically United Kingdom boundary) and Ireland. So I started again with the British Isles extract.

osmosis --read-pbf file=british_isles.osm.pbf --write-xml file=british_isles.osm
perl boundaries.pl -in=british_isles.osm -html=boundaries.htm -csv=boundaries.csv

Note: I have explicitly specified filenames for html and csv in the line above, even though these are the default values. They are not those I actually used (test.htm and test.csv if you were really interested...).

The resulting files are available here (html) and here (csv). They should be updated each morning between 6am and 7am UK local time if I've left the computer on. Update: Routine now starts at 12:25 and takes just under an hour. While the .pbf is timestamped before 3am at Geofabrik, it doesn't upload to their ftp site until something after 9am (UK local time).

New version

I have written an analysis tool which generated data in a format which made filtering easier for me to handle when displaying it here.

Correcting the boundaries

In many cases the boundary ways exist already but are either in the wrong relation or just missing, or roles use is inconsistent. These boundaries are fairly easy to fix. In some cases not all the boundary ways exist yet, in which case the best source (as at time of writing) to add the missing sections from is OS OpenData BoundaryLine (also available in VectorMapDistrict).

The following notes are based on parish_region.shp from the May 2011 OS OpenData BoundaryLine release. They should apply for other types of boundaries too, but haven't yet been tested. Thanks to User:SK53 and User:chillly for their previously documented hints.

• Tweak the parish_region.prj file as described here. Only needs doing before first use.
• In QGIS (on Windows in my case) select Layer, Add Vector Layer and open parish_region.shp
• Select Layer, Open Attribute Table
• Search to find the relevant parish; may require filters on both NAME and FILENAME (for county name) if parish name is non-unique.
• Save Selection As Vector File, specifying WGS84 as projection.
• On an Ubuntu VM run Ogr2osm for the saved selection.
• In JOSM open the generated .osm file
• Download the relevant area (perhaps more of an issue for large well-mapped areas, but not encountered that yet) as new layer.
• Carefully select the missing bits from the parish layer and copy them to the downloaded layer. Add them to relevant relations.
• When happy everything is correct, upload changes. Else exit and discard and try again.

Getting a development copy of Potlatch 2 onto Windows 7 (64-bit)

Requirements

• git - I already had msysgit and TortoiseGIT installed
• Adobe Open Source Flex SDK 3.5 - I downloaded the zip, unzipped it and moved it to c:\flex_sdk
• ant - I downloaded the zip, unzipped it and moved it to c:\ant (purely to keep the paths shorter, as I'm a lazy typer).
• Set ANT_PATH and JAVA_PATH environment variables. Ensure %ANT_PATH%\bin and the path to GIT are in the %PATH% environment variable. I hadn't changed Windows 7 environment variables before, and found this article of use. Note: Java Path is not the path to java.exe under the Windows folder. It is expected that java.exe will be in the bin subfolder of the JAVA_PATH folder. In my case I have JAVA_PATH=C:\Program Files (x86)\Java\jre6
• WinKey-R (for "Run" dialog)
• cmd (press Enter or click OK - takes you to your user folder)
• git clone git://git.openstreetmap.org/potlatch2.git
• cd potlatch2
• ant

This should compile a potlatch2.swf file in resources subfolder. See the "Running" section in the readme.txt for options to launch. I ran potlatch2.html, right-clicked on the Potlatch 2 area (which stopped at 100%) and set global security settings to allow access to the resources folder. There are probably better ways, but this way looked easiest...

This looks at the dev server for testing. If you haven't already signed up on dev, when you first click Save you will have to first signup and login, then authorize the application before you can save anything.

Rendering with Mapnik for a Linux novice

Switch2OSM

With the launch of Switch2OSM.org I thought I would try these instructions. I made a new VMWare minimal install of Ubuntu 10.04 with the mini.iso mentioned in notes above, this time with 1 processor 4GB RAM and 20GB Hard Drive. I hope that's enough for just Essex. I also followed these instructions to install the VMware tools, though had to replace every occurrence of /media/cdrom by just /media to get it to run.

Note 1

At the bit where it says

...
CREATE FUNCTION
COMMIT
...
DROP FUNCTION

I just got

...
CREATE FUNCTION
COMMIT

Note 2

In Installing osm2pgsql I needed to add a mkdir ~/src before cd ~/src (Instructions updated)

Note 3

In Installing osm2pgsql I got errors creating a directory at the make install step, permission denied, so have tried sudo make install which seems better. (Instructions updated)

Note 4

After python replies with >>>> press Ctrl-D to exit.

Note 5

./configure for mod_tile fails when checking for ICU >= 4.4

Thanks to apmon on #osm I edited configure.ac to check for 4.2 not 4.4, then reran autogen and configure steps again

Note 6

get-coastline.sh should be get-coastlines.sh (Instructions updated)

Maperitive Rule pages

Maperitive rules to highlight maxspeeds (mph) - colour scheme 1

Maperitive rules to highlight maxspeeds (mph) - colour scheme 2

Maperitive rules to show nodes/areas with housenumbers but no postcodes - red is node or area with addr:housenumber or addr:housename but no addr:postcode, and green is the same but with addr:postcode. Highways as thin black lines.

I have a few other rules that I've not uploaded here, such as two mono-ish sets of rules for beer festival routes (Clacton and Dudley Canal Trust). Contact me if interested.

Using Maperitive Tiles

Sample HTML for displaying Maperitive generated tiles locally using Windows 7

JOSM Map Paint Style pages

Map paint style to colour addressed properties (number or name) drawn as areas in different colours depending whether they have a postcode or not

Essex towns template

{{User:EdLoach/Essex}} (uses {{User:EdLoach/EssexTowns}})

Wiki development/tests

First stab at Unit Types for Map Features

Test page 1

Test Page 2

Esperanto Lëtzebuergesch

Test of tabbed map features concept

2024

Other

Using JOSM for QA

I wanted a way to find houses that I've added a house number to but haven't added to an associatedStreet relation to identify the related streetname. I was hoping to create a JOSM mapCSS mappaint style, but couldn't work out whether it was possible. Nor did it seem to be in Maperitive. However, sometimes the simplest solutions are the best, and a quick use of the JOSM search facility proffered the following method.

• replace selection

"addr:housenumber" OR "addr:housename"

• remove from selection

child(type=associatedStreet)

Contributions

http://hdyc.neis-one.org/?EdLoach

Using QGIS and OS OpenData to determine postcodes in the Tendring District of Essex

Click for fullsize image

In 2010 the Tendring and Colchester CAMRA branches decided to revise their shared border such that the Tendring CAMRA branch area became the same as that of the Tendring District of Essex. As part of this CAMRA HQ needed to know which postcodes would be affected. I was asked to assist in determining this and thought it easiest to try and use OS OpenData to determine which postcodes were within Tendring, and in the case of areas which straddle the boundaries which weren't. I am aware (and passed on) that midpoints of postcodes that fall near the boundary may apply to a number of properties, some of which are in Tendring district and some are in Colchester borough, but for those pubs nearest the boundary the process below had them all in the correct district.

The steps below may not be optimal, but they worked for me when I didn't really know how to use QGIS and just used likely looking menu options as I went along.

• Start QGIS
• Add Vector Layer in QGIS, selecting district_borough_unitary_region from the OS BoundaryLine dataset
• Select the layer and toggle editing
• Open Attributes Panel
• Click on the Tendring District
• Invert Selection
• Delete Selection
• Save Layer as Tendring_District shapefile
• Close QGIS

• Extract CO.csv from the Codepoint Open data
• Use text editor of choice (e.g. JEdit) to remove CO1-6, CO9-10 to make file smaller and task easier NB: It was later found that three postcodes in CO4 9 fall into Tendring district, so best to leave CO4, or perhaps all postcodes, in the file at this step
• Add top row

Postcode,A,B,C,D,E,F,G,H,I,X,Y,J,K,L,M,N,O,P
(in 2017 this was Postcode,A,X,Y,B,C,D,E,F,G)

• Note the X and the Y are the columns QGIS needs for X and Y references, and thus identifies them correctly.
• Save CSV as a new name (e.g. COamended.csv)

• Start QGIS
• Click the Add Delimited Text Layer button (,,,) on the toolbar (EPSG:27700)
• Open your COamended.csv layer from above
• Save Layer as shapefile COamended
• Remove the CSV layer
• Add the Tendring_District shapefile from above
• Add the new COAmended shapefile from above
• Ensure COamended layer is selected, and toggle editing
• Select all points outside the boundary (perhaps a few at a time), and delete selection until only points within the boundary remain
• Save layer as Tendring_Postcodes shapefile
• Close COamended layer

If desired (and I wanted to be able to email the postcodes in Tendring to someone who does have Excel but probably wouldn't have much use for a shapefile)

• Open Tendring_Postcodes layer
• Save layer as CSV

• Close QGIS