Missing Maps Epworth Zimbabwe Field Mapping 2015 live

From OpenStreetMap Wiki
Jump to navigation Jump to search

Project Brief

Our goal is to map the suburb of Epworth in Harare, Zimbabwe, for the Missing Maps Project. Data collection for the area, including road networks, landmarks used by local people to navigate, admin areas and unofficial 'neighbourhoods' (again used by locals to navigate). To register the community, and to provide accessible records for the function of infrastructure management, water supply and sanitation needs. OSM Task: http://tasks.hotosm.org/project/868

Further, we hope to introduce the advantages of Open Street Mapping to the local community. To consolidate a team of volunteers from the local areas to carry out this task, drawn from MSF Local (National) Staff and members of the Local Administrative Board (mainly Council Engineers), and to strengthen working relationships between humanitarian and local parties.

Date March, 2015 ongoing

Project Managers Kieran O'Sullivan (Humanitarian OpenStreetMap Team), Rupert Allan (Medecins Sans Frontieres), Pete Masters (Missing Maps Project).


Field Paper Edit and Upload!

In the table below you can find the data (Field Papers, notes and gps traces) which were collected by the survey team on the ground in Epworth, Harare. The collected data contains housing blocks, identified by points of interest or community figures. For reference, there are some photos to help with contextualising the work.

The codes in the comment boxes are where the surveys have already been uploaded to Field Papers and you can simply put the code into JOSM. Some of them don't upload (we think due to low-quality scanning) - you can try to upload these but you probably won't get very far!

We need your help to digitize themǃǃ

Please put your (user)name in the table if you begin working on a group. The files are hosted on Dropbox: [[1]].

Quarter Pack & tiles surveyed User Notes
NE A (e.g. Professor Your Name) (any questions about this pack for the validators or subject matter experts)
NE B (B2) Ralph Aytoun B2 INCOMPLETE -plot numbers on plan go up to B108 but detailed survey list stops at B63. Reached plot B55 on survey list which is missing on plan so have not added house numbering or building material from here. Have added tag for brick:type=unburnt as this brick is unstable in heavy rain. I have put all tags on nodes for consistency although polygon houses are good in this area. Only B2 in this pack.
NE C (C1) Matt Witchell added C1 and C2
NE D (D1, D3) Ralph Aytoun D1 D3 Nodes Complete - Added nodes with detail, no boundaries added.
NE E (E1) Nick Allen Although the grid shows an E2, E3, E4 etc, there is only E1 in the packs
NE F (F6) Daphne Allen have started & will continue at mapathon on 21/07/15 - folder only has F6 in it. Now complete
NE G (G1) 23/07/15 - no map in folder so cannot see where to put the nodes. Leaving. Nick Allen
NE H (H8) Nick Allen 23/07/15 all complete, except have not added the field area=E but everything else okay.
NE I (I5) Nick Allen 03/08/15 attempted but given up! PLEASE REVIEW - Duplicated numbers on map & two sheets with B1 to about B13. Note: Landiwe the mapper has clearly tried to resume the mapping later. She has misunderstood the numbering system. Will look into it.
NE J (None) Nick Allen Nothing in dropbox
NE K (K8) Nick Allen Duplicated numbers and sheets making it difficult. Needs a review to work out which numbers to use.
NE L (None)
NE M (None)
NE N (N6) Ralph Aytoun 23/07/2015. Folder only has N6. N6 Complete except ID-B17 and a bit of confusion with numbers bottom right corner.
NW A (None with survey)
NW B (None with survey)
NW C (None with survey)
NW D (None with survey)
NW E (E8) Pedro E8 complete
NW F (F8) Sam F8 complete
NW G (G5, G7, G8, G9) Pedro; Thomas Orrell G5 G7 G8 G9 complete
NW H (H5-H10) Ebenemrks; Sam H5 H6 H7 H8 H10 complete -
NW I (I8, I9) Rachel; Amyn; Sam I5 I6 I8 I9 complete
NW J (J7-J10) Ebenemrks; Pedro; Dan Culli J7 J8 J9 complete J10: Failed upload to FP
NW K (K5, K6, K9) Thomas Hills COMPLETE
NW L (L6-L9) Thomas Hills COMPLETE (L6-L9) L6: vague locations (i.e. two addresses at same point on field paper). L7: Makomo as 'Area' may actually be Makomo Extension???
NW M (M4-M10) Thomas Hills COMPLETE (M4-M10)
NW N (None) Not in Dropbox folder. (T Hills: I think these are in SW, as 'N5 2' (map) and 'N5 survey' (survey))
NW O (O5, O6) These files belong to SW, not NW
SE A (None)
SE B (None)
SE C (C1-C5, C7) C1 David Ross C3 Dave Panchaud. C4 Ralph Aytoun C5 Jose Cisneros David Ross working on C2. C3: 1tg1zqsk C4: gbnfmvcj C5: done (aero97) C7: 4g6g311i C8: rlggtm71 C4: done
SE D (D1, D2, D3, D5, D6, D7) Iana (D2) D1: 5idh238u (B1-B5 Tagged, B6 remaining) D2: xqs8xbkd - tagged everything that was noted on the survey, however there are some clusters of buildings near the border of the map that were not labled or identified in the survey. D4: 4thqk76q D5: 5425pzja D7: 5jtybyy4 (B1-B5 tagged, B6-B9 not yet done, seems like a lot of buildings unlabelled).
SE E (E4-E6) Thomas Hills E5: done (20 Oct pkqk b3 and b5 have same house number?) E4: 3h8kjyf4 (done as much as possible 20 Oct mmharris) E5: 3mr9bv5b E6: 2wgk5i61
SE F (F3, F4, F5) Aga Kreglewska (F3) F3: 17lu1flw (survey mentioned ID# B2(b) and B3(b) that weren't on the map, I assumed this meant there's a second house within that ID#) F4: 1cjdqj9m (done as much as possible 20 Oct mmharris) F5: 1twacqxm
SE G (G2-G6) These surveys don't seem to be complete. Although they should be inputted, they should be re-done on the ground in Epworth.
SE H (H1, H2, H4-H7) H1, H2 rather strange so quite difficult to input but possible. H4: 59h1fvw1 H5: 2zu03hl9 H6: 2wknt501 H7: 4z3ptemt
SE I (None)
SE J (None)
SE K (None)
SE L (None)
SW A (None)
SW B (None)
SW C (C6, C7) Sam C6, C7 completed
SW D (D7) D7: Failed upload to FP
SW E (E6, E7) Mark Nortje E7 completed E6: Failed upload to FP
SW F (F7) Dan Culli F7 completed
SW G (None)
SW H (None)
SW I (I7) Johnny H I7 complete
SW J (J5-J8) Paras V J6: Survey papers 1,2 & 4 entered. Paper 3 not in drop box bundle. 5- 10 remain to be done. J5: 657zct4a /J5: Done up to B66 J6: 2625mdal J7: Failed upload to FP J8: 7fnleffw. Almost done. Just few blocks left on the bottom part of the field paper.
SW K (K5, K6, K8) Madeleine Harris & Daphne Allen K6 1st survey page completed by Madeleine. Some of K5 done by Daphne - ran out of time, will not be able to complete. K5 Failed upload to FP; K6 Failed upload to FP
SW L (L4-L7) L4 DONE. Only 1 BH (borehole?) L5 Survey.jpg has repetitive id's. Can't find corresponding block on map for some numbers. example: all id's over 100. L6 DONE. L7: Failed upload to FP
SW M (M5, M6, M7) Colin B, aero97 (M6), Ella B (M5) M7 done, M6 done, M5 done! M5: 1qto4tn2 M6: 5n99a2e2
SW N (N5, N6) Helen Jones (pattonesque) N5 done, both survey papers entered, but for the second paper, the nodes need to be matched to the buildings. N5: 1lzv1cwq N6: Failed upload to FP
SW O (O7) Fragosus O7 Completed but couple of buildings has no codes
n/a Public Water GPX tracks

How to use the table above

1. On the table, chose an empty row, in my case NE D,

2. Click on edit at the head of the Chapter heading which reads Field Paper Edit & Upload (edit)

3. Scroll down until you find your chosen row - which will actually appear something like this 

  |-
  | NE
  | D
  |
  | 
  |-

4. & add your name so it looks like this; 

  |-
  | NE
  | D
  | Nick Allen
  | 
  |-

5. Scroll to the bottom of the page & click on show preview and if it looks okay scroll to the bottom & click on save changes

6. Right click on this link from the head of the table for The files are hosted on Dropbox & chose either open link in new tab or open link in new window so you don't lose this guide.

7. On your new tab or window, choose the appropriate region (e.g. NE), scroll to the bottom of the list of field papers and then download the GPX file - in my case North East GPX grid.gpx

8. Load this .gpx file into JOSM and you will now be able to see the area zones

9. Your mileage may vary, but Field Papers do seem to work (...now, as at 2015-07-07). Not all the snapshots have been uploaded, so take a look at the links below for your area: 

   * NW: http://fieldpapers.org/atlases/xd8pbm4k
   * NE: http://fieldpapers.org/atlases/mdqsvcdn
   * SE: http://fieldpapers.org/atlases/69wkr7tq
   * SW: http://fieldpapers.org/atlases/qp36xxmz

If you need to upload one, see: https://wiki.openstreetmap.org/wiki/Field_Papers#How_to_Upload_a_Snapshot In JOSM, you can use the Field Papers extension to load this is as a layer then, by copying the snapshot URL (find this by clicking on a tile e.g. NE D1 is http://fieldpapers.org/snapshots/3tscyneb#17/-17.876540462148817/31.152562597348076 )

10. My D1.jpg has B1 written on it, and on D1 Survey.jpg I can see that B1 has house name/number, Construction, Neighbourhood/Street, Area & Ward info.

11. Create a node at the point you think B1 should be, and add the tags for the address - there is a guide under the heading Missing_Maps_Epworth_Zimbabwe_Field_Mapping_2015_live#Case_study:_mapping_a_neighbourhood

12. Now do the same for B2 etc until you have finished all the information in the dropbox folder you downloaded.

13. Finished a pack or had to stop part way through - make sure you add a comment to the table so that progress can be monitored 

  |-
  | NE
  | D
  | Nick Allen
  | Sorry, ran out of time, so only the first page of D1 has been completed.
  |-

14. Congratulate yourself on a job well done, and hurry back to do some more another time.

Workflow

Before you begin

You should have downloaded JOSM and should have it open and running.

Make sure that you have installed the remote control plugin (most new versions will have this automatically) and that it is running.

If you'll be searching doing higher-level searching and filtering rather than just mapping the field papers directly, you should have "expert mode" enabled.

Getting the data on your computer

If you have not reached this point, open Open Street Map and Search 'Epworth, Zimbabwe'. The area will correspond with Task 868 . Pick a pack from the Dropbox page . The Dropbox contains Field Papers, Surveys, Tracks and a Grid file. Field Papers are split into North East, South East, South West and North West Epworth.

You can drag and drop gpx files into your window in JOSM or ID. It is important to download the .gpx grid for whichever quarter you are in, in order to orientate yourself and find your grid square, which refers to your selected field paper. Otherwise, each pack has a Jpeg image of the grids which you should be able to correspond to your map image.

Download the contents onto your computer desktop by clicking on the Download button in the menu above the files in the packs. The grid will enable you to locate your field paper, which is marked in the top right-hand corner. There may be many tables for one Map Jpeg Field Paper. You can pick a pack which interests you.

Note that you are working on it in the table above(Noting this is an important step because it prevents others from duplicating your work, and you from getting into a mapping war with someone who is simultaneously editing the same area.) 


*** Update 23rd June ***

Use nodes within the region traced on the field papers, picking the building outlined if available. Otherwise, use a point/node for tagging. Relations at such a low level get a bit confusing, mess up the map quite a lot, and are better used at higher levels where boundaries cross into different regions. Furthermore, their boundaries are not always included in the field papers.

Boundaries/Relations We are not using relations (see below) to map Epworth, at this level.

I think the hierarchy suggested below should work for most cases.

An example address chain is:

  • name=641.
  • If there is a person's name, consent has been sought and given, and this is the equivalent of a 'community head', to be included as loc_name=Mr Cameron;
  • Ward=2;
  • addr:neighbourhood=Makomo (to be tagged in 'ways/relations' as 'Metropole' level 0);
  • addr:subdistrict=Makomo Extension (to be tagged at 'Metropole' Level 1);
  • addr:subdistrict='M' - a qualifier for the part of the neighbourhood, which is present on some field papers (to be tagged Met. Level 2);
  • addr:hamlet=Chinamo (to be tagged at Met.Level 3);
  • addr:street if mentioned;
  • 'Parcel' or Block which is the Proper Name/Number/Identity of the parcel of land;
  • building:material=brick (common values at OSM wiki building:material page)

Subdistrict is used twice but because the two types are obviously different (i.e. a word in one case and a letter in the other) and are not too common, this should not be too much of a problem. 

***ends***

Uploading to fieldpapers.org

It is very possible to use the field paper to manually draw admin boundaries according to the papers. This may be the more desirable choice: at the last mapathon, we had major trouble uploading to http://www.fieldpapers.org -- swise

Should you choose to try to use fieldpapers.org so that you can trace the boundaries more closely:

  1. Open the Field Paper website http://www.fieldpapers.org
  2. Click on "Upload pages you've photographed".
  3. Choose one of the Field Papers you just downloaded on your computer.
  4. Click "Upload" to upload the selected Field Paper. It can take a few minutes before the Field Paper is localized, but finally the Field Paper will pop up.
  5. Now you can open the selected Field Paper as a background layer in the editor you want. (In JOSM you'll need the Field Paper plugin.)

Getting set up in JOSM

Taking the pack you have ALREADY NOTED ON THE TABLE ABOVE that you're mapping. Every grid square should have a corresponding page of survey notes.

In the root Dropbox folder there is a .gpx grid file, as well as a .jpg of the grid. You can open this in JOSM to help you edit within the confines of the square you are working on (like a manual tasking manager for field paper editing). Zoom to your assigned square and download the existing OSM data. (Unclear how to download? [[2]]) If this is a very busy square and it's difficult for you to edit, look to the next section in this tutorial about overpass-turbo.eu!

You're asked to use Bing or NextView imagery, which you can enable under the "imagery" tab.

During the mapathon we noted that it is often useful to switch to wireframe viewer (control-W or command-W on Mac), that you may want to adjust the colour of ways or relations under Preferences > Main setting > Colors, and that you might find it helpful to modify the visibility with opacity if you can't see the imagery clearly enough (the button in the Layers window with the black to white gradient).

Feel free, for fun, to drag and drop some 'AV Notes' images into your JOSM window. The magic of JOSM allows you to see life on the ground in the area you are mapping. The images also contain Water Survey and borehole notes for the Epworth Water Survey Project (see Below), which needs inputting, too.

overpass-turbo.eu for busy squares

If your square is extremely full of buildings, roads, and so forth, you may find it easier to download only the boundaries of the area and work on those directly. We can use the website overpass-turbo.eu, an external site, to query the relevant tags. Because it pulls the data from OSM when you make the request, you can upload your changes to this dataset with confidence that it will not cause more clashes than if you'd downloaded from OSM directly.

To use overpass-turbo to find all of the administrative boundaries in Epworth:

  1. before anything else, create a NEW LAYER in JOSM and make sure it is active. The active layer will have a check mark next to it in the Layer window.
  2. find the area we want into the small, white search box in the top of overpass-turbo.eu: e.g. “Epworth”
  3. click on Wizard and search “boundary=administrative”, then click "build and run query!"
  4. you should see just the higher-level admin boundaries highlighted on the map
  5. click export: “load data into an OSM editor: JOSM”. Overpass-turbo always whines about wanting to repair the query - click "repair query" and it’ll download fine
  6. you should see JOSM pulling in this data into your new layer

Case study: mapping a neighbourhood

Advanced Users: Relations/Boundaries:

Ward boundaries can already be found on the Epworth Base Map, contributed by Epworth's own Missing Maps team. Neighbourhood and Area boundaries will emerge as inputting continues. At this stage, it may be useful to use 'Relations' where boundaries straddle higher boundaries (i.e. Nighbourhood spanning Ward Boundary).

Before you get into mapping boundaries between, say, different wards, ensure that you have an understanding of how Relations work to be sure that you're editing them correctly. An example of how you might map a neighbourhood is given here to give a sense of the task.

Let's say there's a neighbourhood, and at least one of its boundaries is defined by a higher-level ward boundary. You don't want to recreate this boundary, you want to use the same way that defines the ward boundary to define your neighbourhood boundary - but you also want to group it with other ways which define the neighbourhood. You can use a relation to combine all these ways together, and you can add more detail to the existing ways. Specifically, you'll want to split the way that defines the ward at the point where the neighbourhood boundary ends.

Zoom in to the way which defines the ward and find the place where the neighbourhood ends in order to split it. Click the edge and then type “p” for “sPlit”; move this new midpoint to the desired place along the ward way. If the way was a member of relations, JOSM will have updated those relations to contain the new ways; we don’t have to fix this.

Go around the perimeter of the neighbourhood, splitting ways which help to define it or else creating new ways for undefined sides. When you create a new way, hold down the "ALT" key to ensure that this is a new way, NOT a continuation to anything else. Double click to end the way.

Once all of the boundaries of the neighbourhood exist as individual segments, select all of edges which define the boundaries of the neighbourhood. You can select multiple edges by holding down "CTRL" as you click on each edge. Make sure that you HAVE NOT selected any NODES - only edges should be highlighted. Once all of the edges have been selected, click presets > relations > boundary, which will run a helpful script to help you make a relation. Set the following:

  • name = neighbourhood name from the data
  • boundary type = metropole
  • “new relation”!

When it brings up the next screen:

  • make sure all ways have “Role” set to “outer”
  • check to be sure it’s properly topologically in a loop (if not, the loop indicator next to the individual segment descriptions will turn red to show discontinuities)

The relations can be updated further, by selecting edges and then adding the selected edges to “relation” through the central area of this interface.

Editing the collected data

The field papers show where surveyors have identified individual houses, parcels, neighbourhoods, and so forth. The Epworth address format is irregular. This is an unique opportunity for you to be the first to understand how the locals use identifiers. This is BRAND NEW KNOWLEDGE, and requires YOUR interpretation.

As of a London mapathon of June 16, 2015, we have agreed upon the following convention for how different surveyed elements should be tagged, given the information in the survey field papers (in italics):

  • boundary=metropole
  • metropole_level=0-4 (parcels are 4, hamlet is 3, subdistrict is 2 , area is 1, neighbourhoods are 0)
  • name=HouseName/Number
  • Key:building:material=Construction (should be a value as specified on [[3]])
  • addr:neighbourhood=Neighbourhood/Street
  • addr:hamlet=Area
  • addr:ward=Ward

Epworth has been split into blocks, and these administrative boundaries will need to occupy the admin level key '11', with Neighbourhood being indicated by admin level key '10'. At the moment, this is the best way of inputting the information.

Prefixes/Suffixes:

Epworth's secret is in these qualifiers. Subdivisions happen in the form of Letters (e.g. 'M'/'L'), and in the form of prefixes like Dvira/Overspill, as opposed to 'Overspill'. It is currently agreed that this should be entered in the Block (11) name Admin Level.

Last check and upload

Finished editing your Field Paper? Have a last look on the data you edited.

Upload the data with credit to HOTOSM, MSF and Bing. Comment the changeset with "Epworth #Harare, #Zimbabwe, #HOT-OSM #missingmaps, Group number and source=survey"

Indicate whether you've finished the pack in the "notes" column. Any data which you cannot enter in (because you can't read it, don't understand it, etc.) should also go in the "notes" column so that the original surveyor can make sure to enter it.

... AND REPEAT!

Further notes

Help in editing the data are of course the wiki pages Map Features and WikiProject Epworth.

Mapping of areas is deliberately non-pinpointing, but construction is very important from the point of view of resilience.

Qualifiers from the right hand column are important. Often areas and wards overlap, and, like in a Venn diagram, denote an address with similar attributes, but which could be in a completely different location.

This is a way to pinpoint an address without local intrusion an in line with local consents.

Almost all wells are public, and we await a further Private Well survey layer. There is a dataset of Public Water GPX tracks and Geotagged Photographs on the Dropbox page, which need inputting, too, and can be dragged and dropped into a live JOSM Epworth window, and then manually interpreted and inputted.

Common Tags

This list is to help the remote mappers tagging of the various objects and assure a homogeneity of tagging among the contributors. See TagInfo [[4]] for more.

Highways

Points of interest - For this you only have to put points on the map!

Incomplete squares

These are surveys which have something missing or incorrect.

Quarter Pack & tiles surveyed User Notes
NE M (M6) Map but no survey
NW A (A5, A6) Map but no survey. Surnames and a bit of info on the maps. Inputtable but not mapathon-suitable. (Should we put these surnames on??)
NW B (B4-B8) Map but no survey. Surnames and a bit of info on the maps. Inputtable but not mapathon-suitable. (Should we put these surnames on??)
NW C (C4-C7) Map but no survey. Surnames and a bit of info on the maps. Inputtable but not mapathon-suitable. (Should we put these surnames on??)
NW D (D3-D7) Map but no survey. Surnames and a bit of info on the maps. Inputtable but not mapathon-suitable. (Should we put these surnames on??)
NW E (E4, E5) Map but no survey. Surnames and a bit of info on the maps. Inputtable but not mapathon-suitable. (Should we put these surnames on??)
NW G (G1) Map but no survey
NW I (I5, I6, I7) Survey but no map (I5, I6); Map but no survey (I7)
SE A (A3) Survey but no map
SE B (B4) Survey info on map
SE E (E1) Map but no survey
SE F (F1, F2) Map but no survey (F1), survey but no map (F2)
SE H (H3) Map but no survey
SW L6 Some extra surveys that don't correspond

Identified questions/issues in squares

Quarter Pack & tile User Notes/question
SW J5 Tom Hills In 'other information' there are lots of 'OW' and 'PW', occasionally as O/W or P/W. Any ideas what these could be? They are also placed as nodes on the paper map. B5-B9 missing on map.
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Missing_Maps_Epworth_Zimbabwe_Field_Mapping_2015_live&oldid=1237737"