Talk:Microgrants/Microgrants 2020/Proposal/Addition of USRNs and outreach to public sector

Unique import

Why this import is so unique that "investigate the best way to add Universal Street Reference Number (USRN) data to OpenStreetMap." requires 16GB RAM servers?

Is there an existing import page documenting this import? Is there a clear support for this import among OSM UK community? Mateusz Konieczny (talk) 15:46, 17 May 2020 (UTC)

16GB because that was what has been advised to us for one of the options (well 10GB is the recommended but it was suggested we use a 16GB AWS or similar server). It's an upper/worst case. If we find a cheaper solution works we will not request the full allowance. Feel free to share your view on how to do this - help is always welcome.

On documentation - nothing yet as it's still very early days. The talk-gb mailing list have started to discuss what tag should be used. As noted in the application, we need to be flexible here. The last thing we want is to be forced to push on with this project if it turns out not to be viable or supported by the community. Currently all signs are good but we shouldn't assume that. If it becomes non viable we will not take any of the money. The community will benefit from an updated Conflation wiki page and our lessons (which we'll share).

--RobJN (talk)

I would either investigate data locally, without buying hosting. Possibly investigating only a part of data in one go. As it stands 1000 euro is supposed to be spend on hosting "our toolset" without explanation what it is. Is this toolset the same thing as "A platform will be created to facilitate the community effort to incorporate this new data into OpenStreetMap"? Mateusz Konieczny (talk) 16:30, 17 May 2020 (UTC)

I also think that it would be preferable to establish consensus whatever data will be imported without pressure of "there is grant awarded to add it" or "see, it is obviously good idea - OSMF supported us by awarding us the grant" Mateusz Konieczny (talk) 16:30, 17 May 2020 (UTC)

Just to be clear, we will be playing with the data locally first. The "investigation" we want to do is to determine what is the best tool to use. We have some experience here already as we helped Transport for London to understand what options were available for their mappers to start adding their data into OSM (huge volumes of data collected via on the ground survey and 2 photos of each).

We had hoped that this set out in the proposal, but sounds like it is not clear enough. Step 1 is to complete our investigation of the tools available. We will update Conflation as this is missing a lot of relevant content. Only then, and with community support, do we move to the next stage. We believe that this next step will require a platform/tool that is hosted publicly to allow our community to be actively involved in incorporating this new data in to OpenStreetMap. The benefit is that the whole community can be involved, are reviewing the process and are learning something new (all things that align to the OSMUK LC aims). Costs are requirements are based on something like Hootenanny, however as stated, we will review the options first and pick the best option. If we can bring this in at lower cost then we will.

The point about consensus is a chicken and egg one. This dataset is big and needs a well thought out process and the right tool. Coming up with that is going to take time and hence we have asked for some financial support. We believe we have enough community support to start the process based on the discussions that have already occured on talk-gb.

If you think that our community will say "see, it is obviously good idea - OSMF supported [it]" then you clearly don't know our community :-P They will tell us when they don't think it should proceed - and that is right. The key point is that we would like some resource to get this started, we will deliver benefit at each stage, and we will stop if it becomes unviable.

--RobJN (talk) 19:02, 18 May 2020 (UTC)

OK, it makes sense! Thanks for explaining, now it makes more sense. Hopefully I was not the sole confused person. "then you clearly don't know our community :-P" - yes, I am not familiar with it - and such argument not treated seriously is a sign of healthy community. I am not endorsing for now, solely because I do not consider imports as the most valuable part of OSM mapping. But from import grant proposals this is the best one. Not sure whatever I will have time to compare proposals, I will try to do this. Maybe it is better than competition anyway. Mateusz Konieczny (talk) 20:56, 18 May 2020 (UTC)

Thoughts

It is good to increase the use of OSM by local authorities. No problem there. I like that.

My concern with this project lies in its flirtation with complex data management. It is clear is that OSM is not a GIS. And OSM is not a state entity asset management system. And nor should it be. I don't think we should be attempting to reproduce part or all of a state asset management system to make our system attractive to state entity employees. It is out of scope. It is not what we do.

I am not familiar with the UK system, but the one I am familiar with is cascaded (hierarchical). Beside a road are many sites, one of the sites has three buildings and one of the buildings has 3 lifts and the lifts have 10 doors each and everything has a URI number and a parent item and maybe some children items. Similar with roads. A road has an area and a start point and an end point and it lies on top of lots of property parcels, and there a few traffic signals and a bus stop and so on. Complex. It is good and great and desirable to link them all and to do that you need ESRI and its competitors.

For example: With roads it is going to be hard to make OSM match codes for centrelines so "our" road has the same USRN number as "their" road - the breaks must be matched and the road segments must be matched. And maybe we have two centrelines in divided roads and they have just one. Complex. Probably the road fragments in our map must all be made part of a relation that holds the USRN. Complex. And then there are errors. Governments make mistakes - lots of mistakes - I was a civil servant so I know. How do we handle that? It's hard to say without knowing how the state will handle the same problem. Complex.

How does a new mapper (or an old one) edit a road in iD and not screw up the USRN system? It is an important question.

So, I don't think you should be buying ANY hardware right now. You should be thinking about this a lot. You might want to use the grant to hire someones smarts with GIS and OSM experiences to do some learnings for you.

And then, I may be completely off track here. I hope so. | Craig.za (talk) 19:56, 17 May 2020 (UTC)

I think you are off track here. Far from a "flirtation with complex data" this is a serious attempt to include a major peice of information that will help to grow OpenStreetMap, and the collaborative data model within GB. As noted, it will be mandated that Public bodies use this attribute in future data they share or release. Wouldn't it be amazing if they could visualise their using OSM as the reference dataset thus bringing OSM to a wider audience. Wouldn't it be even better if they start contributing more to OSM safe in the knowledge that they can extra data that includes the USRN.

As per the above comments, we do not plan to rush any hardware purchase. The grant application is set up in such a way that the funds are available to us, if and when they are needed. We are "thinking about this a lot" and have the right GIS and OSM experience within our community (what you suggested is basically what the UK community has provided to Transport for London in regard to their cycle infratsructure data).

--RobJN (talk) 20:49, 18 May 2020 (UTC)