OpenAerialMap/Meeting Feb 19, 2015

Feb 19 19:00:01 Cristiano: Good morning, we're are about to start the OAM weekly meeting
Feb 19 19:00:10 Cristiano: (and good evening)
Feb 19 19:00:10 BlakeGirardot: Good morning
Feb 19 19:00:47 Cristiano: Here's the link to the agenda: https://docs.google.com/document/d/13oKAO9V0P33GW6nx3YiNu3wRxHfp24iklOu_mUHmmrQ/edit?usp=sharing
Feb 19 19:00:58 smathermather: Hi.
Feb 19 19:01:13 Cristiano: Hi smathermather!
Feb 19 19:01:30 Cristiano: Please add anything else that you would like to talk about
Feb 19 19:02:26 dodobas: I'm semi-AFK... will keep an eye on the chat... might not respond immediately
Feb 19 19:02:31 smathermather: Just reading through now.
Feb 19 19:03:24 Cristiano: Hi jj0hns0n!
Feb 19 19:03:32 jj0hns0n: hi Cristiano
Feb 19 19:03:38 Cristiano: Is wildintellect here?
Feb 19 19:04:50 Cristiano: Well, let's start. First, here's the link to the published tech challenge: http://hot.openstreetmap.org/get_involved/openaerialmap_catalog_tech_challenge
Feb 19 19:05:47 Cristiano: Please help to spread it. There's a two week period for proposals
Feb 19 19:06:21 BlakeGirardot: Did we share it on the hot and osm email lists?
Feb 19 19:06:37 smathermather: Will share.
Feb 19 19:06:55 Cristiano: Not yet, please go ahead Blake
Feb 19 19:07:00 smathermather: (in response to christiano
Feb 19 19:07:01 BlakeGirardot: Ok, will do.
Feb 19 19:07:19 Cristiano: I asked OSGeo to post on their @jobs list, but haven't received approval yet
Feb 19 19:07:41 Cristiano: but I guess we can post on @discuss if the job list is not used anymore
Feb 19 19:08:02 Cristiano: (there hasn't been any post since October)
Feb 19 19:08:52 Cristiano: Anyway, any other community of open source devs that may be interested, please circulate, thanks
Feb 19 19:09:59 Cristiano: jj0hns0n: I'm sure you'll forward to Geonode and Django's :-)
Feb 19 19:10:21 jj0hns0n: most of the geonode folks have already looked at it, not sure if there is a geodjango specific list anymore?
Feb 19 19:10:38 jj0hns0n: I guess there is, not that active
Feb 19 19:10:52 jj0hns0n: will alert the eoxserver guys
Feb 19 19:11:03 Cristiano: Cool, looking forward to their proposals!
Feb 19 19:11:49 jj0hns0n: Im curious if its intended to for only people who want to do the work to make proposals?
Feb 19 19:12:42 Cristiano: No, of course anyone's proposal is welcome. Here, through the list or GitHub
Feb 19 19:12:57 jj0hns0n: ok, sounds good
Feb 19 19:13:50 Cristiano: I would also like to get your input on how to evaluate proposals
Feb 19 19:14:16 wildintellect: technical and practical
Feb 19 19:14:33 Cristiano: I would like to make that process as open and collaborative as possible, but making sure to preserve sensitive information
Feb 19 19:14:39 jj0hns0n: yeah, 2 separate things as wildintellect mentions
Feb 19 19:14:47 wildintellect: just assign them some score on a set scale for each property
Feb 19 19:15:00 wildintellect: technical - do they have the background and skills
Feb 19 19:15:20 wildintellect: practical - work history, prev community experience, likelihood of completion
Feb 19 19:15:47 Cristiano: yes, I was not talking about the evaluation process per se, but how to form the evaluation group
Feb 19 19:15:49 wildintellect: you can put the proposals with Names redacted into a a google doc folder
Feb 19 19:15:56 jj0hns0n: and I think it makes sense to consider the techical aspects of the proposal separately
Feb 19 19:16:19 wildintellect: then it's up to you and HOT to pick a group to score each section
Feb 19 19:16:21 jj0hns0n: is it a sound implementation, is it likely to be sustainable over time
Feb 19 19:16:51 jj0hns0n: fwiw, I have used http://screendoor.dobt.co/ for a few projects to evaluate proposals
Feb 19 19:17:02 Cristiano: I would like the process to be participatory, so I will think of the best method to involve you guys
Feb 19 19:17:03 wildintellect: so I would suggest Cristiano you pick or invite people to score each section
Feb 19 19:17:32 Cristiano: That's a good idea, I will check it out
Feb 19 19:17:40 jj0hns0n: Im sure they would give us a free subscription if we want
Feb 19 19:17:50 jj0hns0n: code for america folks
Feb 19 19:17:58 wildintellect: obviously the people here are a portion of possible scorers
Feb 19 19:18:14 jj0hns0n: I guess the question is how many proposals do you expect?
Feb 19 19:18:28 wildintellect: with only 2 weeks to submit there might not be many
Feb 19 19:18:38 jj0hns0n: I realize maybe its a bit too late in the game, but its good to do a kind of "Expression of Interest" first
Feb 19 19:19:27 Cristiano: Yeah, unfortunately we needed to get things rolling, so hopefully we get enough good proposals in two weeks
Feb 19 19:19:49 Cristiano: and we may decide to extend it if we need to
Feb 19 19:20:04 jj0hns0n: yeah, makes sense to wait and see what you get
Feb 19 19:20:17 Cristiano: The application should not take long
Feb 19 19:21:05 Cristiano: 1 page for describing the proposal, 1 page resume and 1 page cover letter
Feb 19 19:21:49 Cristiano: Anyway, now unless there's any other comment about the tech challenge, should we start brainstorming the API?
Feb 19 19:23:28 Cristiano: wildintellect: please go ahead and edit the agenda with your proposal
Feb 19 19:23:57 Cristiano: or should we have a separate doc for dumping and sketching ideas?
Feb 19 19:27:06 Cristiano: wildintellect: I'm looking your proposed design in the agenda how do you define "node"?
Feb 19 19:27:07 wildintellect: its fine where it is
Feb 19 19:27:20 wildintellect: Node is a discrete computer
Feb 19 19:27:32 jj0hns0n: I would say single container whether virtualized or bare metal
Feb 19 19:27:38 wildintellect: yes container
Feb 19 19:27:39 Cristiano: in my understanding there could be catalog and processing on the same computer
Feb 19 19:27:44 wildintellect: yes
Feb 19 19:27:49 Cristiano: (for the portable options)
Feb 19 19:27:52 wildintellect: but they are independent
Feb 19 19:27:55 wildintellect: of each other
Feb 19 19:27:58 Cristiano: OK cool
Feb 19 19:28:25 Cristiano: because I feel node may be confused with instance
Feb 19 19:29:19 jj0hns0n: wildintellect "Distributed tile processing" you mean tile generation right?
Feb 19 19:29:39 wildintellect: not sure I just moved it
Feb 19 19:30:07 wildintellect: but in the old OAM there was some processing to convert images into most usable format
Feb 19 19:30:09 jj0hns0n: well, the question is more whether you would want to do actual imagery processing here or simply tile generation
Feb 19 19:30:17 wildintellect: not necessarily tile seed
Feb 19 19:30:19 jj0hns0n: ok, that would be image pre-processing
Feb 19 19:30:26 jj0hns0n: not done on the tiles but on raw imagery
Feb 19 19:30:38 Cristiano: I would not include other image processing other than reprokjecting and tiling
Feb 19 19:30:40 jj0hns0n: reprojection, stretches, clipping etc
Feb 19 19:31:06 jj0hns0n: so, Im going to edit this
Feb 19 19:31:07 wildintellect: right, the classic example is the conversion of from RGB to YC...
Feb 19 19:31:21 wildintellect: to get huge file size savings
Feb 19 19:31:27 jj0hns0n: hows that?
Feb 19 19:31:44 wildintellect: you familiar with the telascience NAIP 2009
Feb 19 19:31:46 jj0hns0n: there you go
Feb 19 19:31:48 jj0hns0n: yes :)
Feb 19 19:31:55 wildintellect: that processes and stuff like it
Feb 19 19:31:56 jj0hns0n: you should ask winkey all the stuff he had to do
Feb 19 19:32:04 jj0hns0n: yeah, I think I listed the main ones now
Feb 19 19:32:11 wildintellect: I plan to since new NAIP is out
Feb 19 19:32:25 Cristiano: any process that is not fully automatic should not be part of the processing node
Feb 19 19:32:38 Cristiano: it should be done before by the submitter
Feb 19 19:32:49 jj0hns0n: nah no way
Feb 19 19:33:02 jj0hns0n: you cannot ask them to give you files in the optimum format for tiling
Feb 19 19:33:05 wildintellect: Cristiano, we're talking about optimizations
Feb 19 19:33:07 jj0hns0n: they will give you jp2k or mrsid
Feb 19 19:33:07 smathermather: Any need / intent to handle feathering / seam matching / histogram matching... ?
Feb 19 19:33:08 jj0hns0n: et
Feb 19 19:33:09 Cristiano: ideally we only ingest ready-imagery (or almost ready) and just compress it, convert it, reproj and tile it
Feb 19 19:33:19 smathermather: (between datasets)
Feb 19 19:33:30 jj0hns0n: ready imagery can be a huge jp2k that is in a strange projection
Feb 19 19:33:45 jj0hns0n: and you still have to convert that to the most optimal format for quickly rendering tiles
Feb 19 19:33:48 Cristiano: that's fine, then it's all automatic processing to make it ready
Feb 19 19:34:04 jj0hns0n: yeah, it more or less goes in the order we have there under Uploads
Feb 19 19:34:12 Cristiano: right. I'm saying, we should not include functions that require user intervention
Feb 19 19:34:17 jj0hns0n: oh no
Feb 19 19:34:32 smathermather: There are good out of the box tools for automatic translation of format and projection -- not for all use cases / uploads, but for a lot.
Feb 19 19:34:38 jj0hns0n: whether uploading an individual file or indexing a directory, it should just 'go' from there
Feb 19 19:35:07 jj0hns0n: yeah, crschmidt figured out the really most optimal formats for rendering tiles as quickly as possible, gdal can do everything necessary
Feb 19 19:35:45 jj0hns0n: its very inefficient to render off of things like mrsid or jp2k especially if reprojecting
Feb 19 19:37:27 wildintellect: right you'd have to decompress it first
Feb 19 19:37:47 jj0hns0n: yep
Feb 19 19:37:56 jj0hns0n: so all of that stuff goes on in pre-processing
Feb 19 19:38:03 jj0hns0n: but requires no user intervention if done correctly
Feb 19 19:39:02 jj0hns0n: you should make it clear that a tile 'node' could also just be something like S3
Feb 19 19:39:14 smathermather: So if there are standards for upload format that are restrictive, I assume there will be good, easy to use tools to point people to for client side pre-processing... .
Feb 19 19:39:29 jj0hns0n: smathermather I think that would be backward though
Feb 19 19:39:49 jj0hns0n: to put the onus on the imagery submitter, better the processing node is smart and can deal with whatever is thrown at it
Feb 19 19:40:04 jj0hns0n: if its proper orthoimagery, regardless of format, projection etc
Feb 19 19:40:04 smathermather: Oh good -- agreed.
Feb 19 19:40:11 smathermather: Yup.
Feb 19 19:40:15 Cristiano: well, that could be challenging
Feb 19 19:40:20 jj0hns0n: nah, not really
Feb 19 19:40:41 Cristiano: well, we should def suggest formats for upload
Feb 19 19:40:41 wildintellect: of course we'd still make suggestions on easiest things to upload
Feb 19 19:40:42 jj0hns0n: it becomes an iterative thing ... if someone throws something at it that doesnt work, you write a test that demonstrates the failure, fix it and ask them to re-upload
Feb 19 19:41:12 Cristiano: and encourage development of plugins for other tools (e.g. QGIS, Pix4D, ODM, etc)
Feb 19 19:41:26 jj0hns0n: I think we should also be clear in the processing node that you could also just point it at a dropbox, s3 dir, ftp dir etc
Feb 19 19:41:36 wildintellect: Cristiano, sure that's the point of the api
Feb 19 19:41:39 jj0hns0n: and the processing node would slurp that or if possible access it over FTP
Feb 19 19:41:42 jj0hns0n: http I mean :)
Feb 19 19:42:06 jj0hns0n: i.e. with vsicurl
Feb 19 19:42:22 Cristiano: right, or what if we to upload ready-tiles (eg in Dropbox)? What's the best workflow there? Compress for transfer and then deflate on server side?
Feb 19 19:42:47 jj0hns0n: thats on the other side
Feb 19 19:42:56 Cristiano: assuming that the contributor already run gdal2tiles
Feb 19 19:42:58 jj0hns0n: its commonly called "clip and ship" still
Feb 19 19:43:09 jj0hns0n: ah, thats different I guess, I would not encourage that
Feb 19 19:43:19 jj0hns0n: better they give oam the raw imagery then some tiles of unknown origin
Feb 19 19:43:45 jj0hns0n: it would be rare to have someone that had tiles and not the raw imagery (which would be smaller and easier to transport)
Feb 19 19:43:56 Cristiano: Well, If someone already has a QGIS plugin to prepare and tile imager, then they could just send it up to an OAM instance along with a metadata file
Feb 19 19:43:58 jj0hns0n: Im thinking someone saying "here is my dropbox folder with a bunch of raw imagery in it"
Feb 19 19:44:14 jj0hns0n: but by definition the tiles will be much bigger than the raw imagery
Feb 19 19:44:23 wildintellect: I think the QGIS plugin would be an upload to OAM plugin
Feb 19 19:44:24 jj0hns0n: better they send raw imagery
Feb 19 19:44:50 Cristiano: well, yes, that's prob the most common situation. But would be nice if we can offload some of the processing on the client side :)
Feb 19 19:44:53 jj0hns0n: ah, so then in theory you could run the processing node _inside_ qgis and just upload
Feb 19 19:44:55 jj0hns0n: that is better
Feb 19 19:45:05 Cristiano: right
Feb 19 19:45:11 wildintellect: sure that is an option, since anyone can run a processing node
Feb 19 19:45:15 jj0hns0n: but I would not be in favor of accepting a random zip of tiles that someone made
Feb 19 19:45:29 jj0hns0n: well, one more +1 to do this in python then ... so it runs as a qgis plugin :)
Feb 19 19:45:42 smathermather: hehe.
Feb 19 19:46:22 wildintellect: if someone does use a processing node locally then there are only 2 steps after that - 1. push them to a tile node, 2. notify the Catalog once they are on the tile node
Feb 19 19:46:24 jj0hns0n: so, does the use case I mention make sense though? "here is a remote directory of raw imagery, please download it and make tiles from it"
Feb 19 19:46:30 jj0hns0n: correct wildintellect
Feb 19 19:46:31 wildintellect: yes
Feb 19 19:46:46 jj0hns0n: and again "push to tile node" could just be to S3
Feb 19 19:46:51 wildintellect: I expect plenty of people will just have a dir of images somewhere
Feb 19 19:46:57 jj0hns0n: and then you just let the catalog node know "they are here"
Feb 19 19:46:58 wildintellect: ie thats how we pull NAIP
Feb 19 19:47:02 jj0hns0n: right
Feb 19 19:47:09 jj0hns0n: thats the dream to just point it at NAIP and say "go"
Feb 19 19:47:50 Cristiano: Or the L8 on S3 CHolmes just mentioned :)
Feb 19 19:48:03 jj0hns0n: yeah, but they are going to preprocess all that
Feb 19 19:48:20 jj0hns0n: we should just be able to add that to the catalog someday
Feb 19 19:48:29 smathermather: So does OAM ever optionally return raw imagery in cases where only tiles have been uploaded?
Feb 19 19:48:42 jj0hns0n: I would say that is a totally secondary concern
Feb 19 19:49:03 jj0hns0n: I think by and large it is not the goal of OAM to ever return 'raw' imagery
Feb 19 19:49:13 jj0hns0n: right?
Feb 19 19:49:19 Cristiano: It would be nice (in the future)
Feb 19 19:49:43 Cristiano: so that users can do stuff with the imagery other than looking at it or tracing
Feb 19 19:50:02 jj0hns0n: ok, but if you _really_ want to do that kind of thing, you should just look at OMAR and OSSIM
Feb 19 19:50:18 jj0hns0n: OAM should be primarily about tiles IMO
Feb 19 19:50:27 Cristiano: I guess so :)
Feb 19 19:50:35 jj0hns0n: omar/ossim or eoxserver
Feb 19 19:50:56 jj0hns0n: that is a very different use case right?
Feb 19 19:51:25 wildintellect: L8 would still be we just want the RGB, and to process that into tiles
Feb 19 19:51:30 jj0hns0n: i.e. having access to the raw imagery is important to an analyst or scientist, and not to the ordinary user who just wants tiles
Feb 19 19:51:38 Cristiano: Yes, but it would be nice if there's a list some metadata entry so that people can eventually go find the raw data if needed
Feb 19 19:51:39 smathermather: Can those tools ingest a set of tiles and do processing on them?
Feb 19 19:51:57 wildintellect: Cristiano, sure the Catalog should say where the data came from
Feb 19 19:52:06 jj0hns0n: yeah, I would make sure that whenever possible the link to the raw imagery should be stored in the catalog, but it should not be the goal of OAM to actually serve that stuff up
Feb 19 19:52:18 jj0hns0n: yeah, both can
Feb 19 19:52:58 tomkralidis: catalogue is simply yellow pages w/ links
Feb 19 19:53:01 jj0hns0n: omar/ossim in particular is a very robust automated image processing toolchain and catalog
Feb 19 19:53:15 Cristiano: ideally the "central" OAM should be a place where people can find, view and use tiles, but also just find/link free imagery that is available anywhere in the world
Feb 19 19:53:21 jj0hns0n: right, tomkralidis what I mean is that OAM should link back to the raw imagery, but not be concerned with trying to serve it up
Feb 19 19:53:33 tomkralidis: exactly
Feb 19 19:53:33 jj0hns0n: i.e. OAM is not a WCS
Feb 19 19:53:49 jj0hns0n: if you want a WCS, use eoxserver or omar or whatever
Feb 19 19:54:18 Cristiano: Right, so that brings us to the catalog and metadata. How much PyCSW can we just use straight off?
Feb 19 19:54:27 jj0hns0n: 100% of it
Feb 19 19:54:28 jj0hns0n: :)
Feb 19 19:55:09 Cristiano: I'm not 100% familiar with it, so how do we plug it in and how do we use its API
Feb 19 19:55:16 Cristiano: ?
Feb 19 19:57:07 dodobas: pycws sohuld be just another 'endpoint'
Feb 19 19:57:15 tomkralidis: Cristiano: pycsw's a Python library that you deploy standalone CGI or mod_wsgi (latter is better), or embed into your own framework like Django, Flask, etc.
Feb 19 19:57:20 tomkralidis: like GeoNode did
Feb 19 19:57:22 jj0hns0n: yeah, it has to have a datastore too
Feb 19 19:57:33 jj0hns0n: but then yeah, you just have to stick http in front of it somehow
Feb 19 19:57:35 dodobas: that will 'publish' OAM metadata...
Feb 19 19:57:36 jj0hns0n: various ways to do that
Feb 19 19:57:45 tomkralidis: jj0hns0n: agree. Either u can use pycsw's datastore OR it can bind to an existing datastore.
Feb 19 19:57:46 dodobas: no need to integrate anything
Feb 19 19:57:49 jj0hns0n: right
Feb 19 19:57:59 jj0hns0n: in the smallest possible case it can just be a sqlite db right?
Feb 19 19:58:10 tomkralidis: jj0hns0n: yup
Feb 19 19:58:10 jj0hns0n: and uses sqlalchemy in that case? I cant remember
Feb 19 19:58:27 tomkralidis: yes, sqlalchemy, lxml, OWSLib, geolinks are the deps.
Feb 19 19:58:56 jj0hns0n: so yeah, its pretty straightforward IMO
Feb 19 19:59:05 Cristiano: PyCSW handles metadata and other catalog harvesting right? How do you search all that and use it through a Web UI? Is it straight forward or there's some other middleware?
Feb 19 19:59:05 jj0hns0n: tomkralidis is eoxserver using pycsw now too?
Feb 19 19:59:48 Cristiano: I'm saying, can we just expose PyCSW API for all catalog functions?
Feb 19 19:59:48 tomkralidis: Cristiano: yes, harvesting and transactions via HTTP. OWSLib is typically the Python based CSW client used.
Feb 19 20:00:00 jj0hns0n: Cristiano the interface is a csw which can be queried, but you have to put some UI on it, its just an http endpoint that returns json or xml
Feb 19 20:00:27 jj0hns0n: tomkralidis to do harvesting you have to spin a new process in a cron or some queue right?
Feb 19 20:00:39 tomkralidis: for webui you can put simple js together to do the ops
Feb 19 20:01:05 Cristiano: OK, cool. So there's no need of things like elasticsearch to handle searches right?
Feb 19 20:01:15 tomkralidis: jj0hns0n: to do harvesting, you harvest a resource, and set a time interval by which the harvester fetches/refrehses
Feb 19 20:01:26 tomkralidis: Cristiano: IMHO no.
Feb 19 20:01:50 jj0hns0n: tomkralidis the question there is how well does that scale?
Feb 19 20:02:10 tomkralidis: ask data.gov :)
Feb 19 20:02:10 Cristiano: Right, that was my next question :)
Feb 19 20:02:15 jj0hns0n: tomkralidis how do you daemonize the harvester or is it in a cron
Feb 19 20:02:27 jj0hns0n: in data.gov the db is in postgres?
Feb 19 20:02:41 jj0hns0n: are you using postgres full text search or something similar?
Feb 19 20:02:42 tomkralidis: jj0hns0n: cron
Feb 19 20:03:01 jj0hns0n: tomkralidis ok, but that could also be done with something like django-celery or any other queue right?
Feb 19 20:03:03 tomkralidis: jj0hns0n: data.gov is PostgreSQL + PostGIS + PostgreSQL fts, yes.
Feb 19 20:03:06 tomkralidis: jj0hns0n: sure
Feb 19 20:03:11 jj0hns0n: ok, so yeah, fts then for sure
Feb 19 20:03:25 jj0hns0n: so yeah, I would say that obviates the need for elasticsearch et al
Feb 19 20:03:31 Cristiano: and is there a way to sync two or more PyCSW on different OAM isntances?
Feb 19 20:03:46 tomkralidis: Cristiano: harvesting
Feb 19 20:04:00 Cristiano: so, they harvest each other?
Feb 19 20:04:08 jj0hns0n: yeah, its got to be the same methodology
Feb 19 20:04:19 Cristiano: I though harvesting was one way
Feb 19 20:04:32 jj0hns0n: tomkralidis how is it now handling a distributed search across several catalogs that may not ever be in sync?
Feb 19 20:05:34 tomkralidis: distributed search can be sync or async, it's a CSW thing
Feb 19 20:06:18 jj0hns0n: yeah, I guess the real question in my mind is can we do _everything_ within the formal csw protocol or is there something else necessary to make it easier for mere mortals to understand
Feb 19 20:06:26 jj0hns0n: in geonode at least, we have a separate more simple API
Feb 19 20:06:36 jj0hns0n: using tastypie
Feb 19 20:07:03 tomkralidis: IMHO it can, but people have disagreed with me on that :)
Feb 19 20:08:01 jj0hns0n: this is much easier to work with than csw :) http://demo.geonode.org/api/layers/?limit=1&offset=0
Feb 19 20:08:14 Cristiano: And the OAM metadata schema will also be somehow simpler than OGC EO
Feb 19 20:08:16 tomkralidis: jj0hns0n: FYI pycsw supports OpenSearch. Which is dead easy.
Feb 19 20:08:37 jj0hns0n: yeah, I guess it boils down to who is going to consume the API
Feb 19 20:09:00 tomkralidis: IMHO you want a cross cutting offering. for the specialists/GIS crowd CSW, for mass market OpenSearch.
Feb 19 20:09:39 jj0hns0n: yep
Feb 19 20:09:48 jj0hns0n: in geonodes case we use this other api to drive the UI in a lot of places
Feb 19 20:09:57 jj0hns0n: but it all comes from the same data source
Feb 19 20:10:41 jj0hns0n: thats the real key is binding whatever apis you are going to provide to the same datasource and not keep multiple copies of the same catalog data on the same node/instance whatever
Feb 19 20:10:46 tomkralidis: true. Single data source is important. And so is single search design pattern. You don't want someone to use CSW with diff search results than, say UI
Feb 19 20:10:59 jj0hns0n: I think we have found that not to be a real problem so far
Feb 19 20:11:00 Cristiano: The set of metadata to index will be very small compared to other catalogs with full text searches and complex spatial geometries
Feb 19 20:11:41 jj0hns0n: yeah, should be
Feb 19 20:11:52 tomkralidis: I think pycsw + sqlite3 covers the light use case
Feb 19 20:12:09 Cristiano: the only spatial data in the database will be the dataset footprint, either BBOX or polygon coverage
Feb 19 20:12:42 tomkralidis: so the other part here is the metadata model
Feb 19 20:12:59 jj0hns0n: yeah, but should we not just follow ISO completely?
Feb 19 20:13:31 nhv: maybe BBOX, and CRS
Feb 19 20:13:37 Cristiano: I would at least do a subset. And then extend with specific OAM namespaces
Feb 19 20:14:18 Cristiano: Here's the initial draft: https://github.com/hotosm/OpenAerialMap/wiki/Metadata
Feb 19 20:14:42 jj0hns0n: you should try to shove everything into an existing formal standard as much as possible
Feb 19 20:15:07 tomkralidis: Cristiano: the OAM namespaces: need to be aware that extending the metadata model with 'local' elements is fine, but then that affect queryables by standards based things like CSW.
Feb 19 20:15:21 tomkralidis: agree w/ jj0hns0n on leveraging an existing standard
Feb 19 20:15:31 jj0hns0n: ebRIM would be pushing things too far IMO
Feb 19 20:15:45 tomkralidis: that's like a sledgehammer trying to crack a nut
Feb 19 20:15:48 Cristiano: Yes, that was more an abstract level list. It would be nice to fit it to a standard :)
Feb 19 20:15:58 tomkralidis: Dublin Core is a good start.
Feb 19 20:16:13 jj0hns0n: yeah, but just going ISO is the best right?
Feb 19 20:16:19 jj0hns0n: I mean thats the tact we take in geonode for usre
Feb 19 20:16:21 jj0hns0n: sure
Feb 19 20:17:05 tomkralidis: jj0hns0n: vanilla ISO is best/safe, albeit a bit more complex than DC.
Feb 19 20:17:24 tomkralidis: like we do in GeoNode yes
Feb 19 20:17:29 jj0hns0n: yeah, but not sure you can cram everything we need in DC right?
Feb 19 20:18:15 jj0hns0n: but -0 on doing anything that extends an existing profile unless there is a very very good reason for doing so
Feb 19 20:18:40 tomkralidis: agree. vanilla ISO like we do in GeoNode would yield better results.
Feb 19 20:19:03 tomkralidis: -1 on extending, push into ISO 19115/19139 proper
Feb 19 20:19:14 jj0hns0n: yep, I would not be -1 but certainly -0
Feb 19 20:19:21 tomkralidis: based on https://github.com/hotosm/OpenAerialMap/wiki/Metadata, I don't see any gaps in vanilla ISO
Feb 19 20:19:27 jj0hns0n: I didnt either
Feb 19 20:20:12 jj0hns0n: then the real question in my mind is do you also stick another API or do everything through CSW and OpenSearch
Feb 19 20:20:29 jj0hns0n: but I think that can be thought through when building a UI and not before
Feb 19 20:20:31 tomkralidis: how important are facets here?
Feb 19 20:20:39 jj0hns0n: could potentially be very important
Feb 19 20:20:41 tomkralidis: jj0hns0n: agreed, true.
Feb 19 20:21:48 Cristiano: So, how to include e.g. datasource (raw files, WMS, TMS, MBTiles) would be defined when building the UI?
Feb 19 20:22:11 jj0hns0n: not sure I understand that question
Feb 19 20:22:49 Cristiano: or is that something that you can already define in vanilla ISO to start?
Feb 19 20:23:19 jj0hns0n: you mean how would you reference those kinds of attributes in vanilla iso?
Feb 19 20:23:24 jj0hns0n: I think thats totally doable
Feb 19 20:23:26 Cristiano: I'm just wondering if we can define the complete metadata schema before implementing the UI and the other components
Feb 19 20:23:35 Cristiano: OK
Feb 19 20:23:50 jj0hns0n: tomkralidis may get irritated at me for saying this ;) but some people are now shoving a json dict into something like a supplemental_information field
Feb 19 20:23:59 jj0hns0n: and then you can do what you like to 'extend' if you need something for the UI
Feb 19 20:24:04 jj0hns0n: without breaking ISO
Feb 19 20:26:14 nhv: mumbles something about if we also consider video STANAG 4609 maybe worth investigating Feb 19 20:25:29 jj0hns0n: nhv wouldnt we just go the ossim/omar route if we wanted to do that?
Feb 19 20:25:30 Cristiano: That's probably OAM 2 :)
Feb 19 20:25:43 nhv: that is the OMAR route  :-)
Feb 19 20:25:51 nhv: or part of it
Feb 19 20:25:52 Cristiano: live earth video feed?
Feb 19 20:26:14 nhv: yes
Feb 19 20:26:14 jj0hns0n: yeah, or doing something with all the go pro video from every DJI user out there :)
Feb 19 20:27:03 jj0hns0n: that would be something to get STANAG 4609 from a gopro video and a pixhawk log :)
Feb 19 20:27:03 Cristiano: All right, lots of good ideas, thank you guys! We should wrap up
Feb 19 20:27:34 jj0hns0n: I gotta get going too, thanks for leading cris
Feb 19 20:27:37 tomkralidis: jj0hns0n: json in supplemental_information yikes! But yes doable
Feb 19 20:27:48 Cristiano: Please send your comments and ideas to the list or open it in github
Feb 19 20:27:49 jj0hns0n: yeah, better to do that than break ISO :)
Feb 19 20:28:24 Cristiano: Thank you all and see you next week!
Feb 19 20:28:40 BlakeGirardot: Thank you all !
Feb 19 20:28:44 smathermather: thanks!