Humanitarian OSM Team/Working groups/Technical/meeting 2014-08-11
Jump to navigation
Jump to search
Meeting to discuss Tech topics on Monday 11th August 2014
IRC log:
dodobas | pierregiraud: would you like to start... OSMTM2 ? | 17:02 |
---|---|---|
pierregiraud | OK | 17:02 |
harry-wood | hi all | 17:02 |
skorasaurus | hi | 17:02 |
ptressel | o/ | 17:02 |
*** wonderchook (~wondercho@c-98-246-210-103.hsd1.or.comcast.net) has joined #hot | 17:02 | |
pierregiraud | so since WG meeting the V2 has been deployed on production | 17:02 |
pierregiraud | with V1 data migration | 17:03 |
pierregiraud | it went more or less well | 17:03 |
pierregiraud | but we finally got there | 17:03 |
ptressel | \o/ | 17:03 |
pierregiraud | thanks to the help from dodobas | 17:03 |
pierregiraud | I then continued on integrating new features asked by people who contracted with HOT and camptocamp | 17:04 |
dodobas | it's kinda my job, you know :) | 17:04 |
*** FredB (~makina@lan31-9-88-177-159-97.fbx.proxad.net) has joined #hot | 17:04 | |
pierregiraud | there were several releases from then | 17:04 |
pierregiraud | https://github.com/hotosm/osm-tasking-manager2/releases | 17:04 |
pierregiraud | with a lot of new features and bug fixes | 17:04 |
pierregiraud | I suppose everyone here know about this | 17:05 |
harry-wood | Well done everyone involved in that! | 17:05 |
pierregiraud | we had a critical issue last week but this seems to be fixed now | 17:06 |
pierregiraud | anyone have questions about the new features or fixed bugs? | 17:06 |
dodobas | for everyone who wants to follow or add something ... this is the agenda... https://hackpad.com/TWG-Meeting-X14-Tentative-Agenda-tWKdrLid5OU | 17:07 |
pierregiraud | if not I have some specific issues I would like to discuss a bit with you | 17:07 |
dodobas | it's an unordered list, if you were wondering :) | 17:07 |
pierregiraud | https://github.com/hotosm/osm-tasking-manager2/issues/184 | 17:07 |
pierregiraud | this issue is about osm users, their names and ids | 17:08 |
pierregiraud | in V1, the issue was that only the username was stored in the TM’s DB | 17:08 |
pierregiraud | when the user changed its name, it appeared as a new user if they logged in again in TM | 17:09 |
pierregiraud | this issue is fixed since we now store ids | 17:09 |
pierregiraud | however, if the username changes we have no way to handle it | 17:10 |
dodobas | pierregiraud: so the problem is 'how to regulary update usernams ? | 17:10 |
pierregiraud | yep | 17:10 |
pierregiraud | exactly | 17:10 |
pierregiraud | and there’s no service to get the name for a user by its id | 17:10 |
harry-wood | oh really? | 17:10 |
harry-wood | hmmm someone should totally add that | 17:11 |
skorasaurus | nothing in the OSM api? | 17:11 |
pierregiraud | the question has been asked several times | 17:11 |
harry-wood | or the API should stop letting people change usernames :-) | 17:11 |
pierregiraud | there’s a long thread about this I think | 17:12 |
pierregiraud | https://help.openstreetmap.org/questions/9181/how-to-get-the-user-id-for-a-certain-user | 17:12 |
dodobas | pierregiraud: maybe overpass api? http://wiki.openstreetmap.org/wiki/Overpass_API#User | 17:14 |
harry-wood | yeah that works: http://www.openstreetmap.org/api/0.6/user/12345 | 17:14 |
pierregiraud | howdy | 17:14 |
pierregiraud | I didn’t read my mails correctly | 17:14 |
pierregiraud | my bad | 17:14 |
pierregiraud | I got an answer from simonpoole | 17:14 |
dodobas | harry-wood: yeah | 17:15 |
pierregiraud | sorry for the noise | 17:15 |
* pierregiraud slaps himself | 17:15 | |
harry-wood | Still quite a pain you have to call this and then update the name in our DB | 17:16 |
pierregiraud | or maybe « spanks » ? | 17:16 |
pierregiraud | so now the question is, when should we do this? | 17:16 |
dodobas | pierregiraud: so... don't link directly to the user page... create a proxy page that will check the user_name and... redirect to the OSM page | 17:16 |
pierregiraud | but the user may want to see his name on the top right corner | 17:17 |
clara | not necessarily | 17:17 |
harry-wood | do when the user logs in | 17:17 |
harry-wood | if they change their username half way through a session… they can expect a broken link | 17:18 |
clara | it's more important that the user knows that they are logged in - but it doesn't need to say the name | 17:18 |
dodobas | pierregiraud: is ther something in the Oauth metadata ? | 17:18 |
pierregiraud | the user who changes his name expects a broken link | 17:18 |
pierregiraud | others don't | 17:18 |
pierregiraud | I receive oauth metadata only when the user logs in | 17:18 |
pierregiraud | but username is remember for a long time | 17:19 |
pierregiraud | users don’t have to re-log in often | 17:19 |
harry-wood | the user who changes his name …is a pain the ass. They deserve some broken links :-) | 17:19 |
skorasaurus | could a warning saying that you must log-in or reauthorize if you change your user name enough? | 17:20 |
pierregiraud | the tasking manager doesn’t know that the name has changed | 17:20 |
pierregiraud | unless the user re-logs in | 17:20 |
clara | how often do users actually change their name? | 17:20 |
harry-wood | No I think we just say that to people. If somebody complains that it's showing the wrong name, we tell them to try logging out and in again | 17:20 |
skorasaurus | ah, ok, i understand now. | 17:20 |
dodobas | then just show uid :) | 17:21 |
dodobas | problem sloved :) | 17:21 |
pierregiraud | hum | 17:21 |
clara | no - it's very difficult to find out your uid | 17:21 |
clara | so users won't recognize themselves in the number | 17:21 |
pierregiraud | OK, I’ll think about it a bit more and come back with proposals | 17:22 |
clara | and then they get confused trying to change that to their name :) | 17:22 |
* dodobas is just kidding :) | 17:22 | |
pierregiraud | problem is half solved with the link (I didn’t pay attention to) given by simon poole | 17:23 |
pierregiraud | next point is less technical | 17:23 |
pierregiraud | but more about contributions to OSM DB and information about how the data was produced | 17:23 |
pierregiraud | https://github.com/hotosm/osm-tasking-manager2/issues/260 | 17:24 |
pierregiraud | it looks like I didn’t pay enough attention (once more) to the difference between changeset comments and « source » tag | 17:24 |
dodobas | pierregiraud: it's a long issue, can you summarize ? | 17:25 |
pierregiraud | we can leave this question to the github thread unless someone here want to give his point of view | 17:25 |
pierregiraud | ok | 17:25 |
pierregiraud | when an imagery is added automatically by the remote link in JOSM | 17:26 |
pierregiraud | a new layer is created | 17:26 |
pierregiraud | JOSM uses the layer « title » to add a source tag to elements uploaded to OSM DB | 17:26 |
pierregiraud | it’s not clear how it’s done currently | 17:27 |
pierregiraud | the user asks us to change the TM code so that the title is set properly at once | 17:27 |
harry-wood | yeah I agree that would be good | 17:27 |
pierregiraud | but it looks like there are possible caching problems | 17:27 |
pierregiraud | so currently the way to solve the issue is not well defined | 17:28 |
pierregiraud | I think we should leave this for now | 17:29 |
harry-wood | Well if the project admin sets the name of imagery to be clashing with some previous name of imagery… then there's a caching problem. Not otherwise surely? | 17:29 |
pierregiraud | people qualified for this discussion should join us on the thread on github | 17:29 |
harry-wood | yes good point :-) moving on | 17:29 |
*** cypher_ has quit (Ping timeout: 480 seconds) | 17:29 | |
pierregiraud | dodobas: l10n enhancements? | 17:30 |
pierregiraud | https://github.com/hotosm/osm-tasking-manager2/issues/260 | 17:30 |
dodobas | yeah... so l10n ... https://github.com/hotosm/osm-tasking-manager2/issues/238 | 17:31 |
dodobas | we should add proper support for plural forms | 17:31 |
*** TOGA has quit (Quit: Leaving) | 17:32 | |
dodobas | and... I'm not sure how to make JavaScript translatable... | 17:32 |
dodobas | there are different solutions... | 17:32 |
dodobas | but... not sure which are integratable with transifex | 17:32 |
dodobas | or... how to actually update/extract strings from JS files... | 17:33 |
pierregiraud | any example in mind for plural forms currently in the tasking manager? | 17:33 |
pierregiraud | for javascript I tried to the translated string in the mako templates | 17:33 |
dodobas | other thing... you mighthave noticed that we use Transifex https://www.transifex.com/projects/p/osm-tasking-manager2/ | 17:33 |
pierregiraud | to put | 17:33 |
dodobas | there are two resources... master and current | 17:33 |
dodobas | master are strings from currently developed OSMTM | 17:34 |
dodobas | current is currently deployed version | 17:34 |
dodobas | Transifex is nice... integration and everything... but :) | 17:35 |
dodobas | it's really hard (or I don't know how to) latest updates | 17:35 |
dodobas | pierregiraud: not sure... maybe there is some reference in the Pyramid docs | 17:35 |
wonderchook | dodobas: have you talked to the InaSAFE guys? They've been through the transfex set-up recently and might have suggestions | 17:36 |
dodobas | pierregiraud: plural... like this one ... https://www.transifex.com/projects/p/osm-tasking-manager2/translate/#hr/master/28587894 | 17:37 |
dodobas | ${locked} user(s) is (are) currently working on this project | 17:37 |
pierregiraud | OK | 17:37 |
dodobas | wonderchook: will do | 17:37 |
ptressel | Are the plural strings not separately translated? | 17:38 |
ptressel | (Not familiar with Transifex workflow -- we use Pootle for Sahana.) | 17:38 |
pierregiraud | this could be an option | 17:38 |
*** almccon (~almccon@173-164-158-185-SFBA.hfc.comcastbusiness.net) has joined #hot | 17:38 | |
dodobas | wonderchook: i got everything nicely integrated... but... it's hard to follow updates... other then blindly update on a regular basis, or manually read 'latest changes on transifex' | 17:38 |
dodobas | ptressel: it depends on language | 17:39 |
dodobas | english has 2 plural forms... croatian 3... | 17:39 |
wonderchook | dodobas: you mean so you know which translations are at a good point to integrate? | 17:39 |
dodobas | wonderchook: we don't have a policy for that at the moment... | 17:39 |
ptressel | dodobas, So if you're using the English string as the translation key, that doesn't provide distinct keys for Croatian... | 17:40 |
dodobas | for example... less then 85% translated ... doesn't get in | 17:40 |
wonderchook | so is the issue lack of policy? | 17:40 |
wonderchook | or is there another problem? | 17:40 |
dodobas | ptressel: it's not like that... | 17:40 |
dodobas | the system will 'present' a user with a different input form for plural forms... | 17:40 |
dodobas | as long as the source files are prepared to 'render' plural forms | 17:41 |
ptressel | What is "system" here? | 17:41 |
ptressel | Transifex? | 17:41 |
dodobas | ptressel: any that can read PO files | 17:41 |
dodobas | so we need to prapare the sources... 'mark' strings which are plural.. and everything else should be 'automatic' | 17:42 |
ptressel | Not sure trying to pluralize on the fly is really doable. | 17:42 |
dodobas | wonderchook: well, at the moment I've been adding new languages to the osmtm... | 17:42 |
dodobas | and we should think about a policy to control when a language gets added to the osmtm... and when a language gets removed from osmtm | 17:43 |
wonderchook | have you looked at other project policies? how much needs to be translated to be added or removed? | 17:44 |
wonderchook | I think the translation needs to be enough to be useful | 17:44 |
wonderchook | but not sure what that means | 17:44 |
dodobas | ptressel: it was doable at the time PO format was introduced to Unix :) | 17:44 |
ptressel | Ha. | 17:44 |
ptressel | 70s | 17:44 |
dodobas | wonderchook: so enough is ... 45% ? 85% ? major strings translated ? :) | 17:45 |
wonderchook | I think something 50 or 60% | 17:45 |
wonderchook | but that is just me guessing | 17:45 |
dodobas | ptressel: i think it's just a change from 'gettext' to 'ngettext' or something similar ... | 17:46 |
ptressel | Yesbut...that just doesn't work. | 17:46 |
ptressel | In most languages, the change to plural affects many words in the sentence. | 17:47 |
ptressel | There are two common solutions: | 17:47 |
dodobas | wonderchook: qgis had removed all languages below 30%, for 2.0 relese | 17:47 |
dodobas | *release | 17:47 |
dodobas | ptressel: that's why translators need to update all strings... even plural :) | 17:48 |
ptressel | One is, give the translator an indefinite form "There is (are) %n user(s) on the system" and let them come up with an equivalent indefinite form | 17:48 |
ptressel | Second is, give the translator both a singular and plural form | 17:48 |
pierregiraud | shouldn’t we first find an example of implementations in pyramid | 17:49 |
ptressel | That is, the translator gets the entire phrase, not word by word. | 17:49 |
pierregiraud | and/or check if transifex will help us | 17:49 |
dodobas | pierregiraud: they use ngettext... | 17:49 |
dodobas | it's the only way... | 17:49 |
ptressel | ? | 17:49 |
ptressel | Well, we don't do it for Sahana, so I'd contend it's not the only way... | 17:50 |
dodobas | indefinite forms sound really 'idiotic' in Croatian language | 17:50 |
ptressel | See above -- two options. | 17:50 |
dodobas | ptressel: yes yes | 17:51 |
dodobas | i don't like the first one... and think we should do the second one... | 17:51 |
ptressel | We do the second -- generate translation requests for both a singular sentence and a plural sentence | 17:51 |
dodobas | let ths system handle everythin, just be smart when to use gettext and when ngettext | 17:51 |
ptressel | ngettext isn't either of the two options I mentioned, so that's a third. | 17:52 |
ptressel | I don't think it works linguistically though. | 17:52 |
dodobas | it's up to the translator to do the work | 17:53 |
clara | maybe list the different options and discuss it in the issue queue for a bit - if you need to ask somebody else anyway? | 17:53 |
*** marc1909 has quit (Quit: marc1909) | 17:53 | |
dodobas | clara: we are only discussing... we might move this to the mailing list | 17:54 |
ptressel | ;-) | 17:54 |
dodobas | present different options... and then decide... | 17:54 |
dodobas | or not decide | 17:54 |
pierregiraud | would it possible to do POC? | 17:55 |
dodobas | pierregiraud: yeah... | 17:55 |
dodobas | I'll prepare something on master OK ? frst thing tommorrow morning | 17:56 |
dodobas | anything else regarding osmtmv2 ? | 17:56 |
dodobas | or can we move on ? | 17:56 |
pierregiraud | we can move on | 17:57 |
pierregiraud | I thin | 17:57 |
pierregiraud | think | 17:57 |
clara | website? | 17:57 |
dodobas | clara: sure... | 17:57 |
clara | I started with the bi-monthly report about what has been done and what the priorities, and linked it from a new section on the wiki https://wiki.openstreetmap.org/wiki/Humanitarian_OSM_Team/Working_groups/Technical#Reports | 17:57 |
dodobas | and then wonderchook ... OAM | 17:57 |
clara | Mikel ask for displaying profiles and work groups on the website, so I had a go at it. For now just with the obvious info OSM account, twitter, countries, short description, picture, projects and work groups they are involved it. Plus his/her latest updates. | 17:57 |
clara | It looks like this for now: http://hot.ifrik.org/users/mikel | 17:58 |
clara | Same for work groups: that's currently only description, links and members: http://hot.ifrik.org/technical_working_group | 17:58 |
clara | Next step is to ask each work group if they require anything else. Any ideas can be added to the issue: https://github.com/hotosm/hotosm/issues/65 | 17:58 |
clara | ... end | 17:58 |
clara | any questions? | 17:58 |
wonderchook | great, with the profiles will they link from projects and blog posts as well? | 17:58 |
clara | yes | 17:58 |
dodobas | so... we should all write blog posts ?!?! | 17:58 |
dodobas | :) | 17:59 |
clara | they are linked to the user name - even for users that are not active | 17:59 |
clara | and they can be added as references to projects and working groups | 17:59 |
wonderchook | great, I think one thing which is more a communications thing versus the tech side will be to think about the overall organization of the headers on the website | 18:00 |
clara | my next step would be to ask the other work groups whether there are any specific fields they require | 18:00 |
wonderchook | meaning would the working groups really be a top level | 18:01 |
wonderchook | but that isn't so much tech as figuring out how to arrange things | 18:01 |
*** cquest_ has quit (Remote host closed the connection) | 18:01 | |
clara | yes - and that can be easily changed | 18:01 |
dodobas | clara: it's good that you have don't a project manager report... now we have a template to work with :) | 18:01 |
clara | :-) | 18:02 |
dodobas | anything else for the website? | 18:02 |
clara | not from me | 18:02 |
pierregiraud | I should be able to work on mine soon | 18:02 |
clara | pierregiraud: what do you mean? | 18:03 |
pierregiraud | project manager report | 18:03 |
clara | k | 18:03 |
clara | next point then? :) | 18:04 |
wonderchook | OAM? | 18:05 |
dodobas | wonderchook: yes | 18:05 |
wonderchook | okay so there is this website: http://hotosm.github.io/OpenAerialMap/ with the documentation that dodobas put together | 18:05 |
wonderchook | which will continue to be filled out. | 18:05 |
wonderchook | the idea is to have a HOT Tech Challenge for the catalogue part of OAM soon | 18:05 |
wonderchook | we created a Google group specific for OAM: https://groups.google.com/a/hotosm.org/forum/#!forum/openaerialmap | 18:05 |
wonderchook | it seemed the HOT mailing list is probably too noisy for people just interested in OAM | 18:06 |
wonderchook | one discussion topic is at what point is the minimum viable product for delivery of OAM at the end of the grant | 18:06 |
wonderchook | or rather "what features are needed for it to be useful" | 18:06 |
wonderchook | did I miss anything dodobas? | 18:07 |
dodobas | WE NEED HELP ! | 18:07 |
dodobas | :) | 18:07 |
wonderchook | ha, yes we want to recruit more tech people to help | 18:07 |
wonderchook | part of that is the HOT tech challenge | 18:07 |
ptressel | Well, we have a bunch of interested folks in Seattle... | 18:08 |
wonderchook | but also having the mailing list to point interested parties at, so if you know anyone that would be interested in helping please refer them to myself or the mailing list | 18:08 |
ptressel | Have been trying to entice in Michael Patrick. | 18:08 |
*** harry-wood has quit (Quit: Leaving.) | 18:08 | |
dodobas | if you notice there is a new way to contribute... https://hotosm.github.io/OpenAerialMap/#how-to-contribute | 18:09 |
wonderchook | oh awesome I missed that part | 18:09 |
dodobas | once you create an account on http://annotateit.org/ | 18:10 |
dodobas | you can select any text on the page and add a comment | 18:10 |
wonderchook | I will be do a BOF at FOSS4G and we'll have some community online meetings again soon | 18:10 |
wonderchook | but yes at the moment it is mostly dodobas with some help from me | 18:10 |
clara | do you wont to put something about OAM on the website as well? | 18:10 |
dodobas | it's supported by ... http://annotatorjs.org/ | 18:11 |
wonderchook | hi clara, I should make it a project page | 18:11 |
wonderchook | so yes! thanks for the reminder | 18:11 |
dodobas | so... we can even create a HOT instance.. and use it for 'something' | 18:11 |
*** jgc (~jgc@rainbowwarrior.torservers.net) has joined #hot | 18:12 | |
dodobas | anything else... | 18:13 |
ptressel | Say we have a bunch of info on UAV imagery georeferencing...where would we put that? | 18:13 |
dodobas | ptressel: on a OAM Node... :) | 18:13 |
ptressel | Meaning? | 18:14 |
dodobas | at the moment, we don't exactly know what is an OAM Node... so... | 18:14 |
ptressel | This is just info | 18:14 |
dodobas | OAM will also host documentation and best-practice guidelines for processing imagery... and whatnot | 18:15 |
ptressel | Appears we can't look at the sub-pages without an annotateit account? | 18:15 |
dodobas | but we don't know how will it manifest it's self in the OAM | 18:15 |
ptressel | What I mean is, is there a place to put it now? | 18:15 |
ptressel | This is info about how to do it. | 18:16 |
ptressel | It's not a functioning system. | 18:16 |
dodobas | now... I don't know | 18:16 |
wonderchook | there is no specific place to put imagery right now because OAM doesn't exist | 18:17 |
wonderchook | there are all the other ad hoc existing ways to host imagery | 18:17 |
wonderchook | if the issue is there is a current need with specific imagery | 18:17 |
ptressel | Not imagery | 18:17 |
dodobas | or any info on how to georeference UAV... | 18:17 |
ptressel | Info about how to georeference UAV imagery. | 18:17 |
wonderchook | UAV pictures are imagery | 18:17 |
dodobas | ptressel: what subpages ? | 18:18 |
wonderchook | there are ways to georeference UAV imagery, but at the moment this is out of scope of the conversation of where we are with OAM | 18:18 |
ptressel | "What we have here is a failure to communicate." | 18:18 |
wonderchook | if you have a specific UAV imagery needs how about we discuss after the meeting | 18:18 |
ptressel | Poke any of the nav links on the left of http://hotosm.github.io/OpenAerialMap/oam-components/oam-p/ | 18:18 |
ptressel | [11:17] ptressel "What we have here is a failure to communicate." | 18:18 |
ptressel | What we have is *info* about *how to* georeference UAV imagery. | 18:19 |
* clara needs to be off now - bye! | 18:19 | |
dodobas | bye clara... | 18:19 |
dodobas | ptressel: so let me summarize | 18:19 |
pierregiraud | ptressel: Guns | 18:19 |
dodobas | there no specific place to put info about processing UAV... at the moment | 18:19 |
dodobas | however it should be a part of OAM | 18:20 |
dodobas | we are trying to figure out, and propose technical solutions for OAM | 18:20 |
dodobas | and if you have any comments, please share | 18:21 |
* pierregiraud has to go too | 18:21 | |
ptressel | Yes, and we have folks who work with lidar data professionally. | 18:21 |
ptressel | The SFM work was done at the U of Washington. | 18:21 |
dodobas | I have a short sysadmin report... | 18:21 |
ptressel | But going back to the github.io page... | 18:22 |
* pierregiraud stays a bit more | 18:22 | |
dodobas | everything is more or less fine... | 18:22 |
dodobas | other then blackout on 3rd, August | 18:22 |
pierregiraud | hosting problems? | 18:23 |
ptressel | If I poke any of the nav links, like OAM Processing, I get a warning flash that says could not get auth from annotateit | 18:23 |
ptressel | And the page just contains Lorem ipsum -- is that what is actually there? | 18:23 |
dodobas | DC19 lost voltage, and crashed everything.... UPS did not kick in... | 18:23 |
dodobas | anyway... our server was quickly powered up... as some servers took a few hours to power on | 18:24 |
*** hjart (~hjart@185.37.86.110) has joined #hot | 18:24 | |
dodobas | just a sec... brb | 18:24 |
dodobas | thats the second outage in 12months... | 18:26 |
dodobas | which is not to bad | 18:26 |
wonderchook | dodobas: at some point we want a back-up system, right? | 18:26 |
dodobas | server is not under any heavy load... there are more then enough resources | 18:27 |
dodobas | wonderchook: yes | 18:27 |
dodobas | that's something I NEED to do | 18:27 |
ptressel | What database is it using? and can that db do journalling? | 18:28 |
dodobas | the idea is to preapre a Amazon/DigitalOcean image which can be quickly powered on | 18:28 |
wonderchook | dodobas: I think we have a couple different options. If we wanted to buy a server we could probably get free hosting for it. Though there is a certain advantage of not doing it that way | 18:28 |
wonderchook | but yes the amazon option is also good :) | 18:28 |
dodobas | ptressel: well, mysql and postgresql and sqlite ... :) | 18:29 |
ptressel | Hmm... | 18:29 |
dodobas | ptressel: but HA fail over systems ... are not an option | 18:29 |
ptressel | Not what I meant... | 18:29 |
wonderchook | I think the other thing with the systems is there are some that are far more important than others. e.g. the tasking manager is really key | 18:29 |
ptressel | Journalling writes out the transaction before it performs it. | 18:29 |
ptressel | So if the system crashes during transaction processing, it can roll back and replay the transaction | 18:30 |
dodobas | wonderchook: yeah, that is critical | 18:30 |
dodobas | wonderchook: I don't think we need a full failover setup... just something that can be 'started' up with the latest daily backup | 18:31 |
wonderchook | yeah, that is sufficent | 18:32 |
dodobas | so yeah, I'll focus on that... | 18:32 |
*** jgc has quit (Ping timeout: 480 seconds) | 18:32 | |
ptressel | http://www.postgresql.org/docs/9.1/static/wal-intro.html | 18:32 |
ptressel | Looks like postgresql can do journalling. | 18:33 |
dodobas | ptressel: yeah, but if the server crahes... and it can't be recoverd... for some reason... journaling will not help | 18:33 |
ptressel | Depends... | 18:34 |
ptressel | Do you mean a disk crash? | 18:34 |
ptressel | Or power fail? | 18:34 |
dodobas | a meteor burns half of a datacenter | 18:35 |
dodobas | or a mouse eats through a network cable | 18:35 |
ptressel | Those are different cases... | 18:36 |
ptressel | The distinction is, did you lose the local storage. | 18:36 |
ptressel | To protect against losing local storage, can remote mirror / sync. | 18:36 |
*** skorasaurus2 (~skors@192.171.20.163) has joined #hot | 18:36 | |
dodobas | it's an outage, and the server is not reachable, we need to have a working tasking maanger instance | 18:37 |
*** TOGA (~TOGA@p4FE95D24.dip0.t-ipconnect.de) has joined #hot | 18:37 | |
dodobas | you don't know when the issue is going to be fixed | 18:37 |
ptressel | Yes, and you can bring it up using the mirrored db. | 18:37 |
ptressel | The server config is relatively static -- the data isn't | 18:38 |
dodobas | that's why I said... HA Failover are not an option | 18:39 |
dodobas | we only want to mitigate the issue | 18:39 |
ptressel | https://wiki.postgresql.org/wiki/Streaming_Replication | 18:39 |
ptressel | We're failing to communicate. ;-) | 18:40 |
dodobas | and we don't are about 12h of lost data | 18:40 |
dodobas | *care | 18:40 |
ptressel | The standard way to deal with this is database replication. | 18:40 |
dodobas | ptressel: i know all of that... | 18:40 |
ptressel | Ok... | 18:40 |
dodobas | i've done log shipping and multi slave setup... | 18:41 |
ptressel | So, the problem is...? | 18:41 |
dodobas | there is no problem | 18:41 |
ptressel | Ok, nm. | 18:42 |
ptressel | Sounded like there was something you wanted to do re outages. | 18:42 |
dodobas | we are ... | 18:42 |
dodobas | there is going to be a 'stand-in' system that can be quickly powered up... using latest daily database backup :) | 18:43 |
ptressel | Ah, so you don't want any *suggestions* about what to do about outages... | 18:43 |
ptressel | Ok, got it. | 18:43 |
*** skorasaurus has quit (Ping timeout: 480 seconds) | 18:43 | |
dodobas | if there is nothing else to add... | 18:44 |
dodobas | i think we can end this TWG meeting | 18:45 |
dodobas | ill do the usual summary ... | 18:45 |
dodobas | oh... I forgot... | 18:45 |
dodobas | we finally have IRC chat logging... | 18:45 |
dodobas | ptressel: tnx for that | 18:45 |
dodobas | http://logs.sahanafoundation.org/hotosm | 18:45 |
wonderchook | should we tell people we are logging when we join the channel? is there a protocol about that? | 18:45 |
dodobas | it in the topic | 18:46 |
wonderchook | ok | 18:46 |
bdon | why not use amazon RDS for the postgis database? | 18:46 |
dodobas | bdon: it too expensive | 18:46 |
ptressel | How much does it cost? | 18:46 |
dodobas | *it's | 18:46 |
dodobas | https://calculator.s3.amazonaws.com/index.html | 18:47 |
bdon | maybe they'd be willing to donate? HOT is a registered nonprofit right? | 18:47 |
dodobas | so the TWG meeting has ended... | 18:47 |
wonderchook | bdon: yes HOT is a non-profit | 18:48 |
wonderchook | and tax exempt | 18:48 |
dodobas | bdon: there is no point to use RDS, unless you host everything else on Amazon | 18:49 |
bdon | fair enough, out of the box high availability seems like a big win though | 18:50 |
dodobas | to host everything we would need about 5Gb of memory... let me work that out... | 18:50 |
dodobas | so.. we need an RDS pg + RDS mysql | 18:52 |
dodobas | + 3 small instances.. | 18:52 |
dodobas | + 1 micro for testing | 18:52 |
dodobas | so.. without any data trafic... and stored instances... it's about 115$ per month | 18:54 |
dodobas | but.. im not sure if that's wnough | 18:54 |
dodobas | at the moment we pay less than 100$ for a nice server that has ~10% utilization | 18:55 |
dodobas | it's a SINGLE serverthat had 2 (couple of hours) outages in 12 months | 18:55 |
dodobas | i do understand that it's not all about money... | 18:56 |
wonderchook | we used to host everything on Amazon, we are saving money now though :) | 18:56 |
dodobas | ptressel: ahh, yes some pages are just Lorem Ipsum pages... sorry for the confusion | 19:01 |
ptressel | kk | 19:02 |