Talk:Nominatim/Installation on Debian or Ubuntu

From OpenStreetMap Wiki
Jump to navigation Jump to search

Progress

If you're curious about the progress, execute the SQL command select pg_size_pretty(pg_relation_size('placex')); and do the same with place instead of <placex>. When loading is finished, placex will be within +/- 25% of the size of place.

Is this still true? My placex is 36% the size of place. --Brian 252 03:36, 17 July 2011 (BST)


First Installation

Import and index OSM data

First download a Planet File. Using a file in PBF format is recommended. Now start the import: ./utils/setup.php --osm-file <your planet file> --all This will take as little as an hour for a small country extract and as much as 10 days for a full-scale planet import. The import produces a lot of log messages, which you should carefully examine. If you get DEADLOCK messages at some point, then it is quite likely that an error earlier in the process was the cause.

Could you provide some info about the space requirements now and if the whole required space is in the postgresql-db or if it includes the planet-osm as it is the only clue for the progress of the import process (couldn't count all these dots and don't know how many there used to be)? An easier solution would be to provide a real progress-"bar" or just update the progress with percentage as just dots. It might be difficult to generate a progress but for a very big job like the full-planet-import it is mandatory to know how long it will take. Nick3331 16:30, 8 August 2012 (BST)

The "dotty" part is the loading of the placex table. At the time of writing, placex will take up 40GB for a full planet not counting indexes. I'll look into providing percentages for loading. Note that data loading is not the last step in the process. It normally shouldn't take longer than half a day. After loading is done, the indexing process displays an ETA but you should be aware that this is only for the current address level. There are 30 address levels in total to process and indexing it all will make up at least half of the total import time. --Lonvia 17:01, 8 August 2012 (BST)

  • Thanks for your reply, didn't think someone would answer that fast. So the dotty Part should take about half a day for the full planet, is one dot an amount of time or dataset? Because I started my import 16.07.2012 and the placex-import is working since 20.07.2012 (according to the starting-point of the postgres-processes) and still not done yet (13.08.2012). The database-size is at a size of 297GB. Could you give me some contact data of you to tell you my full specs and maybe find the problem? Nick3331 13:12, 13 August 2012 (BST)
  • The dots approximately describe the amount of time passed so far, so they are more a sign that the process is still alive than anything else. A month is definitely too long for loading. This might be because of insufficient hardware, a badly configured postgres DB or because you have actually found a bug. For these kind of installation questions, you best either ask on the geocoding mailing list or join us in the #osm-dev channel on OFTC IRC. --Lonvia 16:11, 13 August 2012 (BST)

Assumptions during setup process

 ./utils/setup.php --osm-file <your planet file> --all

I looked into the setup.php script and it assumes that:

  • Current user uses trust authentication for Postgres - line "passthru('createdb -E UTF-8 '.$aDSNInfo['database']);".
  • For the same line of code - it assumes that the current user is a Postgres superuser who can add new databases.
  • Same for line "passthru('createlang plpgsql '.$aDSNInfo['database']);"
  • osm2pgsql invocation does not pass username - also assumes the current user has trust authentication.

I added some of them to the documentation. Paweł Paprota 12:53, 19 August 2012 (BST)

PHP permission configuration

There is a problem with the pgsqlRunScriptFile PHP function - PHP in my distro (Arch Linux) by default does not seem to have enough permissions to read files... 

ppawel ~/src/osm/Nominatim $ ./utils/setup.php -v --all 
PHP Warning:  file_get_contents(): open_basedir restriction in effect. File(/proc/cpuinfo) is not within the allowed path(s): (/srv/http/:/home/:/tmp/:/usr/share/pear/) in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 55
PHP Warning:  file_get_contents(/proc/cpuinfo): failed to open stream: Operation not permitted in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 55
WARNING: resetting threads to 1
PHP Warning:  file_get_contents(): open_basedir restriction in effect. File(/proc/cpuinfo) is not within the allowed path(s): (/srv/http/:/home/:/tmp/:/usr/share/pear/) in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 55
PHP Warning:  file_get_contents(/proc/cpuinfo): failed to open stream: Operation not permitted in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 55
PHP Warning:  file_get_contents(): open_basedir restriction in effect. File(/proc/cpuinfo) is not within the allowed path(s): (/srv/http/:/home/:/tmp/:/usr/share/pear/) in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 55
PHP Warning:  file_get_contents(/proc/cpuinfo): failed to open stream: Operation not permitted in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 55
WARNING: resetting threads to 0
PHP Warning:  file_get_contents(): open_basedir restriction in effect. File(/proc/meminfo) is not within the allowed path(s): (/srv/http/:/home/:/tmp/:/usr/share/pear/) in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 69
PHP Warning:  file_get_contents(/proc/meminfo): failed to open stream: Operation not permitted in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 69
PHP Notice:  Undefined offset: 1 in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 71
PHP Warning:  file_get_contents(): open_basedir restriction in effect. File(/proc/meminfo) is not within the allowed path(s): (/srv/http/:/home/:/tmp/:/usr/share/pear/) in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 62
PHP Warning:  file_get_contents(/proc/meminfo): failed to open stream: Operation not permitted in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 62
PHP Notice:  Undefined offset: 1 in /home/ppawel/src/osm/Nominatim/lib/lib.php on line 64
Create DB
Create DB (2)
createlang: language "plpgsql" is already installed in database "nominatim"
psql: FATAL:  role "ppawel" does not exist
PHP Warning:  proc_open(): open_basedir restriction in effect. File(/dev/null) is not within the allowed path(s): (/srv/http/:/home/:/tmp/:/usr/share/pear/) in /home/ppawel/src/osm/Nominatim/utils/setup.php on line 534
PHP Warning:  proc_open(/dev/null): failed to open stream: Operation not permitted in /home/ppawel/src/osm/Nominatim/utils/setup.php on line 534
ERROR: unable to start pgsql
unable to start pgsql

Paweł Paprota 12:53, 19 August 2012 (BST)

This was fixed by disabling open_basedir option in php.ini but I think users should be warned to enable it again after setup... Paweł Paprota 13:05, 19 August 2012 (BST)

Nominatim-2.0.1.tar.bz2

File Nominatim-2.0.1.tar.bz2 is compressed only with TAR (without BZIP2).

Extract command: tar xvf Nominatim-2.0.1.tar.bz2

Wojtekk (talk) 19:36, 28 March 2013 (UTC)

Move PBF after initial import (during index phase)?

I finished the initial import phase yesterday night and I'm now in the indexing phase for some time... rank 30 is expected to take approx. 2 more days... am I assuming right that I can move the PBF source file to another partition (to get more space over the weekend, in case the database keeps growing)? Right now it doesn't seem to be open by any process (but don't know if setup.php or another component tries to access it later). --Jot (talk) 14:14, 11 March 2016 (UTC)

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Talk:Nominatim/Installation_on_Debian_or_Ubuntu&oldid=1280592"