OnDemandTileServer

Stale: NOTE! THIS GUIDE IS VERY OLD!

Use the latest guides available on https://switch2osm.org/serving-tiles/

This page provides a set of instructions for setting up a server for on-demand tiles generation. This has been created to assist the Myanmar Sahana installation (wikipedia:Sahana) Currently this server is being configured as a VMWare appliance to simplify the deployment of the software.

This page is intended as a step by step guide to a specific server configuration, but much of the information is copied from the Mapnik page which contains more details on Mapnik configuration.

Base Server Setup

Virtual Machine Configuration

Create a new VMWare server image with 512MB of RAM, and a 50GB hard drive (don't pre-allocate all disk space). Connect the virtual CDROM to the Fedora 8 DVD and start the virtual machine.

Fedora Installation

Base Setup

Use the default partition layout with a 50GB root partition, 100MB boot partition, and 1GB of swap.

Set the hostname to tilegen.bretth.com (change as appropriate).

Set the eth0 network device to use DHCP (will need to be changed to static IP when deployed).

Set the timezone to Melbourne/Australia (change as appropriate).

Set the root password to "tilegen".

Package Selection

Under main package selection, only select “Web server” checkbox and select “Customise now” radio button.

Under Customisation pages:

Applications: Only select “Graphical Internet” and “Text-based Internet”.

Servers: Only select “Web Server”, “PostgresSQL Database”, and “Printing Support”.

Base System: Deselect “Java”.

First Boot Configuration

The Fedora DVD can now be disconnected from the virtual machine.

Firewall

Firewall should be enabled.

Trusted services should be “SSH”, “Secure WWW (HTTPS)” and “WWW (HTTP)”.

Selinux

Disable selinux. Selinux is a difficult beast to get working correctly and isn't essential. Selinux can be enabled later on when more time is available to configure it appropriately.

Date and Time

Select the “Enable Network Time Protocol” checkbox.

Hardware Profile

I don't bother sending for vmware images.

Create User

Create a new user called “tilegen” with password “tilegen” and full name “Tiles Generator”.

A reboot will be required at this point due to selinux changes.

Final Server Steps

Upon reboot login as the tilegen user.

Run Applications->System Tools->Software Updater and update all packages. (Note: If it complains that another process has a lock, kill any processes on the machine that have "yum-updates" in their name.)

Software Installation

This section installs OSM related software some of which is not available in standard Fedora repositories.

Subversion Client

As root, install the subversion source control client.

# yum install subversion

RPM Build Environment

As root, download and install the rpm build tools.

# yum install rpm-build rpmdevtools

Configure the RPM build tree.

$ rpmdev-setuptree

Java

Download Java 6 Update 6 from Java SE Downloads using firefox. Download the self-extracting file, NOT the self-extracting RPM. If only later versions are available the following build and install steps will need to be modified as appropriate.

Move the downloaded Java download to the rpm development tree.

mv Desktop/jdk-6u6-linux-i586.bin rpmbuild/SOURCES/

Download the java src RPM.

$ wget http://mirrors.dotsrc.org/jpackage/1.7/generic/non-free/SRPMS/java-1.6.0-sun-1.6.0.6-1jpp.nosrc.rpm

Install the java src RPM to the rpm development tree.

$ rpm -i java-1.6.0-sun-1.6.0.6-1jpp.nosrc.rpm
$ rm java-1.6.0-sun-1.6.0.6-1jpp.nosrc.rpm

As root, install some java dependencies.

# yum install jpackage-utils libXp

Build the Java RPM binary packages.

$ rpmbuild -bb rpmbuild/SPECS/java-1.6.0-sun.spec

As root, install the main java RPM. The remaining RPMs are not necessary for osmosis.

# rpm -Uvh ~tilegen/rpmbuild/RPMS/i586/java-1.6.0-sun-1.6.0.6-1jpp.i586.rpm

Verify that java was installed correctly.

$ java -version
java version "1.6.0_06"
...

Remove unnecessary java files.

$ rm -r rpmbuild/SOURCES/jdk* rpmbuild/SOURCES/java* rpmbuild/RPMS/i586/java* rpmbuild/SPECS/java* rpmbuild/BUILD/jdk*

Osmosis

Download and install the osmosis application.

$ cd ~/osm
$ wget http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-latest.zip
$ unzip osmosis-latest.zip
$ ln -s ~/osm/osmosis-0.36 osmosis
$ rm osmosis-latest.zip
$ ln -s ~/osm/osmosis/bin/osmosis ~/bin/osmosis
$ chmod u+x ~/bin/osmosis

Verify the osmosis installation.

$ osmosis
osmosis

Example Usage
...

Myanmar Data Replication

Download the myanmar data and scripts (in this order).

$ cd ~/osm
$ mkdir myanmar
$ cd myanmar
$ wget http://www.bretth.com/myanmar/timestamp.txt
$ wget http://www.bretth.com/myanmar/configuration.txt
$ wget http://www.bretth.com/myanmar/myanmar.osm.gz
$ wget http://www.bretth.com/myanmar/replicate_osm_file.sh
$ chmod u+x replicate_osm_file.sh

Edit the ~/osm/myanmar/replicate_osm_file.sh file and replace the CMD variable with "/home/tilegen/osm/osmosis/bin/osmosis".

Edit the tilegen crontab with this command.

$ crontab -e

Edit the file to look like this.

MAILTO=<replace this with your email address>
40 * * * * cd ~/osm/myanmar; ./replicate_osm_file.sh

If you wish to verify this is working without waiting until 40 minutes past the hour, replace the 40 with a *. Make sure you replace the 40 when you verify correct operation to reduce load on the central OSM planet server.

PostgreSQL and PostGIS

As root, install PostGIS.

yum install postgis postgresql-server
service postgresql initdb

Tune the server configuration to support our workload. Open /var/lib/pgsql/data/postgresql.conf and edit the following variables. More details are provided on the Mapnik#Tuning_the_database page if further customisation is required.

shared_buffers = 64MB

checkpoint_segments = 10

maintenance_work_mem = 128MB

As root, increase the maximum shared memory size allowed by the kernel.

# sysctl -w kernel.shmmax=268435456

Make the change permanent by adding the following entry to /etc/sysctl.conf

kernel.shmmax = 268435456

As root, start the database server and make it start on boot.

# service postgresql start
# chkconfig postgresql on

As root, change to the postgres user.

su - postgres

As the postgres user, create the gis database and make tilegen the owner.

$ createuser tilegen
Answer "y" to make the new role a superuser.
$ createdb -E UTF8 -O tilegen gis
$ createlang plpgsql gis

As the postgres user, configure the gis database as a PostGIS database.

$ psql -d gis -f /usr/share/pgsql/contrib/lwpostgis.sql
$ echo "ALTER TABLE geometry_columns OWNER TO tilegen; ALTER TABLE spatial_ref_sys OWNER TO tilegen;" | psql -d gis

Osm2pgsql

As root, install the osm2pgsql dependencies.

# yum install geos-devel proj-devel postgresql-devel libxml2-devel bzip2-devel make gcc kernel-devel gcc-c++

Create an osm directory to hold all osm software.

$ mkdir ~/osm

Download and compile the osm2pgsql software.

$ cd ~/osm
$ svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/
$ cd osm2pgsql
$ make

Load the myanmar data into the postgis database.

$ cd ~/osm/osm2pgsql
$ ./osm2pgsql -m -d gis ~/osm/myanmar/myanmar.osm.gz

Mapnik

As root, install the mapnik dependencies.

# yum install python python-devel boost-devel fribidi-devel libtool-ltdl-devel libpng-devel libjpeg-devel
# yum install libtiff-devel zlib-devel freetype-devel proj-devel gdal-devel libpqxx-devel boost zlib freetype proj gdal

Build the mapnik software.

$ wget http://snecker.fedorapeople.org/mapnik/f8/mapnik-0.5.0-1.fc8.src.rpm
$ rpm -i mapnik-0.5.0-1.fc8.src.rpm
$ rpmbuild -bb ~/rpmbuild/SPECS/mapnik.spec

As root, install the mapnik software.

# rpm -Uvh ~tilegen/rpmbuild/RPMS/i386/mapnik-*

Remove unnecessary mapnik files.

rm -r mapnik-*.rpm rpmbuild/BUILD/mapnik-0.5.0/ rpmbuild/RPMS/i386/mapnik-* rpmbuild/SOURCES/mapnik* rpmbuild/SOURCES/ppc64* rpmbuild/SPECS/mapnik.spec 

Download the bzip2 compressed DejaVu fonts using firefox from http://dejavu.sourceforge.net/wiki/index.php/Download.

Extract the fonts.

$ cd ~/osm
$ mkdir fonts
$ cd fonts
$ tar jxf ~/Desktop/dejavu-fonts-*

As root, install the fonts.

# cd /usr/lib/mapnik/fonts
# mkdir orig
# mv *.ttf orig
# cp ~tilegen/osm/fonts/dejavu-fonts-*/ttf/*.ttf .

Remove unnecessary font files.

$ rm -r ~/osm/fonts
$ rm ~/Desktop/dejavu-fonts-*

Download and extract world boundary data.

$ cd ~/osm
$ mkdir world_boundaries_download
$ cd world_boundaries_download
$ wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz (51MB)
$ wget http://beta.letuffe.org/ressources/processed_p.zip (~230MB)
$ tar zxf world_boundaries-spherical.tgz
$ unzip processed_p.zip
$ mv coastlines/* world_boundaries
$ rmdir coastlines
$ mv world_boundaries ..

Remove unnecessary boundary files.

$ cd ~/osm
$ rm -r world_boundaries_download

Download the mapnik scripts from the OSM subversion repository.

$ cd ~/osm
$ svn co http://svn.openstreetmap.org/applications/rendering/mapnik

Edit the ~/osm/mapnik/set-mapnik-env file as follows:

Set the MAPNIK_MAP_FILE parameter to be ~/osm/mapnik/osm.xml

Set the MAPNIK_SYMBOLS_DIR parameter to be ~/osm/mapnik/symbols

Set the MAPNIK_WORLD_BOUNDARIES_DIR parameter to be ~/osm/world_boundaries

Set the MAPNIK_TILE_DIR parameter to be ~/osm/tiles/ (must end with a slash)

Set the MAPNIK_DBNAME parameter to be gis

Generate the mapnik configuration file.

$ cd ~/mapnik
$ source ./set-mapnik-env
$ ./customize-mapnik-map > $MAPNIK_MAP_FILE

Create the mapnik tiles directory.

$ mkdir ~/osm/tiles

Test the mapnik installation by rendering a test image of the UK.

$ cd ~/osm/mapnik
$ ./generate_image.py
$ eog image.png (this step requires X windows to be running)

Mod_tile

As root, install the mod_tile dependencies.

# yum install httpd-devel agg-devel

Download the mod_tile source code.

$ cd ~/osm
$ svn co http://svn.openstreetmap.org/applications/utils/mod_tile

Note: Most of the following path edits are not necessary for the current mod_tile version which started to use config files.

Within the ~/osm/mod_tile/gen_tile.cpp file, edit the following entries:

The line starting with static const char *mapfile becomes static const char *mapfile = "/home/tilegen/osm/mapnik/osm.xml";

The line datasource_cache::instance()->register_datasources("/usr/local/lib64/mapnik/input"); becomes datasource_cache::instance()->register_datasources("/usr/lib/mapnik/input");

The line load_fonts("/usr/local/lib64/mapnik/fonts", 0); becomes load_fonts("/usr/lib/mapnik/fonts", 0);

Within the ~/osm/mod_tile/render_config.h file, edit the following entries:

The web server root directory constant WWW_ROOT "/home/www/tile" becomes WWW_ROOT "/var/www/html"

The tile expiry constant PLANET_INTERVAL (7 * 24 * 60 * 60) becomes PLANET_INTERVAL (60 * 60)

Compile the mod_tile software.

$ cd ~/osm/mod_tile
$ make

As root, install the mod_tile software.

# cd ~tilegen/osm/mod_tile
# make install

As root, create the tile render directories and set the correct permissions.

# cd /var/www/html
# mkdir osm_tiles2
# mkdir direct
# chown tilegen:tilegen osm_tiles2
# chown tilegen:tilegen direct

As root, create a file called /etc/httpd/conf.d/mod_tile.conf with the following contents.

LoadModule tile_module modules/mod_tile.so

As root, restart the web server (ignore a FAILED message stopping the server because it probably isn't running at this point) and ensure it always starts at boot.

# service httpd restart
# chkconfig httpd on

Run the renderd daemon.

$ cd ~/osm/mod_tile
$ ./renderd
Rendering daemon

Load http://localhost/osm_tiles2/5/24/14.png in firefox. A single tile covering the south of Myanmar and some of Thailand should be displayed.

Myanmar Data Loading

The #Myanmar Data Replication will ensure that the Myanmar data file is kept current. In order to load it into the mapnik database every hour, create the following cron entry.

Edit the tilegen crontab with this command.

$ crontab -e

Add the following entry to the crontab.

45 * * * * cd ~/osm/osm2pgsql;./osm2pgsql -m ~/osm/myanmar/myanmar.osm.gz;touch /tmp/planet-import-complete