OpenHistoricalMap/Dev
- About
- OpenHistoricalMap's original server-side configuration
- Reason for being historic
- The server-side configuration is now tracked in various GitHub readmes. See OpenHistoricalMap/Development for starting points.
- Captured time
- 2015
Introduction
Documentation of things for OHM development
OHM Github repo: https://github.com/organizations/OpenHistoricalMap
Key pieces of infrastructure: - Rails Port - Mod_tile
Server Setup
The server was mostly set up with packages according toː http://switch2osm.org/serving-tiles/building-a-tile-server-from-packages/
Originally the older mapnik osm.xml stylesheet in /home/tim/src/mapnik-style/osm.xml was used
Then the ohm-carto carto css was deployed. This currently lives in /home/tim/ohm-carto/
The shapefiles I downloaded and put into ohm-carto/data.zip which need to be extracted into ohm-carto/data to use.
the carto node module was installed into this ohm-carto directory.
The mapnik.xml was created using the carto command:
:~/ohm-carto$ ./node_modules/carto/bin/carto project.mml > mapnik.xml
The default tiles site was disabled from Apache. Instead we just added the one line to the OHM VHOST:
LoadTileConfigFile /etc/renderd.conf
The renderd conf:
[renderd] stats_file=/var/run/renderd/renderd.stats socketname=/var/run/renderd/renderd.sock num_threads=4 tile_dir=/var/lib/mod_tile [mapnik] plugins_dir=/usr/lib/mapnik/input font_dir=/usr/share/fonts/truetype/ font_dir_recurse=true [default] XML=/home/tim/ohm-carto/mapnik.xml #XML=/home/tim/src/mapnik-style/osm.xml URI=/ohm_tiles/ DESCRIPTION=This is the standard osm mapnik style HOST=openhistoricalmap.org
Replication / Planet
Osmosis was installed outside the packages and is installed in /usr/local/bin
OHM API -> Replication change files: home/tim/replication/minute/data
every 5 minutes
crontab for tim user
*/5 * * * * cd /home/tim/replication/minute/; osmosis -q --replicate-apidb authFile=/home/tim/osmosis/dbAuth.txt validateSchemaVersion=no --write-replication workingDirectory=data
Replication change files -> mapnik database and refresh tiles etc:
crontab for www-data user
*/5 * * * * /var/lib/mod_tile/openstreetmap-tiles-update-expire.sh
/var/lib/mod_tile: configuration.txt - points to baseUrl=file://localhost/home/tim/replication/minute/data openstreetmap-tiles-update-expire.sh - collation, import of changes and expiry of tiles replag - a script to show the lag between imports, called by the expire.sh script
Planet can be generated via
osmosis --read-apidb authFile=/home/tim/osmosis/dbAuth.txt validateSchemaVersion=no --write-xml file="ohm_planet_20150303.osm.bz2"
as of 3 March 2015 it took around 2 1/2 hrs 7943383 ms
osm2pgsql custom things
Osm2pgsql must be compiled with lua support.
in the https://github.com/openhistoricalmap/osm2pgsql repository - there are two files of interest:
ohm_style.style - here we specify that start_date and end_date should be dates in the database, and to add an extra start_date_year and an end_date_year coluumn
ohm_tag_transform.lua - this is a Lua script to parse out and ensure that the dates are in YYYY-MM-DD formatting. It also parses out the year into the _year column value
example execution:
osm2pgsql --slim -S ohm_style.style --tag-transform-script ohm_tags_transform2.lua -d ohm_gis -C 16000 --number-processes 4 ohm_test.osm
Custom layers
OHM Readme
The 2008 and 2009 styles are meant to be used with the OHM osm2pgsql style and tag transforms which use start_date and end_date which are postgresql date types and start_date_year and end_date_year which are number types
import the sql commands to create the two 2008 and 2009 views and add them to geometry columns
psql -d ohm_gis_hack -f views_2008_2009.sql
Compile the carto stylesheets
1. install carto locally
npm install carto
2. compile new project files for each:
./node_modules/carto/bin/carto project_2008.mml > mapnik_2008.xml ./node_modules/carto/bin/carto project_2009.mml > mapnik_2009.xml
3. set up renderd configuation for each
[ohm_2008] XML=/home/tim/ohm-carto/ohm-carto/mapnik_2008.xml URI=/ohm_2008/ DESCRIPTION=A custom ohm map showing those things in 2008 HOST=openhistoricalmap.org [ohm_2009] XML=/home/tim/ohm-carto/ohm-carto/mapnik_2009.xml URI=/ohm_2009/ DESCRIPTION=A custom ohm map showing those things in 2009 HOST=openhistoricalmap.org
troubleshooting
Renderd
Run renderd from the command line
sudo service renderd stop renderd -f -c /etc/renderd.conf
see what errors you find