Spreadnik/Tutorial

Setting up our work environment

This tutorial will assume the following:

• You have downloaded the OSM mapnik scripts.
• You have downloaded and uncompressed the coastlines.
• You have a postGIS DB up and running, with some OSM data imported into it.
• You have tested out the generate_image.py script with the default OSM style.
• You have installed the PHP5 command line interface (on Ubuntu: sudo apt-get install php5-cli)

Making an (almost) blank XML stylesheet

So you've had a peek at the osm-template.xml file and wished it was all a bad dream. We've all been there. So let's put that apart, and write a very basic XML stylesheet with just the coastlines:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE Map
[
       <!ENTITY world_boundaries_dir   "/home/foo/mapnik/world_boundaries/">
]>
 <Map bgcolor="#b5d0d0" srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over">
	
	
  <Style name="world">
    <Rule>
      <MaxScaleDenominator>250000000000</MaxScaleDenominator>
      <MinScaleDenominator>600000</MinScaleDenominator>
      <PolygonSymbolizer>
        <CssParameter name="fill">#e3f2e4</CssParameter>
      </PolygonSymbolizer>
    </Rule>
  </Style>
  <Style name="coast-poly">
    <Rule>
      <MaxScaleDenominator>600000</MaxScaleDenominator>
      <PolygonSymbolizer>
        <CssParameter name="fill">#e3f2e4</CssParameter>
      </PolygonSymbolizer>
    </Rule>
  </Style>
  <Style name="builtup">
    <Rule>
      <MaxScaleDenominator>2500000</MaxScaleDenominator>
      <MinScaleDenominator>500000</MinScaleDenominator>
      <PolygonSymbolizer>
        <CssParameter name="fill">#ddd</CssParameter>
      </PolygonSymbolizer>
    </Rule>
  </Style>
  
  
  <Layer name="world" status="on" srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over">
    <StyleName>world</StyleName>
    <Datasource>
      <Parameter name="type">shape</Parameter>
      <Parameter name="file">&world_boundaries_dir;/shoreline_300</Parameter>
    </Datasource>
  </Layer>
  <Layer name="coast-poly" status="on" srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over">
    <StyleName>coast-poly</StyleName>
    <Datasource>
      <Parameter name="type">shape</Parameter>
      <Parameter name="file">&world_boundaries_dir;/processed_p</Parameter>
    </Datasource>
  </Layer>
  <Layer name="builtup" status="on" srs="+proj=merc +datum=WGS84 +over">
    <StyleName>builtup</StyleName>
    <Datasource>
      <Parameter name="type">shape</Parameter>
      <Parameter name="file">&world_boundaries_dir;/builtup_area</Parameter>
    </Datasource>
  </Layer>
  
  
 </Map>
 

Be sure to change world_boundaries_dir to the right directory, and save the file as (e.g.) tutorial.xml.

Next, edit the generate_image.py script to use this basic stylesheet instead of the default one:

if __name__ == "__main__":
#   try:
#       mapfile = os.environ['MAPNIK_MAP_FILE']
#   except KeyError:
#       mapfile = "osm.xml"
   mapfile = "tutorial.xml"
   map_uri = "image.png"

At this point, run generate_image.py. You should get an image of land and sea, meaning that the coastline files are being used. If you get an all-white or an all-blue image, check the output for any errors.

Structure of a Spreadnik spreadsheet

OK, now you're ready to load up your favourite spreadsheet software. Be sure to always save your spreadsheets as .csv files, as spreadnik does not understand .ods nor .xls files. Have this in mind if you're using some fancy formulas and you have to keep a .ods file around in order not to lose them.

Edit your first spreadsheet to look like this:

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O	P	Q	R	S	T	U
1	highway	pass	symbolizer	z1	z2	z3	z4	z5	z6	z7	z8	z9	z10	z11	z12	z13	z14	z15	z16	z17	z18
2
3	motorway	casing	line.stroke						black	black	black	blue	blue	#000080	#000080	#000080
4	motorway	casing	line.stroke-width						0.5	0.5	1	1	2	2	2	3

Let's understand how this works:

The first row contains, in this order:

• The tag keys you want to filter stuff by. We'll be using highway=motorway as an example, so first goes "highway". If you need to filter by any other tag key, just insert more filter columns.
• The "pass" and "symbolizer" are columns with an automagical meaning. Use the "pass" column to keep symbolizers grouped so you can order passes later.
• The z1-z18 columns mean the different zoom levels you'll apply the symbolizers. You should already be familiar with the "zoom level" concept used in OSM tiles.

Any row not having a pass or a symbolizer is treated like a comment. Feel free to put a scale factor reminder for every zoom level, extra space for fancy formulas, or whatever. Just make sure these rows have a blank pass and symbolizer.

If you've seen how mapnik's symbolizers work, the rest of the sheet should be self-explaining. Any features matching highway=motorway will be rendered with a LineSymbolizer, its color and width varying with the zoom level. Neat, huh?

TODO: running spreadnik

First, copy the spreadnik script into the directory that contains your *.csv files.

Then, open up a console, change into the directory that contains your *.csv files, make sure the script is executable (chmod 755), and just run spreadnik:

cd foo/whatever/spreadnik
./spreadnik.php

If you're using MS windows, you might have to run it like:

c:\php5\php.exe spreadnik.php

You'll get an output like:

Processing tutorial
0%...20%...40%...60%...80%...100%

Et voilà!. You'll find some *.sty files alongside your *.csvs. Those files contain XML snippets to be included into the main mapnik stylesheet.

TODO: linking the .sty files into the XML

You'll have one *.sty file for every pass inside every *.csv file. Now you've got to put that stuff into the main XML stylesheet. You could just copy-paste it, but then the file would become a huge incomprehensible shapeless blob. But if you're read Mapnik wiki on how to manage large XML files, you already know that you have to use XML entities to link stuff. So let's revisit the basic XML file we wrote earlier:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE Map
[
       <!ENTITY world_boundaries_dir   "/home/foo/mapnik/world_boundaries/">
       <!ENTITY dbtype "postgis">
       <!ENTITY dbhost "localhost">
       <!ENTITY dbuser "foobar">
       <!ENTITY dbname "osm">
       <!ENTITY dbprefix "planet_osm">
       <!ENTITY tutorial-casing SYSTEM "/home/foo/spreadnik/tutorial-casing.sty">
]>
<Map bgcolor="#b5d0d0" bleh bleh >

&tutorial-casing;

 <Style name="world"></Style>
 <Style name="coast-poly"></Style>
 <Style name="builtup"></Style>
   
 <Layer name="world"></Layer>
 <Layer name="coast-poly"></Layer>
 <Layer name="builtup"></Layer>
 
 
 <Layer name="tutorial">
 <StyleName>tutorial-fill</StyleName>
   <Datasource>
     <Parameter name="type">&dbtype;</Parameter>
     <Parameter name="user">&dbuser;</Parameter>
     <Parameter name="dbname">&dbname;</Parameter>
     <Parameter name="table"> (select way,highway from &dbprefix;_line where "highway" is not null) as roads </Parameter>
     <Parameter name="estimate_extent">false</Parameter>
     <Parameter name="extent">-20037508,-19929239,20037508,19929239</Parameter>
   </Datasource>
 </Layer>
</Map>

Save the XML file and run generate_image so that you can see the amazing results. If you're using generate_tiles or mod_tile, clean up your tile cache and render some tiles. You'll see that the zoom levels specified in the spreadsheet are kept.

As you can see, spreadnik is not entirely automagical - you still have to have to write the <layer>, link the <stylename>s, and write down the database stuff. However, as you won't be creating layers all the time, this becomes very manageable. Also note the <parameter name='table'>. Spreadnik doesn't know and doesn't care about your postgis database, so you'll have to decide if you want to render lines, polygons or points.

When typing in that SQL query, the rule of thumb is to select the geometry column ('way') and every filter you're using in the spreadsheet (in this tutorial, we're only using 'highway'). Also, don't select the whole table, as this will make mapnik fetch a huge lot of data from the DB into memory (your HDs will scratch more than neccesary) just to discard it later (because it doesn't match any rendering rule).

Stepping up: more than one filter and one pass

Using just one filter (highway) and one pass (casing) is no fun. We'll see how to render highway casing plus fill, and render toll motorways in a different way.

When using several passes, remember that mapnik uses the painter's algorithm. The point is to first render the casings of all motorways, and only then start painting the fills.

This explanation is boring, so just copy something like this into your spreadsheet:

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O	P	Q	R	S	T	U	V
1	highway	toll	pass	symbolizer	z1	z2	z3	z4	z5	z6	z7	z8	z9	z10	z11	z12	z13	z14	z15	z16	z17	z18
2
3	motorway,trunk		casing	line.stroke						red	red	red	red	red	red	red	red	red	red	red
4	motorway,trunk		casing	line.stroke-width						0.5	0.5	1	1	2	2	2	3	3	4	4
5
6	motorway	yes	fill	line.stroke						yellow	yellow	yellow	yellow	yellow	yellow	yellow	yellow	yellow	yellow	yellow
7	motorway	no	fill	line.stroke						white	white	white	white	white	white	white	white	white	white	white
8	motorway		fill	line.stroke-width						0.25	0.25	0.5	0.5	1	1	2	2	2	3	3

Again, this is pretty much self-explaining if you look at it for a while.

We've added a 'toll' filter. This means that you have to include 'toll' into the <parameter name='table'> in the main XML. And for that to work, your PostGIS DB has to contain a 'toll' column. Maybe you'll have to re-run osm2pgsql.

Anyway, you can see that all motorways, tolled or not, will have the same width for both the casing and the fill lines, but the color of the fill will depend on the toll.

Note that you don't have to worry about specifying the fill width for both tolled and non-tolled roads. If you leave a filter empty, that symbolizer parameter will apply to all features with any value for that filter.

On the other hand, if you specify multiple values for a filter, that symbolizer parameter will apply to any of those. In this example, trunk roads will be displayed with a red line without an inner casing.

When you're ready making changes, just re-run spreadnik. If you've added a filter or a pass, you'll have to edit the main XML file...

(bleh bleh)
       <!ENTITY tutorial-casing SYSTEM "/home/foo/spreadnik/tutorial-casing.sty">
       <!ENTITY tutorial-fill SYSTEM "/home/foo/spreadnik/tutorial-fill.sty">
]>
<Map bgcolor="#b5d0d0" bleh bleh >

&tutorial-casing;
&tutorial-fill;

(bleh bleh)
 
 <Layer name="tutorial">
 <StyleName>tutorial-casing</StyleName>
 <StyleName>tutorial-fill</StyleName>
   <Datasource>
     <Parameter name="type">&dbtype;</Parameter>
     <Parameter name="user">&dbuser;</Parameter>
     <Parameter name="dbname">&dbname;</Parameter>
     <Parameter name="table"> (select way,highway,toll from &dbprefix;_line where "highway" is not null) as roads </Parameter>
     <Parameter name="estimate_extent">false</Parameter>
     <Parameter name="extent">-20037508,-19929239,20037508,19929239</Parameter>
   </Datasource>
  </Layer>
</Map>

Note that there is no "toll is not null" clause ni the <parameter name='table'>. This is because we want to show all roads, even if the toal tag is nonexistant or empty. You'll have to apply some common sense to tell mapnik which features you want to be shown.

Also, note the order of the tutorial-casing and tutorial-fill styles inside the "roads" layer. Mapnik depends on the order of the layers and styles to paint stuff on top of other stuff. The layers which are specified first are painted first, and inside a layer, the styles which are specified first are painted first.

List of supported symbolizers

Currently, spreadnik only supports the following symbolizers (which must be abbreviated in the spreadsheet):

• LineSymbolizer (line)
• PolygonSymbolizer (poly)
• TextSymbolizer (text)
• PointSymbolizer (point)

Spreadnik will try to find the images which are referred to in PointSymbolizer (and in a future, in PolygonPatternSymbolizer), and check for its domensions so you don't have to. You will have to both configure a variable and define a XML entity to point to the right directory if you want all that to work fine.

Also, if you want to draw multiple symbolizers of the same type (e.g. two linesymbolizers with alternating dashing) in the same pass, you may append a number to the symbolizer type. e.g. use "line.stroke" and "line2.stroke".

Tips

• Have care with spreadsheet localization, particularly with the decimal dots and commas. Spreadnik tries to auto-correct these, but some cases may go unnoticed.
• It's a spreadsheet, so use it to your advantage! Use the functionality of the spreadsheet software (clone data, insert columns, lock the first rows, use formulas). With formulas, you can make the width of some features depend on the width of some others.
• Feel free to use more passes as you see fit to totally control the overlappings. highway=service rendering on top of highway=trunk? No problem, just split roads-fill into roads-fill-motorway and roads-fill-trunk...