Tiles@home/Install Guide

Prerequisites

Warning Tiles@Home needs copious amounts of RAM for complex tiles. A machine with at least 2 GB of physical RAM is recommended, although complex tiles can use in excess of 16 GB of memory.

Also be aware that the generated Data can quickly eat up gigabytes on your harddrive. (make sure to set the option in tilesAtHome.conf to delete zip files after upload)

join the mailinglist.

Software

Perl
Inkscape (first choice) or Batik (libbatik-java; might need less_? memory, but might be (very) much slower in some conditions; might be hard to get working^(?))
pngcrush (faster) or optipng (better)
pngnq (at least version 0.5) for color reduction (optional, but highly recommended)
ImageMagick (done with Perl: perlmagick/ImageMagic-perl)
Java (java, openjdk-* (jre) or sun-java6-jre (non-free repo))
Zip (to upload your generated tiles)
XmlStarlet
libxml2 and libxslt

Some versions of Inkscape are known not to work (produce only black tiles):

Fail: ~~v0.40, 0.41~~. Work: 0.42, 0.43, 0.44, 0.45*, 0.45.1*, 0.46
- only when rendering z17 area as one bitmap (RenderStripes=0)

Also, you may need to start Inkscape once through your X server, so that it creates various "preferences" files, but people are reported to run Tiles@home on completely headless servers, so YMMV. If you run a headless server (without X installed), you can still run Inkscape in graphic mode by starting it from another computer:

you@home_machine$ ssh -X -l remote_user remote_machine
remote_user@remote_machine$ inkscape &

Also remember to protect this file so it later not overwritten, see "Known Issues"

Perl modules

Debian (minimal, server):

apt-get install yaml-perl*

 cpan -i Set::Object Math::Trig Math::Complex Math::Vec LWP::Simple URI File::Copy GD AppConfig Error IPC::Run \
 Class::Accessor Image::Magick HTML::Parser XML::Parser::PerlSAX XML::XPath XML::Writer

Set::Object (perl-Set-Object on Fedora & libset-object-perl in Debian)
Math::Trig, Math::Complex (both in the standard Perl package)
Math::Vec (available in libmath-vec-perl on Debian, see the "Known Issues" section)
LWP::Simple
URI
File::Copy (~quote: "is part of the perl(-5.8.9) distribution")
GD (on Ubuntu libgd-gd2-perl on other systems, like Gentoo just media-libs/gd. On Fedora install perl-GD) version 2
AppConfig (on Fedora also perl-AppConfig, on Debian libappconfig-perl)
Error (ppm install http://www.bribes.org/perl/ppm/Error.ppd)
IPC::Run (ppm install http://www.bribes.org/perl/ppm/IPC-Run.ppd; libipc-run-perl)
Class::Accessor
Image::Magick is required for the lowzoom script (perlmagick on Ubuntu, ImageMagic-perl on Fedora).
HTML::Parser is needed to upload computed data
For Osmarender/perl we require additional modules: libxml-perl libxml-sax-perl libxml-writer-perl libxml-xpath-perl libset-object-perl
- XML::Parser::PerlSAX
- XML::XPath
- XML::Writer

Installation instructions

Use the following commands in your system console to install all required components:

Arch Linux

pacman -S zip subversion inkscape libxml-perl ttf-dejavu ttf-thai ttf-indic-otf optipng perl-yaml perl-gd \
perl-uri perl-xml-writer perl-xml-xpath perl-html-parser perl-appconfig perl-error perl-class-accessor

cpan -i IPC::Run Image::Magick Math::Vec LWP::UserAgent

from AUR: pngnq, pngcrush, xmlstarlet, perl-set-object

Debian 4.* Etch, 5.* Lenny, 6.* Squeeze (stable) or wheezy (unstable)

Note: Debian version lenny (oldstable) contains old version of inkscape. Please upgrade to 6.0.3 squeeze or lenny (5.0.3).

Install all packages. As root:

aptitude install xmlstarlet subversion libgd-gd2-perl libmath-vec-perl inkscape \
pngcrush optipng zip libwww-perl pngnq imagemagick \
ttf-dejavu xfonts-thai ttf-indic-fonts libxml-writer-perl libset-object-perl \
libappconfig-perl libxml-xpath-perl libxml-perl libxml-sax-perl liberror-perl \
libipc-run-perl libclass-accessor-perl libmodule-pluggable-perl

Etch-only:

In order to get it right running on etch you have to backport and upgrade pngnq to 0.5-3. Etch uses by default version 0.4 which shows correctly the version with option -V but doesn't know anything about the for t@h mandatory option -e.

Run aptitude install dpkg-dev build-essential cdbs debhelper libpng12-dev
Download source wget http://ftp.se.debian.org/debian/pool/main/p/pngnq/pngnq_0.5.orig.tar.gz http://ftp.se.debian.org/debian/pool/main/p/pngnq/pngnq_0.5-3.diff.gz http://ftp.se.debian.org/debian/pool/main/p/pngnq/pngnq_0.5-3.dsc
Unpack dpkg-source -x pngnq_0.5-3.dsc ; cd pngnq-0.5
as user root: Build echo 5 >debian/compat ; dpkg-buildpackage -b -us -uc -d
as user root: Install dpkg -i ../pngnq_0.5-3*.deb

That's it. If you want to get rid of the previously installed development packages, which are no longer needed, just remove them with aptitude purge build-essential libpng12-dev dpkg-dev debhelper and you're done.

If some package is missing in etch check for a newer version from http://www.backports.org

Running a old version of pngnq from sid, testing or lenny do not show the version number when running pngnq -V Please run a package update and upgrade and you should get the fixed version 0.5-3. Bug#491860 Can also get binary here and install the file with dpkg -i filename.

You could also build your own package if you have a matching deb-src line in /etc/apt/sources.list. Run aptitude install dpkg-dev build-essential ; apt-get build-dep pngnq ; apt-get source pngnq ; cd pngnq-0.5/debian edit the file rules and change to VERSION=\\\"0.5\\\" then run cd .. ; dpkg-buildpackage -b -us -uc ; dpkg -i ../pngnq_0.5-2*.deb

FreeBSD

Install these packages:

either using the pre-built packages

# pkg_add -r subversion inkscape xmlstarlet pngcrush zip p5-GD p5-Math-Vec p5-libwww p5-AppConfig \
p5-Error p5-Class-Accessor p5-IPC-Run p5-Module-Pluggable p5-Set-Object p5-XML-XPath p5-XML-Writer \
p5-libxml dejavu pngnq

or from ports with portinstall (from ports-mgmt/portupgrade):

# portinstall subversion inkscape xmlstarlet pngcrush zip p5-GD p5-Math-Vec p5-libwww p5-AppConfig \
p5-Error p5-Class-Accessor p5-IPC-Run p5-Module-Pluggable p5-Set-Object p5-XML-XPath p5-XML-Writer \
p5-libxml dejavu pngnq

or with this custom metaport

If you're running a 32 bit (i386) version of FreeBSD (as opposed to amd64 version) the default max data size per process is 512MB. Inkscape will sometime need much more RAM than that, you'll need to add a few lines to your /boot/loader.conf :

kern.maxdsiz="1610612736" # 1.5GB
kern.dfldsiz="134217728" # 128MB
kern.maxssiz="134217728" # 128MB

maxdsiz is the maximum data size which is the maximum size a process can be, dfldsiz is the initial data size limit and maxssiz is the maximum stack size. maxdsiz has nothing to do with how much RAM you have, it's just the upper limit of how big a process can get.

Gentoo

Don't forget to set the png USE-Flag for dev-perl/GD.

emerge -auv virtual/perl-File-Spec dev-perl/Math-VecStat dev-perl/libwww-perl dev-perl/libxml-perl \
dev-perl/GD app-text/xmlstarlet dev-perl/Math-Vec media-gfx/pngcrush media-gfx/inkscape \
app-arch/zip app-arch/unzip dev-vcs/subversion media-gfx/optipng dev-perl/AppConfig \
dev-perl/Error media-gfx/pngnq media-fonts/dejavu dev-perl/Class-Accessor \
virtual/perl-Module-Pluggable dev-perl/IPC-Run dev-perl/URI media-fonts/lohit-fonts

#for or/p:
emerge -auv dev-perl/XML-XPath dev-perl/XML-Writer dev-perl/Set-Object

#fonts:
emerge -auv media-fonts/thaifonts-scalable media-fonts/wqy-bitmapfont media-fonts/wqy-unibit media-fonts/vlgothic
eselect fontconfig enable 85-wqy-bitmapsong.conf # this font currently needs manual enabling.

openSUSE

as root:

yast2 -i subversion inkscape xmlstarlet pngcrush optipng zip dejavu perl-libwww-perl perl-GD \
   perl-PerlMagick perl-Set-Object perl-XML-Writer perl-XML-Parser perl-XML-XPath perl-AppConfig \
   perl-libxml-perl

cpan Math::Vec
cpan Error
cpan IPC::Run
cpan Class::Accessor
cpan Module::Pluggable

On openSUSE 11.0 and 11.1 you need to set in tilesAtHome.conf: XmlStarlet=xml. On openSUSE 11.1, 11.2 and 11.3 package 'optipng' is not used. Instead package 'pngnq' has to be installed.

Hint: 'yast2 -i foo bar' reinstalls packages unnecessarily. Where zypper is available 'zypper in foo bar' is the better choice.

Mac OS X 10.5

Be sure to have the Apple developer tools installed and updated (version 3.1 at the time of writing). You will also need MacPorts to download and build most of the software needed from source. Otherwise you will have to compile them by hand or find pre compiled packages. Finally cpan will be used to install all missing perl libraries.

Check your PATH environment for /opt/local/bin and /opt/local/sbin. If both directories are not in the PATH add them in your .profile in your home or if it not exists create ist with

echo PATH=/opt/local/bin:/opt/local/sbin:$PATH >.profile

to set the PATH for MacPorts.

After installing MacPorts and setting the PATH open a terminal and type in:

sudo port install inkscape

This will keep your computer busy for at least 2 hours and ends in an error anyway. Just start it again and again ... until it is finished without error.

After that install the remaining ports with:

sudo port install xmlstarlet pngcrush p5-gd p5-appconfig p5-libwww-perl p5-libxml-perl p5-xml-writer \
p5-xml-xpath p5-set-object p5-error p5-ipc-run p5-class-accessor p5-module-pluggable

After installing the ports you need to install a missing perl library using cpan. Select a download mirror near to you and leave all other settings at their default values (just hit enter at all questions...). Compared to the ports installation this is pretty fast.

sudo cpan
install Bundle::CPAN
reload cpan
install Math::Vec
exit

The library will most likely fail to build, but this is no problem. You can do this manually. If you didn't changed the default settings the tarball is downloaded to:

.cpan/sources/authors/id/E/EW/EWILHELM/

Unzip and untar it and change in the now created source directory and run the build scripts.

sudo perl Build.PL
./Build
./Build test
sudo ./Build install

All the other needed libraries are already installed with the ports.

Now you need the DejaVu fonts. The installation is straight forward. Download the font from DejaVu unzip it and install it with the Font Book Application.

Finally you need pngnq. I didn't manage to compile pngnq from source properly. The original Makefile results in an build error and after modifying the PATH for PNGINC and PNGLIB the build process is OK but the result is a not properly working binary. Invoked without options it seems to work but 'pngnq -e .tmp.png -s1 -n256 <put_your_tile_name_here>.png', which is necessary for t@h, results in a Bus error. Any help is welcome here.
Running the binary from sourceforge works with a simple workaround. It looks for libpng12.0.dylib in /usr/local/lib but it is in /opt/local/lib. Just create the directory and symlink with

sudo mkdir /usr/local/lib
sudo ln -s /opt/local/lib/libpng12.0.dylib /usr/local/lib/libpng12.0.dylib
sudo cp pngnq /opt/local/bin

and it should work.

Ubuntu

sudo apt-get install xmlstarlet subversion libgd-gd2-perl libmath-vec-perl inkscape pngcrush perlmagick zip libwww-perl ttf-dejavu \
xfonts-thai libxml-writer-perl libset-object-perl libappconfig-perl libxml-xpath-perl libxml-perl libxml-parser-perl libxml-sax-perl \
pngnq optipng liberror-perl libipc-run-perl libclass-accessor-perl libmodule-pluggable-perl

Building xmlstarlet from source, you might need to add a couple EXTRA_LIBS to the make, like so:

make EXTRA_LIBS="-lgcrypt -lgpg-error"
[or]
make EXTRA_LIBS="/usr/lib/libgcrypt.a /usr/lib/libgpg-error.a"

Problem with pngnq: Some versions of Ubuntu[which? No problem on 10.04] pngnq do not show the version number when running pngnq -V. There is a new version available. You can download the Package for your specific architecture here.

Fedora

For a detailed Fedora installation guide read /Fedora 7. This command line installs all dependencies in Fedora 11, Fedora 12, Fedora 13 and Fedora 14:

yum install libxml2 libxslt inkscape xmlstarlet pngcrush pngnq optipng perl-Math-Vec perl-URI perl-GD perl-AppConfig \
ImageMagick-perl perl-HTML-Parser perl-XML-SAX perl-XML-XPath perl-XML-Writer perl-libwww-perl perl-libxml-perl \
perl-Error perl-Class-Accessor perl-IPC-Run perl-Module-Pluggable perl-Set-Object \
dejavu-fonts-common wqy-bitmap-fonts wqy-unibit-fonts VLGothic-fonts lohit-*

Historical Fedora Problems

~~pngcrush is not available in Fedora, so set the script to use optipng instead in tilesAtHome.conf~~

I have build Fedora 9 RPMs for pngcrush, pngnq and perl-Set-Object. They are available here. (Patrick Steiner)

pngcrush and pngnq are available in Fedora 9 updates now (11 Nov 2008).

The perl-XML-Writer RPM in Fedora 9 was broken before version 0.604-1. Make sure to install the latest updates. (R kleineisel 07:04, 24 October 2008 (UTC))

~~perl-Set-Object is in the package review and should follow soon (11 Nov 2008).~~

perl-Set-Object is now available

Fonts

Tiles@home uses the font DejaVu. Please make sure you have it installed on your system.

To install the DejaVu font on Gentoo: emerge media-fonts/dejavu

on Debian or Ubuntu: sudo apt-get install ttf-dejavu

on FreeBSD: pkg_add -r dejavu

on Fedora: yum install dejavu-fonts-common

on OpenSuSE (normally already installed) zypper in dejavu

on OS X Leopard use the Font Book application: File > Add Fonts... Navigate to the unzipped folder and click "Open"

on Windows XP, get and unzip the regular ttf font set:

Start > Control Panel > Fonts > File > Install New Font ... 
Navigate to the unzipped folder and click "Select All"

Non-latin fonts

Unfortunately DejaVu doesn't cover non-latin characters, especially CJK characters... DejaVu language coverage table Indic characters are also not rendered with dejavu . So kindly install indic fonts ( also known as lohit).

So you should ensure you install some additional language fonts otherwise rendering of text in any tiles that include those characters will be corrupt or missing.

Fedora 10: yum install wqy-bitmap-fonts wqy-unibit-fonts VLGothic-fonts for Chinese and Japanese

Debian: There are three packages: ttf-wqy-microhei, ttf-wqy-zenhei, xfonts-wqy. On squeeze ttf-wqy-microhei is sufficient: aptitude install ttf-wqy-microhei

Fedora: try thaifonts-scalable thai-scalable\*.

Fedora (recent versions) yum install wqy-bitmap-fonts wqy-unibit-fonts VLGothic-fonts vlgothic-fonts lohit-* thaifonts-scalable thai-scalable\*

Gentoo:

 emerge -uDavt vlgothic wqy-unibit lohit-fonts \
wqy-bitmapfont thaifonts-scalable; eselect fontconfig enable 85-wqy-bitmapsong.conf # (thaifonts doesn't seem to use eselect font system)

Checking the font setup

It is highly recommended after setting up the t@h client to check the installation. To test your setup you can use the runTests.pl script

DISCLAIMER: The test itself might need maintainance since it reports successful atttempts as "failed". Clearly a bug and you are welcome to fix this runtests.pl program bug. Whatever the test program tells you don't be discouraged. Check in the /tmp/ subdirectory where the generated test tile files are.

open a command window and change to the directory you installed the client
start the runTests.pl script:
perl runTests.pl fonttest
Visually compare the png files with the sample pictures in the tests subdirectory.
clean up by removing the contents of the uploadable directory. This is to prevent the upload of the files rendered just for testing purposes.
if you have questions join the mailinglist

Download the program

The recommended way to get the files is a checkout with "subversion" or "svn" for short

 svn co https://svn.openstreetmap.org/applications/rendering/tilesAtHome-dev/tags/Vizag tilesAtHome

the above will create a directory "tilesAtHome" and download all necessary files into that directory

Tiles@home will automatically update when running. To manually update simply change into the "tilesAtHome" directory and run svn up

This will not clobber any files or directories you created, but it will restore necessary files you deleted. So if you need to repair a tiles@home installation you can delete any file or directory (except the .svn directory) that is the probable cause and do an svn up afterwards.

When you are behind a proxy edit your ~/.subversion/servers file and add the lines

[groups]
osm = svn.openstreetmap.org
[osm]
http-proxy-host = yourproxy.somewhere
http-proxy-port = 80

Windows users can install the SVN client TortoiseSVN

join the mailinglist.

Configuration

Linux / Mac OS X / *BSD

Copy tilesAtHome.conf.linux to tilesAtHome.conf
For many systems you will have to change the XmlStarlet configuration option to xml in tilesAtHome.conf. This includes Mac OS X (not for macport), Mandriva Linux, openSUSE, RedHat EL/Centos, and FreeBSD.
Copy authentication.conf.example to authentication.conf
Edit authentication.conf and insert your upload username and password (see the Passwords Required section)

Windows

Copy tilesAtHome.conf.windows to tilesAtHome.conf
Copy authentication.conf.example to authentication.conf
Edit authentication.conf and insert your upload username and password (see the Passwords Required section)
See windows@home for some tips on getting the program running on Windows.

Running the program

The program should then generate 1365 tiles if area is densely mapped and less when empty subtiles is skipped.

Generate a specific tileset: perl tilesGen.pl xy 1234 5678 This is the x and y coordinates of a zoom-12 tileset you want to create. See Slippy map tilenames for what those numbers mean.
Continously generate tilesets as requested by the server and upload: perl tilesGen.pl loop This is what is most normally used.
Generate one tileset decided by the server: perl tilesGen.pl once
Upload any generated tilesets to the server (password required): perl tilesGen.pl upload
When using many cpu-cores or computers in your network, please use an UploadDirectory and run: perl tilesGen.pl upload_loop that will monitor the directory and upload any tilesets in it.
Normally the newly rendered tilesets are put in "WorkingDirectory/uploadable", where also the upload(_loop) instance always looks for files. However, when the UploadToDirectory is enabled, the render instances put their results in "UploadTargetDirectory". So in practise you should always use a path with "uploadable" as the last directory for the UploadTargetDirectory setting, otherwise they will not be uploaded. (version 23290 (Vizag))

To run Tiles@Home behind a proxy set the http_proxy variable of your shell. Within bash type export http_proxy=http://myproxy.domain:port/ before running tilesGen.pl

Join the mailinglist to keep yourself up2date with the latest developments and issues running the t@h client. Consider including your hardware setup at Tiles@home/Hardware_Comparison to compare the efficiency on various platforms and hardware configuration.

Checking your status

Thanks for rendering! If you're curious about what your machine has rendered, look on the t@h Show Users page for your userid (remember that authentication.conf may contain either your userid or your email address). It should be within the first hundred or so users if you're currently rendering and uploading. Click on your userid and you'll see what you've rendered recently and what you're currently working on.

Debugging the program

One way is to turn on trace, run and look at all that text passing by. Use perl -d tilesGen.pl xy 1234 5678 Then press t for trace and then c for continue.

Stopping the program

If you run in loop mode you might not want to abort a render, but make the client stop before it fetches the next job from the server. Put a file called stopfile.txt in the same directory as tilesGen.pl and it should exit after finishing (and uploading) the current tileset.

touch stopfile.txt

Alternatively, you can call

./tilesGen.pl stop

which does exactly the same.

Passwords Required

Rename "authentication.conf.example" to "authentication.conf", and enter the following information

UploadUsername = required
UploadPassword = required

This is now your OSM password. The username has to be your emailaddress for the first login. Later on your username will work too, but there is no reason to prefer this over the emailaddress.

If there's trouble join the mailinglist.

Known Issues

...with libGD aborting

Can't use an undefined value as a symbol reference at /usr/lib/perl5/GD/Image.pm line 175.

or

Can't use an undefined value as a symbol reference at /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux/GD/Image.pm line 175.

This means Inkscape did not create a png file from the svg. This can have multiple causes:

too complex XML for xmlstarlet (too much data to process) -- currently no solution/workaround
other problems in the XML->SVG->PNG chain

If this happens to you and none of the workaround/fixes works for you, contact the tilesathome mailing list.

...with Math::Vec

Apparently Math::Vec doesn't install that easily from CPAN. You may need to force CPAN.pm to use Module::Build:

o conf prefer_installer MB

If not...

in Gentoo Linux use the dev-perl/Math-Vec package.
For Debian (and Ubuntu) you can use the libmath-vec-perl debian package.
For Fedora you can use the perl-Math-Vec package.
For SuSE you can add a new repository to your package manager. You can find the repository via Webpin on [1].

Or else try get the module with CPAN, when it fails to build, you build it manually:

Manual build:
cd ~/.cpan/build/Math-Vec-0.04/; 
perl Build.PL
    ./Build
    ./Build test
    ./Build install

Or you can just install Module::Build, then the usual perl module installation routine will work.

...without DISPLAY set

When tilesGen.pl is run on a Linux computer without an X server running, or the DISPLAY variable not set, inkscape will output the error

(inkscape:<pid>): Gdk-CRITICAL **: gdk_display_list_devices: assertion `GDK_IS_DISPLAY (display)' failed

which is actually harmless, because a display is not needed in the conversion process, inkscape still renders the tiles OK.

In some cases though, inkscape quits with a segmentation fault. The cause can be corrupted or outdated font config files (not sure how this happens). Regenerating these with fonts-config (as root) helps.

If you are using a headless server, it probably is a good idea to use Batik instead. Make sure to set Batik=1 and correct BatikPath in tilesAtHome.conf.

...with DISPLAY set to an inaccessible display

inkscape forces useage of X if DISPLAY is set, even in batch mode. If inkscape is not allowed to use said display it will fail with an error.

Workaround:

unset DISPLAY

before running the client

...with Inkscape preferences files

Some versions of Inkscape may, in error situations, create a zero-length file in ~/.inkscape/preferences.xml or if using gnome then in ~/.config/inkscape/, and when invoked later, complain of a fatal error when trying to parse that file. If no X display is available at the time, the following will happen:

 ~/.inkscape/preferences.xml:1: parser error : Document is empty 
 ~/.inkscape/preferences.xml:1: parser error : Start tag expected, '<' not found
 Emergency save activated!
 Emergency save completed. Inkscape will close now.
 ** Message: Error: Inkscape encountered an internal error and will close now.

Just remove the empty file and all is well.

At Other times Inkscape may, instead of zeroing it's config, just garble it, the following *can* be a symptom of this (a defect libglade can, too)

gdk_pango_context_get_for_screen: assertion `GDK_IS_SCREEN (screen)' failed

if you get the above message (amongst a heap of others), and have no custom settings in your ~/.inkscape/preferences.xml file, just remove it. If that doesn't fix it you have to look at GDK, pango, glade and the likes for the cause.

To protect the file from another overwrite you can write protect it with:

 chmod -w ~/.inkscape/preferences.xml

...with Inkscape garbage collection

Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS

Please fix this if you encounter it, otherwise the tilerequest will be sent to another t@h client and put extra load on the servers.

If you can't fix it, then setting 'MaxTilesetComplexity = 16300000' in tilesAtHome.conf should avoid triggering the problem.

This may show up as just a stuck process (possibly only when running headless / in console mode), and if 'ps uawwx | grep inkscape' shows a potentially stuck process, try running exactly that command manually, which may evoke the Too many heap sections error.

Fixes below, or see also this mail

Ubuntu 10.04 LTS

Fetch, build with CFLAGS=-DLARGE_CONFIG, and install libgc1c2, example method below:

apt-get install build-essential devscripts
apt-get build-dep libgc1c2
apt-get source libgc1c2
cd libgc-*
#
sed -i.orig 's:^\t./configure:\tenv CFLAGS=-DLARGE_CONFIG ./configure:' debian/rules
dch -l large Added CFLAGS=-DLARGE_CONFIG
debuild -us -uc
#
debi --upgrade
echo "libgc1c2 hold" | dpkg --set-selections

This works as of the Ubuntu 10.04 LTS server 2010-06-08

Debian

Debian users can make a new package like this if you have a matching deb-src line in /etc/apt/sources.list

aptitude install dpkg-dev build-essential ; aptitude build-dep libgc1c2 ; apt-get source libgc1c2 ; cd libgc-6.8/debian

edit the file rules and change to:

       # First build the shared library
       env CFLAGS=-DLARGE_CONFIG ./configure $(CONFIG_OPTS) --enable-cplusplus --disable-dependency-tracking \

If you dont want that the next update reinstall the "old" version you have to increase the version number with this command before you install the package:

debchange -i

Build the package:

cd .. ; dpkg-buildpackage -b -us -uc

Install the package:

 dpkg -i ../libgc1c2_6.8*.deb

Gentoo

Gentoo users can do this:

Find out which ebuild of boehm-gc you have installed:

# equery list boehm-gc
[ Searching for package 'boehm-gc' in all categories among: ]
* installed packages
[I--] [  ] dev-libs/boehm-gc-6.8 (0)

I have 6.8 installed, so I'll use that version in the example commands below. Modify them to include your version.

Create a local portage overlay dir and copy some necessary files over:

mkdir -p /usr/local/portage && echo 'PORTDIR_OVERLAY="/usr/local/portage"' >> /etc/make.conf
mkdir -p /usr/local/portage/dev-libs/boehm-gc
mkdir -p /usr/local/portage/dev-libs/boehm-gc/files
cp /usr/portage/dev-libs/boehm-gc/boehm-gc-6.8.ebuild /usr/local/portage/dev-libs/boehm-gc/
cp -R /usr/portage/dev-libs/boehm-gc/files /usr/local/portage/dev-libs/boehm-gc

Now you need to add the --enable-large-config flag to the ebuild file. Edit the /usr/local/portage/dev-libs/boehm-gc/boehm-gc-6.8.ebuild file and modify the following line to look like this:

local myconf="--enable-large-config"

If there were already other options there, leave them and just add your change.

You need to create a digest, before the ebuild will be accepted by portage:

ebuild /usr/local/portage/dev-libs/boehm-gc/boehm-gc-6.8.ebuild digest

You're now ready to recompile dev-libs/boehm-gc:

# emerge -av boehm-gc

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   ] dev-libs/boehm-gc-6.8  USE="threads -nocxx" 0 kB [1]

The 'R' flag tells you it will rebuild the package. If this command says you have no packages to (re)install, you may out of habit included the -u upgrade flag in the emerge command. Don't do that, since we want to install the same version again, this time the modified version out of the overlay, and not a new version.

When that's done, it's time to recompile inkscape:

emerge -av media-gfx/inkscape

Sit back and take a drink. The recompile will take a while.

One thing that you need to remember is that if a new version of boehm-gc is released through the official portage channels, that one will then be emerged and the older version you have in your local overlay will be ignored. If the new version still doesn't include the --enable-large-config flag, you're going to have to repeat the previous process for the new version.

References: http://gentoo-wiki.com/HOWTO_Create_an_Updated_Ebuild

...with multiple instances of tilesGen

The client must not run in several instances from the same directory, and different clients must not use the same working directory. Especially compressing and upload are not safe.

For multiprocessor machines edit the Fork config option in your tilesAtHome.conf file:

# If set to 1 or greater, it will enable forking to render tiles
# Fork = 0   -> 1 process (normal behaviour)
# Fork = 1   -> 2 processes
# Fork = 2   -> 4 processes
# Fork = 3   -> 8 processes
Fork=0

Warning: This may cause multiple instances of inkscape to run concurrently which increases memory consupmtion considerably.

...with batik installation

for problems with inkscape for the tile of big complexity, we can use batik: i've tested with debian sqeeze x64 (now stable):

aptitude install sun-java6-jre sun-java6-jdk jython libxerces2-java libxml-xerces-perl

download batik at http://xmlgraphics.apache.org/batik/ in the directory /usr/local/share

Unpack the distribution with Java’s JAR utility:

jar xvf batik-1.7.zip

modify your tileAtHome.conf:

Rasterizer=Batik
JavaHeapSize=4G
BatikJVMSize=12G

perform a test :

perl runTests.pl fonttest

it's must be ok ;-)

Ubuntu/Debian init script

This init script automatically starts the tiles@home client in the default runlevel. The software is started with loop option and performs svn update on startup.

Installation instructions

Save the script as /etc/init.d/tiles-gen-client (as root).
Save the start script as start.sh in the base path of your tiles@home client.
Save the check script as check in /etc/cron.hourly (as root).
Fix the permissions.

chmod 755 /etc/init.d/tiles-gen-client
chmod 755 .../tilesAtHome/start.sh

Edit /etc/init.d/tiles-gen-client.

vi /etc/init.d/tiles-gen-client

- Change TILES_USER to username which should start the tiles@home client. The username must exist. Do not run the client as root!
- Change TILES_HOME to the base path of your tiles@home client. I.e. if you checked out the client to /opt/tilesAtHome you would have to set TILES_HOME="/opt/tilesAtHome"
- Change NICELEVEL between -19 (high priority) and 20 (low priority) if you want to run the client with a higher priority. Values lower than 0 will make the client take serious CPU time from other programs.
- Change TIMEOUT to the time you are prepared to wait for the client to exit. After that time the client will be killed hard.
Add the script to default runlevel, but started as one of the last services and stopped first.

sudo update-rc.d tiles-gen-client defaults 99 01

Edit start.sh.
- Change LOG to a file writeable by the t@h user, so you can follow the output with tail -f. Or use /dev/null to discard.

Start script

#!/bin/sh

PERL=/usr/bin/perl
LOG=/tmp/tiles.log # must be writeable by the t@h user

# workarounds for inkscape problems when started from user environment
export LANG=C
unset DISPLAY

# set ulimits here to keep your system from thrashing

rm -f stopfile.txt > /dev/null 2>&1

svn up >> $LOG 2>&1

exec $PERL tilesGen.pl loop >> $LOG 2>&1

Init script

Attention major changes: now needs an extra script, waits for the client to exit gracefully.

#!/bin/sh
#
# tilesGen@home client software init script.
# feel free to change the script.
#
### BEGIN INIT INFO
# Provides:          tiles-gen-client
# Required-Start:    $remote_fs $network
# Required-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Starts tilesGen@home client software at boot time
### END INIT INFO

# import lsb functions for init.d scripts
. /lib/lsb/init-functions

PIDFILE=/var/run/tilesGen.pid
# tilesAtHome directory
TILES_HOME=INSERTPATH2tilesAtHome
PERL="/usr/bin/perl"
# niceness level for client software
NICELEVEL=20
# username for tiles process
TILES_USER=CHANGEME2SYTEMUSER
# timeout to wait for the client to finish on shutdown
TIMEOUT=1200
# name of start script
START=start.sh

case "$1" in
    start)
        cd "$TILES_HOME"
        log_begin_msg "Starting tilesGen@home client software..."
        if start-stop-daemon --start --quiet --pidfile "$PIDFILE" --nicelevel "$NICELEVEL" --background --make-pidfile \
                --chdir "$TILES_HOME" --chuid "$TILES_USER" --exec "/bin/sh" -- $START > /dev/null 2>&1; then
            log_end_msg 0
        else
            log_end_msg 1
        fi
        ;;
    stop)
        log_begin_msg "Stopping tilesGen@home client software..."
        cd "$TILES_HOME"
        touch stopfile.txt
        # we have to wait for $PERL because perl was exec'ed by /bin/sh.
        if start-stop-daemon --stop --quiet --chuid "$TILES_USER" --signal 0 -R $TIMEOUT --pidfile "$PIDFILE" --exec "$PERL"; then
            log_end_msg 0
        else
            log_end_msg 1
        fi
        ;;
    restart)
        $0 stop
        exec $0 start
        ;;
    *)
        log_action_msg "Usage: /etc/init.d/tiles-gen-client {start|stop|restart}"
        exit 1
        ;;
esac

Check script

#!/bin/bash
#script de verification du fichier log: si celui-ci est trop ancien,
# le service tiles-gen-client est hors service
# (le plus souvent une erreur fatale pendant le téléchargement)

#parametre du fichier a verifer
fichi=/tmp/tiles.log

#fichier log de la verification
echo `date` >> /tmp/verif.log
echo $fichi >> /tmp/verif.log

#verification existance fichier
if [ -f $fichi ]; then
  #date actuelle
  dteN=`date "+%s"`
  #date du fichier en seconde
  dteF=`date -r $fichi +%s`
  #ecart en seconde entre la date du fichier et la date actuelle
  diffSec=$((dteN-dteF))
  #si l'ecart est superieur a 4h (4*3600=14400)
  if (($diffSec > 14400)); then 
      echo "service redemare" >> /tmp/verif.log
      /etc/init.d/tiles-gen-client stop
      /etc/init.d/tiles-gen-client start
  fi
else
  echo "fichier log inexistant" >> /tmp/verif.log
 # si le fichier n'existe pas 
  #verifier le nom de fichier, si celui-ci est correct, decomanter les 2 lignes suivantes
    #/etc/init.d/tiles-gen-client stop
    #/etc/init.d/tiles-gen-client start
fi

Troubleshooting

Look at $LOG to see command output. Is the file/directory writeable?
Waiting for the process to finish can take a long time.
join the mailinglist.

Ubuntu/Debian deb package

Based on the current version of the init.d script I had build a deb package skeleton. It can generate an installable deb package of the tiles@Home client.

Download: http://www.megaupload.com/?d=1W02PSRO

tar -xjf tiles-at-home-client.tar.bz2
cd tiles-at-home-client
dpkg-buildpackage

cd ..
dpkg -i tiles-at-home-client_20090916-1.deb

TODO

Service does not shut down cleanly. Perhaps a bug in init.d script with perl parameter.

Tiles@home/Install Guide

Prerequisites

Software

Perl modules

Installation instructions

Arch Linux

Debian 4.* Etch, 5.* Lenny, 6.* Squeeze (stable) or wheezy (unstable)

Etch-only:

FreeBSD

Gentoo

openSUSE

Mac OS X 10.5

Ubuntu

Fedora

Historical Fedora Problems

Fonts

Non-latin fonts

Checking the font setup

Download the program

Configuration

Linux / Mac OS X / *BSD

Windows

Running the program

Checking your status

Debugging the program

Stopping the program

Passwords Required

Known Issues

...with libGD aborting

...with Math::Vec

...without DISPLAY set

...with DISPLAY set to an inaccessible display

...with Inkscape preferences files

...with Inkscape garbage collection

Ubuntu 10.04 LTS

Debian

Gentoo

...with multiple instances of tilesGen

...with batik installation

Ubuntu/Debian init script

Installation instructions

Start script

Init script

Check script

Troubleshooting

Ubuntu/Debian deb package

TODO

Navigation menu

Search