User:Mmd/Overpass API/Raspberry

Overpass API · Language reference · Language guide · Technical terms · Areas · Query examples · Sparse Editing · Permanent ID · FAQ · more · Web site
Servers status · Versions · Development · Technical design · Installation · XAPI compatibility layer · Public transport sketch lines · Applications · Source code and issues

Overpass turbo · Wizard · Overpass turbo shortcuts · MapCSS stylesheets · Export to GeoJSON · more · Development · Source code and issues · Web site

Overpass Ultra · Examples · Overpass Ultra extensions · MapLibre stylesheets · URL Params · more · Source code and issues · Web site

Overpass API on Raspberry Pi 2

Raspberry Pi 2, Model B

The other day I thought about running a small Overpass API instance on a Raspberry Pi 2. It's not exactly the kind of hardware you would expect for an Overpass API instance. To have all required packages available, we use a recent Raspbian image. For the Overpass API, I used my Overpass API test branch. This includes all available compression options (to keep the space consumption low) and the amazingly fast lz4 compression (to keep cpu consumption low as well).

Let's see how far we can get with it...

Compiling

For compiling and installing Overpass API, follow the steps described on the Install page. Follow install option 2 and make sure to use the alternate GitHub Download URL. Relevant branch to check out is the test754 branch (git checkout test754) - or - use the pre-release version 0.7.54_mmd available on Github.

This test branch has additional dependencies on top of the ones listed in the official documentation:

sudo apt-get install liblz4-dev libpcre3-dev

To reduce compilation time, I commented out the test sub-directory "test-bin" in src/Makefile.am (line: # SUBDIRS = test-bin).

Compiling with highest optimization level as described in the Installation Guide triggered some random crashes during later execution. As a work around, I used the following more conservative settings:

../src/configure CXXFLAGS="-Og -g -fno-omit-frame-pointer -march=native"

Loading the database

In addition to the usual installation instructions, we have to cater for the rather limited main memory by setting the flush size to a small number:

osmconvert extract_file.pbf --out-osm | ./update_database --db-dir=/home/pi/overpass/db --meta --flush-size=1

Populating the database with a 43 MB .pbf test file resulted in a 600 MB Overpass API database size (including areas, meta, but no attic data). That's faily ok for a small instance.

Note: Overpass API doesn't handle .pbf out of the box, that's why you need some tool like osmconvert to produce .osm file format.

Test run

Here's a small test run:

pi@raspberrypi ~/osm-3s-dev-version/build/bin $ ./osm3s_query --db-dir=/home/pi/db
encoding remark: Please enter your query and terminate it with CTRL+D.
node[~"."~"."];out count;
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base=""/>

  <count total="88851" nodes="88851" ways="0" relations="0" areas="0"/>

</osm>

real	0m3.734s
user	0m3.490s
sys	0m0.240s

Not too bad for an 900MHz CPU... By the way, the same query (limited to the same bbox) takes about 700ms on the production hardware, though running a planet size database.

Dependencies

Let's take a look at some of the dependencies: they include lz4 as well as pcre, both which are not part of the official Overpass API installation. Installation was fairly easy though, as all packages are available in Raspbian out of the box.

pi@raspberrypi ~/osm-3s-dev-version/build/bin $ ldd ./osm3s_query 
	linux-vdso.so.1 (0x7ee4f000)
	/usr/lib/arm-linux-gnueabihf/libarmmem.so (0x76f37000)
	libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0x76efb000)
	liblz4.so.1 => /usr/lib/arm-linux-gnueabihf/liblz4.so.1 (0x76ee4000)
	libpcre.so.3 => /lib/arm-linux-gnueabihf/libpcre.so.3 (0x76e71000)
	librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0x76e5a000)
	libexpat.so.1 => /lib/arm-linux-gnueabihf/libexpat.so.1 (0x76e28000)
	libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0x76d4b000)
	libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x76cd0000)
	libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x76ca3000)
	libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76b66000)
	/lib/ld-linux-armhf.so.3 (0x54b1b000)
	libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x76b3e000)

Conclusion

Well, it's not terribly fast, but it sort of runs on a very small extract. I really don't see much of a use for such an instance, but it's interesting to see at least some results after all. Mission accomplished.

Have fun!