Planet.osm

From OpenStreetMap Wiki
(Redirected from Extracts)
Jump to navigation Jump to search

Planet.osm is the OpenStreetMap data in one file: all the nodes, ways and relations that make up our map. A new version is released every week. It's a big file (on 2024-11-01, the plain OSM XML variant takes over 1977.0 GB when uncompressed from the 143.6 GB bzip2-compressed or 77.4 GB PBF-compressed downloaded data file).

There are also files called Extracts which contain OpenStreetMap Data for individual continents, countries, and metropolitan areas.

Format

The planet server offers five different dumps of OSM data

  • planet data,
  • planet data with history,
  • notes,
  • changesets, and
  • changesets with discussions.

This page is about the first two. They are both available in PBF or OSM XML. PBF is a more compact format that is faster to process and should be used whenever possible.

For an overview over all osm file formats and conversion tools have a look at OSM file formats.

If you are using traditional GIS tools you may want to look at Processed data providers.

Downloading

See downloading data for tools to make downloading easier, including an option to use all mirrors at once, automatically do md5 hash validation, and other features.

Do not attempt to download the planet in a web browser. Instead, use command-line tools to download. Most Linux systems will come with cURL, which can be used

curl -OL https://planet.openstreetmap.org/pbf/planet-latest.osm.pbf

If you have the AWS CLI tools installed, you can download directly from AWS in the EU Central 1 and US West 2 regions. This is faster than downloading normally[1]. The AWS bucket layout differs from the historical planet.openstreetmap.org layout, so you can see where the files are with

aws s3 ls --no-sign-request s3://osm-planet-eu-central-1/
aws s3 ls --no-sign-request s3://osm-planet-us-west-2/

If you want to use BitTorrent to download the files, you can do so. On Linux machines, the easiest way to do this is with the aria2c command after finding the appropriate .torrent file.

aria2c --seed-time=0 https://planet.openstreetmap.org/pbf/planet-latest.osm.pbf.torrent

BitTorrent does not save the OSM servers bandwidth, as the s3 buckets have unlimited bandwidth, but it may be faster than the AWS CLI in rare cases.

RSS feeds for torrents are available on the planet server for automatic torrenting.

Planet.osm mirrors

OpenStreetMap data is mirrored to a large number of hosts. It used to be necessary to download mirrors to conserve OSM bandwidth, but that is no longer necessary. If you want to use mirrors, the easiest way is to use the torrent files which will automatically download from multiple mirrors in parallel.

If downloading from a mirror directly, prefer HTTPS over HTTP or FTP.

Mirror Available Planet data Network protocols
Preferred URL Loc Updated Diffs .osm.pbf .osm.bz2 HTTP HTTPS FTP RSYNC Torrent IPv6
planet.openstreetmap.org/ (original source)
GB
(Source) yes yes yes no yes no yes yes yes
ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/
DE
Daily yes yes yes no yes yes yes yes yes
download.openstreetmap.fr/
FR
Minutely yes no no yes yes no no no yes
planet.maps.mail.ru/
RU
Minutely yes yes yes yes yes no yes yes no
mirror.marwan.ma/openstreetmap/
MA
Minutely yes yes yes yes yes no yes yes yes
ftpmirror.your.org/pub/openstreetmap/
US
Daily yes yes yes yes yes yes yes yes yes
ftp.fau.de/osm-planet/
DE
Daily no yes no yes yes yes yes yes yes
mirror.init7.net/openstreetmap/
CH
Daily no yes yes yes yes no yes yes yes
ftp.snt.utwente.nl/pub/misc/openstreetmap/
NL
Weekly no yes yes yes yes yes yes no yes
ftp.spline.de/pub/openstreetmap/
DE
Weekly no yes yes yes yes yes yes yes no
ftp.osuosl.org/pub/openstreetmap/
US
Weekly no yes yes yes yes yes yes no yes
download.bbbike.org/osm/planet/
DE
Weekly no yes yes no yes no no no no
downloads.opencagedata.com/planet/
DE
Weekly no yes yes yes yes no no yes no
ftp.nluug.nl/maps/planet.openstreetmap.org/
NL
Weekly no yes yes yes yes yes no no yes
planet.passportcontrol.net/pbf/
JP
Weekly no yes no no yes no no no no
archive.org/details/osmdata (Internet Archive, available under various items. Instructions on searching items is available in the "About" tab. Not part of official torrent swarm)
US
Weekly yes yes yes yes yes no no yes no
osm.cquest.org/torrents/ Since 2021-04-21 does not act as (hidden) webseed anymore, and publishes regular torrents
FR
Weekly
but in test and manual update.
no yes no yes yes no no no no
planet.osm-hr.org Last 30 days
HR
Daily no yes yes yes yes no no yes yes

Extracts

OSM data for the full planet is large. If you only want data for an area, you can download an extract that has that data only, extracted from the full planet file. Extract providers may also offer processed data in other formats.

Worldwide extract sources

Mirror Area Updated Diffs HTTPS Metadata included
www.overpass-api.de Overpass API excerpts up to ~300MB uncompressed Minutely yes, since v0.7.50 yes Optional
download.openstreetmap.fr/ Minutely diffs and daily extracts in PBF:
  • Entire continents
  • Many countries in all continents
  • Sub-country admin regions for several countries
Minutely yes yes
download.geofabrik.de Daily extracts in PBF stripped from personal metadata (user names, user IDs, changeset IDs):
  • Entire continents
  • Most countries
  • Sub-country admin regions for Brasil, Canada, France, Germany, Italy, Japan, Poland, Russia, UK, US
Daily yes yes Version and timestamp only
osmtoday.com Over 1000 sub-regions in Europe, Asia, South and North America, Africa and Australia. Daily no yes All fields
osm.download.movisda.io more than 3000 precisely cut administrative areas worldwide as PBF Weekly no yes Version only
osm-internal.download.geofabrik.de Same as download.geofabrik.de but with full personal metadata:
  • Weekly full history extracts for all regions mentioned above (no diffs)

Note: Login with OSM account required. Files are subject to data protection regulation.

Daily yes yes All fields
download.bbbike.org/osm/ Different extracts offered:
  • more than 200 cities and regions worldwide
  • extract your own individual area

Note: Extracts are not suitable for editing OSM, because they omit metadata such as version number. See the FAQ.

Weekly no yes No metadata
protomaps.com/extracts Extracts up to ~100 million nodes in PBF
  • extract rectangular area or draw polygon
  • updated minutely
Minutely no yes Version for all objects; Timestamp for ways, relations and tagged nodes
interline.io/osm/extracts Over 200 extracts for city and metropolitan regions:
  • inspired by Michal Migurski's Extractotron and Mapzen's Metro Extracts
  • extract formats include OSM PBF, GeoJSON, and GeoJSONL
  • powered by PlanetUtils, a "high level" library that makes use of Osmosis, OSM C tools, and Osmium among other open-source components
Daily no yes

Regional extract sources

Mirror Area Updated Diffs HTTPS Metadata included
osm.kewl.lu/luxembourg.osm/ Luxembourg (beta) Hourly Yes but not Osmosis/Pyosmium compatible directory structure yes Partially
planet.osm.ch Swiss extract from the Swiss OpenStreetMap association. Including all lakes crossing the Swiss boundary and a few km of the surrounding countries. Hourly yes yes All fields
osm.fit.vutbr.cz/extracts/czech_republic/ Czech Republic Daily no yes version and timestamp only
data.osm-hr.org Croatia, Slovenia, Bosnia and Herzegovina, Montenegro, Macedonia, Serbia, and Kosovo, Albania, Bulgaria, Hungary, Romania
  • daily extracts in PBF
  • daily Garmin maps
  • daily Osmand maps
  • Croatia archive
  • Croatia diffs
  • planet thematic extracts
Daily yes yes all fields
osmit-estratti.wmcloud.org Italy; daily extracts for Italian regions, provinces and municipalities; supported file formats: PBF, GeoPackage and OsmAnd OBF (Regions only). Daily no yes Partially
osm.kcwu.csie.org/download/tw-extract/ Taiwan; only o5m format Daily Yes but not Osmosis/Pyosmium compatible directory structure yes all fields

Global thematic extracts

  • OpenStreetMapData provides shapefiles of global coastlines, land polygons or water polygons.

Update frequency

A new version of planet.osm is released weekly (currently on Fridays). We have these, going back to the start of April 2006. Due to the license change to ODbL on September 12th 2012, older CC-BY-SA licensed planet and diff files are only available in a dedicated folder with CC-BY-SA licensed data.

The size of a planet.osm file can be seen in the header information on the planet download page. Planet files are very large (tens of GBs) so please check the size before starting to download.

The weekly dump normally starts at around 01:10am UK time on Monday morning and is guaranteed to contain all updates prior to that time. The dump is constructed from a database dump using conversion software, and the result should ensure referential integrity. Please note that this doesn't always apply to extracts. The result is usually ready on Fridays.

Note: The timestamp of the file is recorded in it like this: <osm version="0.6" generator="OpenStreetMap planet.c" (...) timestamp="2013-01-02T01:10:14Z"> it will be useful to decide what "diffs" to use if you want them :

Additionally we offer regular diffs. These are produced as daily (under the 'daily' subdirectory), hourly and minutely diffs. For minutely diffs, we're using osmdbt, hourly and daily diffs are generated using Osmosis by combining multiple minutely diff files for a given time span[2]. Diff files can be used to reconstruct the full dataset (see examples at OSM Wiki pages Osmosis and osmupdate). Since these only contain the differences, they are much smaller files; A daily diff is generally about 40-80 MB, but can easily grow to 300-400 MB (both compressed) in exceptional cases. For more information please see Planet.osm/diffs

License

Planet files remain licensed under the same license as the master OpenStreetMap geo-database from which they are extracted - currently this is the Open Database License. Planet files from before September 12, 2012 have a Creative Commons Attribution-ShareAlike 2.0 license.

Source code of planet.osm.org

Code powering primary website providing planet.osm is at https://github.com/openstreetmap/chef/tree/master/cookbooks/planet/files/default/html and https://github.com/openstreetmap/chef/tree/master/cookbooks/planet

  1. EC2 metadata lookup might slow things down, see https://github.com/aws/aws-sdk-cpp/issues/1511 for discussion
  2. Once https://github.com/openstreetmap/osmdbt/issues/32 has been implemented, osmosis will no longer be used.