Osm2pgsql/benchmarks/R610 osm12 Proxmox ZFS
Hardware:
- Dell R610 server
- 2 x Xeon X5670 (6 cores@2.93GHz each +HT)
- 192GB RAM @1333MHz
- Samsung 3.6TB NVMe SSD, using ZFS + lz4 compression, 16K recordsize
Software:
- OS: proxmox 6
- import in LXC container running Ubuntu 20.04 with 160GB RAM
- Postgres 13.2 / postgis 3.1
- osm2pgsql 1.4.1
osm2pgsql --create -C 65000 --number-processes=12 -k -m -G -s -S ./osm2pgsql-choosen.style -d osm -H /var/run/postgresql/ --keep-coastlines --flat-nodes flat-nodes.raw --middle-way-node-index-id-shift 5 -r pbf ../planet-210322.osm.pbf
2021-04-04 21:44:39 osm2pgsql version 1.4.1 (1.4.1)
2021-04-04 21:44:39 Database version: 13.2 (Ubuntu 13.2-1.pgdg20.04+1)
2021-04-04 21:44:39 PostGIS version: 3.1
2021-04-04 21:44:39 Node-cache: cache=65000MB, maxblocks=1040000*65536, allocation method=11
2021-04-04 21:44:39 Setting up table 'planet_osm_point'
2021-04-04 21:44:39 Setting up table 'planet_osm_line'
2021-04-04 21:44:39 Setting up table 'planet_osm_polygon'
2021-04-04 21:44:41 Setting up table 'planet_osm_roads'
2021-04-05 02:43:19 Reading input files done in 17918s (4h 58m 38s). 30.2/s)
2021-04-05 02:43:19 Processed 6794233629 nodes in 1323s (22m 3s) - 5135k/s
2021-04-05 02:43:19 Processed 752633951 ways in 9433s (2h 37m 13s) - 80k/s
2021-04-05 02:43:19 Processed 8804337 relations in 7162s (1h 59m 22s) - 1k/s
2021-04-05 02:43:19 Clustering table 'planet_osm_polygon' by geometry...
2021-04-05 02:43:19 Clustering table 'planet_osm_point' by geometry...
2021-04-05 02:43:19 Clustering table 'planet_osm_roads' by geometry...
2021-04-05 02:43:19 Clustering table 'planet_osm_line' by geometry...
2021-04-05 02:43:19 Building index on table 'planet_osm_rels'
2021-04-05 02:43:19 Building index on table 'planet_osm_ways'
2021-04-05 02:55:54 Creating geometry index on table 'planet_osm_roads'...
2021-04-05 02:58:14 Done postprocessing on table 'planet_osm_rels' in 895s (14m 55s)
2021-04-05 03:04:26 Creating osm_id index on table 'planet_osm_roads'...
2021-04-05 03:04:59 Analyzing table 'planet_osm_roads'...
2021-04-05 03:05:01 All postprocessing on table 'planet_osm_roads' done in 1302s (21m 42s).
2021-04-05 03:18:19 Creating geometry index on table 'planet_osm_point'...
2021-04-05 04:19:37 Creating geometry index on table 'planet_osm_line'...
2021-04-05 04:24:17 Creating osm_id index on table 'planet_osm_point'...
2021-04-05 04:27:35 Analyzing table 'planet_osm_point'...
2021-04-05 04:27:37 All postprocessing on table 'planet_osm_point' done in 6258s (1h 44m 18s).
2021-04-05 06:22:44 Creating osm_id index on table 'planet_osm_line'...
2021-04-05 06:28:42 Analyzing table 'planet_osm_line'...
2021-04-05 06:28:49 All postprocessing on table 'planet_osm_line' done in 13530s (3h 45m 30s).
2021-04-05 06:55:30 Creating geometry index on table 'planet_osm_polygon'...
2021-04-05 09:26:42 Done postprocessing on table 'planet_osm_ways' in 24203s (6h 43m 23s)
2021-04-05 11:46:37 Creating osm_id index on table 'planet_osm_polygon'...
2021-04-05 11:57:56 Analyzing table 'planet_osm_polygon'...
2021-04-05 11:58:05 All postprocessing on table 'planet_osm_polygon' done in 33286s (9h 14m 46s).
2021-04-05 11:58:05 node cache: stored: 6794233629(100.00%), storage efficiency: 83.66% (dense blocks: 932440, sparse nodes: 241372767), hit rate: 100.00%
2021-04-05 11:58:10 osm2pgsql took 51211s (14h 13m 31s) overall.
ERROR: relation "planet_osm_ways_idx" does not exist
CREATE INDEX
CREATE INDEX
CREATE INDEX
real 872m56,506s
user 256m51,344s
sys 19m58,967s