bulk_upload.pl

For current work, see the python script bulk_upload.py, which is compatible with API 0.6

Software described on this page or in this section is unlikely to be compatible with API v0.6 (current version) deployed in 2009.
If you have fixed the software, or concluded that this notice does not apply, remove it.

bulk_upload.pl was a script for bulk importing OSM data via the API. It is still available in SVN:

https://svn.openstreetmap.org/applications/utils/import/bulk_import/

Old details of the script (historical interest only)

For uploading to API v0.6 you will need to use the python script bulk_upload.py The following details are of historical interest only, unless a perl enthusiast wanted to update this.

(old) features:

• Can continue even when it receives an error from the server
• Reads both JOSM file format and osmChange
• Progress counter and time to complete
• If the program is interrupted for any reason at will pick up where it left off
• Support for placeholders (via negative IDs)
• Low memory usage, independent of number of objects
• Can add tags to every object uploaded
• Exit code is determined by success of upload

This program has been used for the AND and TIGER imports and is regularly used for coastline uploads. It's support API v0.5. (though most of the work is in the [library].

Imports and automated edits should only be carried out by those with experience and understanding of the way the OpenStreetMap community creates maps, and only with careful planning and consultation with the local community.
See Import/Guidelines and Automated Edits code of conduct for more information. Imports/automated edits which do not follow these guidelines might be reverted!

Quick start

To upload a file created by JOSM, use:

./bulk_upload.pl -f -l -i input.osm -u username -p password

The -f is needed to keep the program going even when it gets Precondition failed errors and like. This is because the changes in JOSM files are not in order. This together with the -l option will keep the program going until it has uploaded as much as it can.

If you have an osmChange file then you can use osmosis to sort the changes first. This will reduce the overall time to complete, since the program will only have to do one pass. In that case you can drop the -f -l.

Note: The API is not 100% reliable, you occasionally get timeouts and 500 errors. The uploader is clever enough to notice this and not upload incomplete objects. However, it does mean that in practice you will need to specify -f -f -l on any file of non-trivial size.

Note: The code uses the [library] so you should have downloaded SVN to include at least that. Easiest is to include anything below the utils directory.

Command line

   bulk_upload.pl -i input.osc -u username -p password [-a http://server/api/version]
 
     -i input.osc                  File read for changes
     -u username                   Username for server
     -p password                   Password for server
     -a http://server/api/version  Server to upload to
                                   default: https://www.openstreetmap.org/api/0.5/
     -f                            Continue even if server returns an transient error
                                   (e.g. connection timeout)
     -ff                           Continue no matter what
     -n                            Dry-run (just parse, don't do anything)
     -l                            Keep redoing file until nothing more can be done
                                   Useful if file is not sorted
     -t key=value                  Add given tag to every uploaded object
     -3/4/5                        Select API version
     -v                            Verbose output
     -x                            Extract unuploaded objects in JOSM format