GSoC project: Fast Track Deployment of Customised OSM Tile Server
"Fast Track Deployment of Customised OSM Tile Server" is a Google Summer of Code/2011 project by Parveen Arora also named as Mera Map at the early design stage.
It will be one complete package that will automatically install all the components required to set up your own customized server which will update automatically with sync. of OpenStreetMap data for an area specified by implementor, It will also suggest him/her and will also give a option to choose default options i.e weekly bases , with easy user interface and browser based customization of Map Style, adding of Icons, local language and additional search feature.
Overview
OSM Tile Server is Map Server which can run on any system after installation and all the Setup and can also provide Map Services. It takes the data of OpenStreetMap which is freely available and make its local copy, thus empowering any citizen to have his/her customised map, to meet his/her requirements by making it to display information as relevant to his project requirement.
I am proposing to make a one complete package named Fast Track Deployment of Customised OSM Tile Server. That will automatically install all the packages required to set up own customised server with a simple install script which will update the data automatically with synchronising OpenStreetMap data for an area specified by implementor, It will also suggest him/her with according to his/her system configuration and will also give a option to choose default i.e weekly bases , with easy user interface and browser based customisation of Map Style (Colours, Ways, Selection of ways), adding of Icons, local language and additional search feature.
Problem Description
Installation of OSM Tile Server has never been an easy task for anyone who wants to set the customised server to meet his/her own needs, and want to provide OSM Services. Although if there is anyone familiar with the installation procedure he/she has to go with the long installation procedure, with including a large number of packages and a lot of other dependency issues arises while installation.
Objective
This will be a a one complete package , That will automatically install all the packages with Install Script required to set up own customised server which will update automatically with sync. of OpenStreetMap data for an area specified by implementor, It will also suggest him/her and will also give a option to choose default options i.e weekly bases , with easy user interface and browser based customisation of Map Style, adding of Icons, local language and additional search feature.
Project Details
Fast Deployment of Customised OSM Tile Server will include easy installation procedure with a simple command i.e. one will be able to install it using simple install script on his Linux system preferably ubuntu, with customising map style and other attributes of map through easy user interface (preferably browser based) and will get option to add his/her own language, and icons. This will be automatically update area with using the data of OpenStreetMap as decided by the user (after selecting area in browser) and will also give and advice based on resources of implementor with a default option. It will also ask user to add his custom icons to on the map if they are not present on OpenStreetMap and he/she want to show them in his/her own map. A search option will also be installed with that map which will search for the exact location of map and will the show more information about that after clicking.
OSM Tile Server's Installation package will include Installation of PostGreSQL database with PostGIS extension and editing configuration files in user friendly manner, osm2pgsql, loading planet file into the database with osm2pgsql, installation of Mapnik libraries with its tools and installation of osmosis.
While installing it manually requires a lot of packages to be compiled and install separately and changes in files. So It will do user's job easy.
To start with it will be made for Linux operating system preferably ubuntu, Because most of people now using ubuntu and people on the developers mailing list of osm also suggested me to go for it with ubuntu.
End User Functional Specification
User will be provided with the Installer interface which will ask him for all the customisations to do on his map, or to skip all the options and just to set a simple map like OSM. The software can be provided in form of CD or as a software package. In customisation it will ask for Area Selection, Update frequency with giving some suggestions on the basis of his hardware, Colours for maps, zoom level upto which he wants to show the map, to show the selective roads on different zoom levels, adding of local area icons and replacing of default icons if one wants to show his own icons, their own language fonts.
End Users
Schools, Colleges, Police, Defence, Security Agencies and Companies will be the users of our product. Who will be having their dedicated servers. We will also include the normal home users in our next version of software who will be able to use this with his hosting account.
Target audience type:
Users with dedicated servers will be able to use it. Users with virtual server will be able to use it. Home users as a non functional requirement, we will add this types of users in our next version.
Hardware Requirements:
Bare minimum hardware requirements:
| RAM | Disk |
|---|---|
| 1 GB | 200 GB |
Recommended minimum hardware requirements
| RAM | Disk |
|---|---|
| 4 GB | 1 TB GB |
Potential production hardware requirements
| RAM | Disk |
|---|---|
| 64 GB | Many of TB of 15,000 RPM RAID |
I have got hardware requirement table from the wiki of OpenStreetMap.
Expected Features:
- Easy Installation with installer.
- Easy area setup.
- Automatic Map Updation and rendering
- Search for particular type of tags in selective area. e.g. schools in city, banks in cit etc.
- Editing Features will be:
- Editing of map style (Colors of Map, Colors of roads)
- Use of Local Icons (One will be able to replace the default icons, and will be able to new icons if required to him)
- Use of Local Language fonts (One can render the map in his own language, If that language is available there on the map)
Advantages:
- Every one who will use this, will be having his own map application running for special purpose.
- Enrichment of Maps: This will be the main advantage that it will leads to a lot of enrichment of OpenStreetMaps. Because organisation using his own map application for his own purposes will always want to have a good map to use and to show. So they will try to make their area best and will take care of it, In result Enrichment of OpenStreetMaps.
- It will definitely save a lot of time which it takes while manual installation every time we install it in case of any failure. Then you will need not to be worry about that.
- It will attract more users and new developers towards OpenStreetMap to as a data collectors, contributors, and developers.
- All the OpenStreetMap Users will got a strong reason to promote OpenStreetMap and to claim that how every one can use the data of openstreet map and how he can take benefits from it.
- A mapping website or a mapping application will be set up in less time with zero cost.
Future Scope:
I aimed it to make the hallmark for the OpenStreetMap, I want everyone promoting OpenStreetMap and working on it should recommend user to use data of openstreetmap. This can be the very useful package or technique that will lead to enrichment of maps. It can have the features of distance measuring functionality, Open Source Routing Machine etc. to integrate with it in future.
Design Overview
- Script for Installing the Server at back-end and Interface to run that script at front end.
- Selection of area of Implementer's choice with a default option with interface showing a Map to select the area of interest to set up map application.
- Selection of the zoom level upto which user wants to setup the map
- Automatic updation and rendering setup
- Setting of renering frequency time, by asking user and also suggesting him based on his hardware with a default option of week or month.
- Adding search facility, which will search particular type of tags on the map as explained above in feature’s point 4.
- To make a style editor which will be able to edit the map colors and roads colors
- To select which roads to display according to requirement and which to not.
- To give a option to add new icons and replace the previous ones.
- To give option to use local language fonts.
Technology It will use
Operating System: Ubuntu/Linux 11.04
Programming Language: Python
Database: PostGreSQL+PostGIS
Softwares: subversion, autoconf, screen, munin-node munin htop, osm2pgsql,osmosis, Mapnik with its all dependencies, osmosis with all dependencies, mod_tile and all other software and dependencies that will be required.
Project Hosting Sarovar and GitHub
Weekly Reports
Week 1
(May 26, 2011 to June 3, 2011)
In the very first week of starting of project I was having My final exams of Semester on 23rd and 25th, So I started working from May 26, 2011.
- First of all written the wiki and asked OSM Dev Community to give suggestions and comments about project, In result I got a very good response and good number of suggestions. All the suggestions are evaluated and and responded accordingly. Discussion Archive (Wayback Machine)
- Learned to work on Doxygen, Doxygen is a automatic Documentation Genrator.
- Followed some good Python Tutorials for refreshing.
- Selected the Short Name for Project, A lot of brainstorming is done, Suggestions were taken from the Indian Community. Finally I have selected "MeraMap" as my project's name. I liked it because it is a simple small word, easy to remember and easy to pronounce. In includes Mera which refers to the community's own Map, and Map already has a well defined definition i.e A diagrammatic representation of an area of land or sea showing physical features, cities, roads, etc. I have joined these two words, to make it just single word, And with this also eliminating the single meaning of Mera, because its MeraMap now. Name Choices
- Decided to Host My project for frequent code updates on Github, and for versions release sarovar.org will be used. So Set up git system and created repository. MeraMap Repository on Github.
Week 2
(June 4, 2011 to June 11, 2011)
- Tried Rendering System Rendering At Home, founded bugs and reported, But it has not worked at first as some files are missing at the places where path is given.
- Designed Diagram of complete working Model for MeraMap. Working Model
- Designed Diagram for End User Interface of MeraMap. End User Interface
- Tested the generation of Tiles of pre-built system with changing its icons with default icons.
- Set up OSM Tile Server Manually, with fixing some problems faced during the Installation, In result very cleared with manual Installation and got to about how to solve the problems faced during manual Installation. Image Generated before rendering of tiles can found here. Image Genrated
Week 3
(June 12, 2011 to June 18 2011)
- Made the Gantt Chart of Project Time Line and Progress report using Gnome Project Planner.
- Made some changes in the genrate_tiles.py, generate_image.py, and set-mapnik-env according to requirement and committed on Github.
- Written a single script to Install Tile Server. Script
- Improved the Map of my own area Uggi
- Organised the Online Mapping Event named Hack The Map
Week 4
(June 19, 2011 to June 25, 2011)
- Discussion with OSM Development Community about World Boundaries that they are required for small areas or not. Discussion Archive
- Learned how to make a debian package and made a simple package at the first which would be able to run the script written in that.
- Learned about building of PPA and then how to apt-get the own package.
- Made a Test Debian Package which can Install all the required software and dependency.
- Did manipulation on configuration files with sed, after learning sed.
- Submitted Request to approve project from Sarovar.org, where the versions of software will be released.
Week 5
(June 26, 2011 to July 3, 2011)
- Tried different graphical software to build debian package e.g Ubucompilator, Debianpackagemaker, Gift Wrap. But they didn't worked, because all of these are at the very initial state of development.
- Made patch to apply on postgresql.conf, which will alter the user's PostgreSQL settings with required settings for Tile Server. PostgreSQL.conf Patch, and released it on github.
- Set up Website for Project on Github under different branch of same repository. MeraMap Website
- Solved the problem of to met with dependencies, that are required to Install along with the debian package.
- Project accepted to host on Sarovar.org, Set up project repository there along with the set of website on the same host.MeraMap on Sarovar
- Learned to use rsync tool, and released Version 0.1 at Sarovar.org
- Made a Tested Debian package to Install all the required software and dependency to set the Rendering system, but it doesn't include the Osmosis.
Week 6
July 4, 2011 TO July 10, 2011
- Written the All Scripts required to make Debian package. Scripts
- Fixed the Errors and bugs in the code.
- Added comments to most of the files.
- Selected GPL V3 License for the Software, and released it on github along with code. LICENSE
Week 7
(July 11, 2011 TO July 17, 2011)
- Removed two extra scripts, because Debian package directory structure itself can do the work of that scripts.
- Fixed some bugs in all the scripts.
- Openlayer files added to show the map.
- Prepared Installation Guide and added in the README File
- logfiles directory added to save log files separately.
- Apache2 added as a command to install if not already Installed.
- Added temporary simple Debian package which only installs all the defined dependencies and scripts run differently.
Week 8
(July 18, 2011 TO July 24, 2011)
- Revised the list of all the dependencies, some extra dependencies removed and apache2 added as dependency in the control file.
- Condition added to check the tiles path, before giving the Congratulations message.
- Resolved Database authentication issue.
- Comments added to postinst (Master Installation script)
- Directory recursion issue resolved, which was generating due to link in link in form of Infinite loop. Issue resolved with applying if condition to remove it if link already exists.
- Final Package to Install all the components and sample of map Ludhiana city made work and ready with its installation documentation.
Week 9
(July 25, 2011 TO July 31, 2011)
- Issue of showing map on browser even after generating tiles fixed, with some changes in index.html file.
- Made Sample package for all ubuntu versions 10.04, 10.10 and 11.04, all are tested.
- Made different debian for installing the shape files differently.
- Made the MeraMap.deb without shape-files.
- Tested the packages for all Ubuntu 10.04 or greater.
Week 10
(August 1, 2011 TO August 7, 2011)
- Made the plan to work on customisation part.
- Root authentication issue resolved.
- OpenLayers Package added to the sample map package to run it without internet connection.
- Proposed Meramap Server Configuration System
- Get a Head start to work on customisation of map with the help of my mentor Dr. Graham Jones.
Week 11
(August 8, 2011 TO August 14, 2011)
- Done with the parsing of meramap.json file to python to get the value of bounding box from it.
- Made a script to generate to generate another python script according to the new bounding box values as selected by user.
- Made that file output.py with variables to take the the value by reding from json files and then to read it in the json files.
- Added zoom levels, so that user can also give zoom levels of his choice to generate the map.
- Made the script DownloadOsmData.py to download the osm data by using jaxpi.
- Written scripts for to include in sutomise_map.
Week 12
(August 15, 2011 TO August 22, 2011)
- Tried to run script through cgi and ran some examples.
- Solved some issues to run the main script through cgi, but some are still left.
- Stuck with the security issue of using www-data user to do everything to install the map.
- Worked on the automatic Map updation procedure, and Installed Minutely Mapnik Manually.
- Written the scripts for automatic osmosis installation and map updation procedure.
- Made output.py able to generate one more file to area.txt with new bounding box selected by the user which will
- Fixed the issues of path and variables and message display message of processing.
- Made a directory structure diagram of code. [Image][text]
- Finally made the Installation guide for software in README file.
Difficulties Faced
As I was doing my project, I was getting different difficulties everyday, and most of them are resolved on daily bases. I am very thankful to my Mentors Dr. Graham Jones and Dr. H.S Rai who have always given me helping hand with right suggestions and put me on right track. At the start I project started smoothly but in the 3rd and 4th week I stuck to some points which made me slow to meet project timeline. But In the last two weeks I got my speed back and project on the track and meet the timeline upto mid term evaluations.
Error in Loading OSM Data with osm2pgsql
After following all the procedure manually to set OSM Tile Server, I was getting error at the last step and later I came to know that data in the pgsql is not loaded correctly, so I loaded data into pgsql using osm2pgsql and make run the OSM Tile Server manually.
Debian Package Building
I have never built any debian package before, do it was a new thing to me. So It has taken time for me to learn and built the Debian package.
Dependency Errors for Debian Package
This was the time I struggled most, because software components require lot of dependencies and some software that need to be installed before running Debian package, and I was getting problem in defining the exact dependency. So It also taken the most of the time. But at last I was successful and dependency problem resolved.
Code Commits
I have made the regular commits of my works and code on Github in Project's repository. All my code can be found on here.MeraMap on Github
--Parveen Arora 18:40, 10 July 2011 (BST)