This post is about some new tools I and others have made to facilitate community-based import of data into OpenStreetMap. These tools cover converting .csv files to .osm files and splitting .osm files for the Tasking Manager, and I am writing this post hoping they can be of help to others. The code is available in the mtl2osm GitHub project (“mtl” stands for “Montreal”).

(Please note: mostly if not all links here are to French web pages.)

The Montreal community import process

Here in Montreal we are slowly preparing to import open data provided the city. Said data is licensed under CC-BY 4.0, and we got the required legal authorization from the City to import them into OpenStreetMap.

Our import process is documented here. We have not started any data import yet. We are currently in the process of getting feedback from the local community through the OpenStreetMap Montréal user group.

Tools & dataset

I recently made a presentation about the various datasets provided by the Ville de Montréal at a Montréal Python meeting. The presentation was recorded and can be viewed on YouTube (the presentation is in French).

The datasets offered are various and many of them very pertinent for OpenStreetMap:

• address points
• bike parkings
• police & fire stations
• community gardens
• parks
• and many others

These datasets are provided mainly in three different formats: Shapefile, CSV and KML. Shapefile is the data format powering the ESRI GIS suite, and which QGIS can also work it. CSV is the well-known comma-separated values tabular format and KML is the XML format popularized by Google Earth.

Using ogr2osm

The org2osm script can handle both Shapefile and KML. This program converts these files into .osm files that can be opened directly into JOSM. Using the concept of “translators”, it becomes possible to convert the tags and attributes of the original data source into values accepted by OpenStreetMap.

For example, for police stations, the data provided by the Ville de Montréal is a KML file that contains the name of the station, the centroid and the address formatted as HTML in the description field. The file was thought of for display purpose in a viewer and obviously said tags cannot be used as-is into OpenStreetMap. This is why I wrote a translator for ogr2osm to parse the data and output OSM compliant tags. The script is available here and instructions to run it available here.

csv2osm: a tool for converting .csv files

However, ogr2osm cannot easily read .csv files as an input, so I wrote a script that is similar to ogr2osm and provide the same translator concept, and work exclusively for .csv files. The script is called csv2osm and can be found here.

Translators for this script have the same entry point as ogr2osm (the filterTags function). For example, Montreal’s open dataset for bike parkings is provided as a .csv file. The translator I wrote for this dataset parses the description of each bike parking to extract the capacity and provides the following tags for each points:

• amenity=bicycle_parking
• capacity=(number of places)
• operator=Ville de Montréal

Using the HOT Task manager for community-driven import

Lastly, our import effort is going to be based on the HOT Tasking Manager. Our instance is going to be located at http://gt.osmqc.ca/ (“gt” standing for “gestionnaire de tâche”, i.e. tasking manager).

To ease importing the .osm dataset into JOSM, we had to write a script that splits the .osm files into a series of sub-files, one per sub-task. Essentially the .osm file is split-up according to a grid defined in JSON format. This allows to see in JOSM only the part of the data for the task, instead of having to download the data for the whole city. Code is available here.

Conclusion

I hope these tools can be of benefit to the OpenStreetMap community. Do not hesitate to comment or write me if you have any question. I know the documentation on these tools is not 100% complete, so I’ll certainly post more details if there is interest for them.