Microsoft have recently released their “Worldwide building footprints derived from satellite imagery”. This is a dataset containing the shape and positions of houses. It covers many parts of the world and is derived from Bing aerial imagery with the use of deep learning.

I find these footprints interesting for numerous reasons. The focus here will be to use the data to aid in mapping. There is further information on the OSM wiki.

In this diary entry I want to explain how I got the data into JOSM. In a following entry I will explain how I use this data to speed up building mapping.

Obtaining the data

The data is freely available on Github. It is licensed under ODbL and is compatible with OpenStreetMap. Each available country has its own zip file. Some of the files are of the order of gigabytes - far larger than can be opened in regular software. Pick a country and download the zip file.

Issues with the data

Care should be taken to examine and understand the data, and this set is no different.

• I found that numerous buildings were repeated in the data. This meant that JOSM had duplicated buildings and duplicated nodes after importing.

• There are various false-positives and true-negatives in the data. Some things that are buildings are not given in the data. Likewise the dataset thinks some objects are buildings which are not.

• The rotation of buildings can be off.

• The data may contain overlapping buildings, such as a garage overlapping with the house next to it.

Unzip and geoJSONL

The zip file contains a .geojsonl file (notice the ‘l’ at the end of the filename). This file type cannot be directly imported into JOSM. In addition, files on the order of gigabytes will clog up JOSM too much to be usable. (*Edit: see vorpalblade-kaart’s comment below for better information)

I found suggestions about splitting the large file into smaller files using the split command. However, the data is not spatially ordered within the file, so each split chunk has data from all over the country.

In order to create a small file of a subset of the data localised in an area of interest I turned to python and geopandas.

Python and Geopandas

Geopandas is a python library that can read and convert geoJSON(L) files, and can limit items to within a bounding box. I can specify the coordinates of a box and only the buildings within will be kept. This allows creation of a manageable file size focused on an area of interest. I’m sure what I used this for can be done with software like QGIS, but I’ve never used that type of software before.

A script

In the end I created a python script that would read in an unzipped .geojsonl file downloaded from the repository, cut out a selected region, remove duplicate geometries, and save the resulting geometries to a .geojson file that can be safely imported into JOSM.

The script is available on GitHub. You should have python installed. It needs the geopandas and numpy libraries, which can be obtained through the command pip install geopandas numpy. The script can be called with python extract_region.

Conclusion

The image below shows the data imported into JOSM

I’ve been able to extract geospatially grouped building footprints into JOSM. In testing how large files can be, and I’ve found files on the size of 300MB to be the limit that JOSM on my computer can handle. I have 24 GB ram.

My next entry will talk about how I use this data to speed up the mapping of buildings in OpenStreetMap and JOSM.