OpenStreetMap logo OpenStreetMap

Yet another diary entry about vector tiles

Posted by SomeoneElse on 29 May 2024 in English. Last updated on 11 November 2024.

A simple "shortbread" / "VersaTiles Colorful" maplibre-gl vector map

Lots of people have written diary entries and there have been other announcements and demos. You might wonder why we need another?

The idea of this is to provide a “soup to nuts” guide in a similar style to the existing “Switch2osm” Serving Tiles guides for raster tile servers.

Introduction

There are two major pieces to this. The first piece creates a set of vector tiles that can be served by a web server. The second provides a way for clients to access that data - and clients also decide how to display it.

The code in this example is for web clients, but it’s also possible to create other sorts of clients too (for example, Android or iOS apps).

One important choice is “what data should be in in the vector tiles, and at what zoom level”. It’s not possible to have everything in OSM available at every zoom level - the size of each tile would just get impractically large. Handily, people have already created schemas that target different use cases. One general purpose one is shortbread.

That documentation doesn’t just define what data is in the tiles, it also covers how to create them. The actual software used to create the tiles is Tilemaker. The documentation of both of those is pretty much good to follow directly - what follows below is basically just a worked example.

As an example map style, I’ve used “colorful” from VersaTiles. The implementation below differs slightly from the examples on VersaTiles’ site, in order to make it clearer how it all works together; but do be aware that that site offers a lot more than just a couple of web vector map styles.

Setting up a server

As a test server I chose a CAX21 at Hetzner - that has enough memory and disk for the worked example below, and is relatively cheap (€0.25 a day or so). Other cloud providers are available; and of course machines with this sort of specification is going to be available to many people locally too.

I chose Debian 12 as the operating system, selected IPV4 + IPV6, added my ssh keys, made a name up, and added that as a fully qualified domain name to my DNS. I then created a non-root account for “normal” usage and added it to the “sudo” group. I turned off password access to ssh and forced all logins to use ssh keys.

I installed some initial software

sudo apt install emacs-nox mlocate screen git tar unzip wget bzip2 apache2 socat net-tools curl

In that list “emacs-nox” is just my preferred editor and “socat” is used when obtaining an SSL certificate for the web server which we’ll cover below. They’re just peripheral to serving tiles.

At this stage browsing to “http://myserverFQDN” (obviously use your name instead of the text “myserverFQDN”) should display an “it works” page.

sudo apt install postfix mailutils

Also peripheral to the main job in hand, “postfix” is a mail server and “mailutils” provides command-line mail access. These are useful on a remote server for reporting when e.g. scripts have failed, but are less essential if the server is local. When installed, postfix asks how mail should be sent (“Internet Site” in this example) and what the mail name should be (I used the FQDN from earlier).

With an internet-facing server with a known name it’s easy to obtain a free SSL certificate using a service such as Letsencrypt or ZeroSSL. I used acme.sh for this, but there are lots of other options. Once I had the certificate files I then ran:

a2ensite default-ssl
a2enmod ssl

and edited “/etc/apache2/sites-enabled/default-ssl.conf” to add

SSLCertificateFile      /root/.acme.sh/myserverFQDN_ecc/fullchain.cer
SSLCertificateKeyFile   /root/.acme.sh/myserverFQDN_ecc/myserverFQDN.key

below “documentroot”, and then ran “systemctl reload apache2”.

Browsing to “https://myserverFQDN” now works without an SSL error or warning. I set up an iptables firewall to block access that I didn’t want but allow e.g. web and ssh traffic, over both ipv4 and ipv6. Various online guides exist, including this one. When you’re happy that both ipv4 and ipv6 are configured correctly:

sudo apt install iptables-persistent

and say “yes” to saving both ipv4 and ipv6 rules.

Shortbread and Tilemaker

When logged in as the non-root user you created above, we’ll need to install some prerequisite software:

sudo apt install gdal-bin apache2-dev libsqlite3-dev

and then obtain some other software that needs building. First, “mod_mbtiles” is an add-on to apache to allow it to server vector tiles.

cd
mkdir src
cd src
git clone https://github.com/systemed/mod_mbtiles
cd mod_mbtiles
sudo apxs -lsqlite3 -i -a -c mod_mbtiles.c
sudo systemctl restart apache2

Next, get shortbread-tilemaker, which defines the schema that we’re using - it comes with the rules that Tilemaker will follow.

cd ~/src
git clone https://github.com/shortbread-tiles/shortbread-tilemaker

and then Tilemaker itself

cd ~/src
git clone https://github.com/systemed/tilemaker

Now we need to build tilemaker locally as described here.

sudo apt install build-essential libboost-dev libboost-filesystem-dev libboost-iostreams-dev libboost-program-options-dev libboost-system-dev liblua5.1-0-dev libshp-dev libsqlite3-dev rapidjson-dev zlib1g-dev
cd ~/src/tilemaker
make
sudo make install

Next, download some data. I’m going to use “Britain and Ireland” from Geofabrik. At the time of writing this (just) fits into this specification of server.

cd
mkdir data
cd data
wget https://download.geofabrik.de/europe/britain-and-ireland-latest.osm.pbf

We need some “background data” - there’s a script to get that in the “shortbread-tilemaker” directory:

cd ~/src/shortbread-tilemaker
time ./get-shapefiles.sh

We can now run tilemaker. The bounding box here covers most of Europe (it needs to be bigger than our data download area). Generating tiles for the whole world takes much longer so we won’t do that. Use Geofabrik’s tile calculator to choose appropriate values.

cd ~/src/shortbread-tilemaker
time tilemaker --config config.json --process process.lua --bbox -11.39,48.44,2.28,61.36     --input ~/data/britain-and-ireland-latest.osm.pbf  --output ~/data/shortbread-tiles.mbtiles

When generated, copy the tiles into the web server drectory

sudo cp ~/data/shortbread-tiles.mbtiles /var/www/html/

and edit your appache config files (by default, “000-default.conf” and “default-ssl.conf”) to add:

MbtilesEnabled true
MbtilesAdd shortbread-tiles /var/www/html/shortbread-tiles.mbtiles

below “DocumentRoot”, and restart apache.

VersaTiles Colorful

One of the examples listed above, Paul’s TileKiln demo includes VersaTiles “colorful” style. We’ll fetch a copy of that and edit it to work with our tiles. As your non-root user:

cd ~/src
git clone https://github.com/pnorman/tilekiln-shortbread-demo
cd tilekiln-shortbread-demo
wget https://demo.tilekiln.xyz/shortbread_v1/tilejson.json

Open “colorful.json” in a text editor. This definition:

"glyphs": "https://tiles.versatiles.org/assets/fonts/{fontstack}/{range}.pbf",

defines where fonts (for text styles) are loaded from. It defaults to VersaTile’s own site, which is fine for the purposes of this demo. This definition:

"sprite": "https://tiles.versatiles.org/assets/sprites/sprites",

defines where all the icons on the map come from. They’re all in one big “sprites.png” file and a separate “sprites.json” file defines which one is where.

Finally, edit the line

"url": "https://demo.tilekiln.xyz/shortbread_v1/tilejson.json"

to instead read

"url": "https://myserverFQDN/tilejson.json"

(or “http” if you have not set up an SSL certificate for your website) We’ll also edit our local copy of “tilejson.json” to replace

["https://demo.tilekiln.xyz/shortbread_v1/{z}/{x}/{y}.mvt"]

with

["https://myserverFQDN/shortbread-tiles/{z}/{x}/{y}.pbf"]

That “shortbread-tiles” matches the name in the “mod_mbtiles” confguration above

Next copy “index.html” and your modified “colorful.json” and “tilejson.json” to the root of your web server “/var/www/html”. Restart apache and start a new private browser.

Browse to

https://myserverFQDN/index.html#18/54.501519/-8.190062

and you should see a map!

If you don’t, then something has gone wrong. After correcting whetever it is, you’ll probably need to restart Apache and/or close your private browser and restart it. Browser web developer tools (often control / shift / I), especially the “network” tab, can be helpful at understanding what has gone wrong.

Location: Larganhugh, Melvin Electoral Division, Manorhamilton Municipal District, County Leitrim, Connacht, Ireland
Email icon Bluesky Icon Facebook Icon LinkedIn Icon Mastodon Icon Telegram Icon X Icon

Discussion

Log in to leave a comment