Mapping public transport routes on OpenStreetMap can be a challenge, keeping them correct and up-to-date even more so. Last year I mentored the Mapillary plugin for GSoC. Before that, I had been working on public transport a lot, and to add the stops precisely where the poles are, horizontal imagery is a big help, when it’s available. The idea to propose a GSoC project for public transport this year was a logical consequence of some Python scripting I had started on. It’s possible to run Python in JOSM, but it involves too much setup for it to be practical for general use by other mappers.

Still, automation is key. Without it, it’s next to impossible to even find all the problems with the route relations, let alone fix them.

Also the way we’re mapping public transport, at the moment (who knows one day route segments become the thing), is very repetitive and fixing all those routes for all variations of the lines gets tedious fast. This in turn results in very few people interested in doing it. And the ones that do burn out sooner or later.

I’m sure the new plugin will be a big help to solve that problem.

At the moment the plugin is very good at detecting problems with the routes:

• going against oneway traffic
• gaps that are caused by wrongly ordered ways (automatic fix available)
• routes that don’t start/end on a stop_position node
• ways that still need to be split on those terminal nodes
• buses or trams that travel over ways unsuitable for that mode of transport

… See full entry

I recently configured OBS (Open Broadcast Software) to conveniently make screencasts toward Twitch.tv.

Usually I use this to show HOT contributors what I did while validating their tasks/tiles. I get some nice response to this. People like the feedback and the tips. I think it works as a motivator to keep going as well.

For the past few weeks I had this plan to create more to the point video material to show how to map with JOSM and why I think it’s so much more convenient and efficient to do so.

I had a problem with Bing imagery that was solved, by dumping my profile, so I seized the opportunity to show to get started with JOSM. While doing this I noticed there was imagery for Svalbard available and WOW, that is great imagery! So went on to improve my mapping of the Global Seed Vault. The video shows how to improve way accuracy, how to make nice bends in the roads, how to draw buildings very accurately, how to link to Wikipedia and Wikidata. How to jump to these articles in your web browser from withing JOSM and so much more. As usual I went a bit overboard, sorry about that.

I did it once more, this time with sound: Setting up JOSM for HOT mapping

These have no sound:

Setting up JOSM (part 1)

Setting up JOSM (part 2)

Setting up JOSM (part 3)

Setting up JOSM (part 4)

This blog entry describes how and why the following module was developed:

https://en.wikipedia.org/wiki/Module:OSM

In the mean time I added extensive documentation to it, which might be more interesting than what I wrote here… so go and have a look!

https://en.wikipedia.org/wiki/Module:OSM/doc

With the advent of Wikidata my interest in Wikipedia flared uponce more. I’m a mapper for Openstreetmap in the first place, but last year I already made a little sidestep to Wikivoyage. That’s a project which is closer in many ways to Openstreetmap than Wikipedia.

Recently I started adding wikidata tags to streets and other objects in Openstreetmap. Which in itself doesn’t do all that much. So a way is needed to make this available to the rest of the world. Wikipedia seems like the way to accomplish this.

I had already experimented a bit with links to saved queries on Overpass Turbo. But creating these queries from scratch over and over again and creating such links to them, isn’t the most straightforward way to get things done.

When it became clear to me that sometimes an additional query of wikidata may be necessary, before invoking Overpass, I knew I had to tackle the problem in another way.

This made me look into Lua, the scripting language which was chosen to automate the plethora of MediaWiki projects. After one day I came up with a working script (in the mean time it’s been developed quite a bit further):

On the Dutch Wikipedia reception was luke warm. I’m afraid, I hadn’t been able to carry across what I was trying to achieve. On the English Wikipedia a Lua developer saw the potential and he helped me to improve the code and explained me how to go about adding test cases. This helps a lot to not introduce bugs while attempting to add new features. For the case of querying several Q-numbers at once using a regex query, help came from a specialist in Overpass.

So this saved query:

[http://overpass-turbo.eu/s/aFt]

now looks like this in the article:

… See full entry

Met de komst van Wikidata is mijn interesse in Wikipedia weer wat opgeflakkerd. Ik ben in de eerste plaats een mapper voor Openstreetmap, maar vorig jaar heb ‘k ook al ‘s een zijsprongetje gemaakt naar Wikivoyage. Een project dat vrij goed aansluit bij Openstreetmap.

Nu ben ik recent begonnen met het voorzien van straten en andere objecten in Openstreetmap van wikidatatags. En aan welk project voor vrije content denkt een mens meteen, om ervoor te zorgen dat dat werk ook voor de rest van de wereld ter beschikking komt? Inderdaad: Wikipedia.

Ik had al wat geëxperimenteerd met links naar opgeslagen queries op Overpass Turbo. Maar heel handig is dat toch niet. Een query moeten aanmaken voor elk wikidata-item en elke nieuwe soort bevraging.

Daar komt dan nog bij, dat een Kerkstraat vernoemd naar een bepaalde kerk naar die specifieke kerk verwijst, maar als je alle Kerkstraten wilt terugvinden tesamen met alle ‘Rue de l’Église’, moet je eigenlijk eerst wikidata gaan bevragen. Maar dat zal nog niet voor direct zijn.

Het heeft me er wel toe aangezet om me ‘s in Lua te gaan verdiepen, de scriptingtaal gekozen voor de MediaWikiprojecten. Een dag later hebben we nu dus een script dat Overpass queries genereert:

https://nl.wikipedia.org/wiki/Module:OSM

Nu komen de Wikipediamedewerkers en ikzelf niet altijd zo goed overeen. Als ik buslijnen toevoeg, krijg ‘k te horen dat ze geen busboekje willen zijn. ‘t Kost dus allemaal nogal wat moeite, want dan moet ‘k op zoek naar bronnen en referenties over het ontstaan en de geschiedenis van zo’n route.

M’n test case werd alweer ongedaan gemaakt. Gelukkig is het mogelijk om naar een specifieke versie van een pagina te verwijzen:

https://nl.wikipedia.org/w/index.php?title=Fontenoille&diff=prev&oldid=44648493

Dus wel verloren voor het nageslacht, maar niet voor wie mijn hersenspinsels hier leest.

Ook op de pagina over Leuven had ‘k al eerder een query gezet die alle straten die naar Leuven vernoemd zijn ophaalt.

… See full entry

Mapillary for Android was updated and I’ve been meaning to write about it here.

Blog entry by developer of the app

The app now has a big green hot-button, which is very practical to go into picture making mode quickly.

The app now allows automatic shooting of pictures every 2 seconds even when standing still (or walking/cycling, the intended usage). The way to make it pause is to point it downwards. It’s not perfect at detecting this though, so I end up with quite a few pictures of my feet… It also stops shooting when you shake the phone too much. Maybe I should try that instead to pause it :-)

Oh, it’s possible to pause manually as well, of course, but that has the side effect of starting a new sequence. Sometimes that’s what you want, but not always.

It’s possible to let the app upload as soon as it’s in range of Wifi. That’s practical, but I disabled that option in the mean time. I prefer to browse through my pictures on the larger screen of the portable, as it allows to weed out the bad ones more easily. The way to review pictures on the phone was improved, but it still feels like a lot a lot of finger movement and my phone doesn’t seem to understand the ‘long click’ very well. I also lose track of which sequences I already reviewed and how far into them I had gotten, before having to context switch to irl.

Using the PC also has the advantage of the keyboard. On top of that, I can work with the pictures in the usual way of working with images in JOSM. Just load whatever GPX file, then use right mouse button on its name in the layer list to load pictures from the Mapillary folder on your phone’s memory card. This works, because they are already geotagged. I do that while it’s connected over the USB cable.

The advantage is that it’s possible to review the pictures once more and you don’t have to wait for the pictures to be processed.

After all that reviewing, I then release the pictures to Mapillary by enabling wifi and going into review mode.

… See full entry

Deze kaart laat toe om te zien hoeveel bomen en groenruimtes er zijn in Leuven. De data komt uit Openstreetmap.org.

Aan de linkerkant bovenaan kan u in/uitzoomen met de +/-, maar dat gaat eveneens met het muiswieltje.

Onder deze iconen staat een knop, waarmee u lagen kan selecteren. Klik eens op het oog van de heatmap-laag, om de ‘concentratie’ van de bomen te zien.

Dit is voorlopig niet echt representatief, aangezien niet elke boom al ingebracht werd in Openstreetmap. Wij hopen dat we dat wel kunnen realiseren met de hulp van de groendienst, de KU Leuven en u.

Voor sommige bomen werd species (soortnaam), genus en soms zelfs een foto toegevoegd. Als de foto op Mapillary.com staat, is het zelfs mogelijk om een virtueel bezoek aan die plaats te brengen. Dat wil zeggen dat er daar iemand is langsgeweest, die nog meer foto’s gemaakt heeft van de omgeving van de boom.

U kan zelf bomen toevoegen aan de Openstreetmapdata. Klik op de knop More en dan de voorlaatste knop. Waarschijnlijk is het het eenvoudigste om daarvoor de iD-editor te gebruiken. Dan hoeft u geen extra software te installeren. Als u veel bomen wenst toe te voegen, geef ik graag een woordje uitleg over hoe dat efficiënt gedaan kan worden met JOSM.

U kan ook voorstellen formuleren, over waar er volgens u bomen of groenruimte kan worden toegevoegd. Daartoe kan u de knoppen aan de rechterkant gebruiken, nadat u op dat potloodje heeft geklikt om naar editeermodus te gaan.

Spijtig genoeg is het niet mogelijk om slechts bepaalde stukken van de kaart voor iedereen editeerbaar te maken. Het is alles of niets. Daarom is het nodig dat u eerst een account aanmaakt.

osm.org/user/new

En deze dan hier toevoegt: http://umap.openstreetmap.fr/nl/ (Inloggen / Log in)

Als u me dan een boodschap stuurt, voeg ik u toe als bewerker voor de kaart.

Wat ook kan, is hier oefenen. Hier kan alles aangepast worden, maar de consequentie is dat ook alles gewist kan worden door eender wie…

Oefenkaart

… See full entry

After participating in the Missing Maps in Antwerpen, I went to the one in The Netherlands yesterday. Big thanks to Philip for making this possible for me!

The people from MSF/AZG/DWB/Red Cross had 2 Tasks in mind for us.

A though one in Africa where we had to add natural wood and rivers, as those are the (breeding) habitat for tse-tse flies, which cause sleeping disease. The beginners using iD had to abandon that one, as it’s impractical to zoom in far enough to work and still see the bigger picture. With JOSM one can draw a general contour of the wood at low zoom, then refine it by using the Improve Way Accuracy mode zoomed in a lot further. It’s still a daunting task to get it right though. What can be considered wood/forest? How does one recognise wetlands?

The other task was in Haiti. It was not easy either. A densely populated area and MSF wants to know how many buildings there are. (It’s explained better in the description…)

It peaked my interest for 2 reasons:

1. They used a drone to create superior aerial imagery
2. The Red Cross drove around with a smartphone used as a dashcam. The pictures were uploaded to Mapillary. Press that little play button and enjoy the ride.

This allows to go and have a look around and see what’s actually there. The width and state of the road, the state of completion of the buildings and so on.

Coverage is more limited than what can be seen from above, but it’s possible to read the name of a school and then it’s clear as well that it is a school.

The great thing about Mapillary is that the barrier of entry to contribute yourself is a lot lower than say, fly a drone to create excellent imagery…

All you need is a smartphone with a descent camera and GPS. Bonus points if it has a electronic compass, but that information can also be gotten from the GPX track. (electronic bread crumbs trail of where you’ve been).

… See full entry

In Finland they seem to think it’s a good idea to also add the road signs themselves. I tend to agree with them, but trying to add them all, may not make sense, of course, as there are a gazillion of them.

There are no applications making use of this information, but for us it would enable double checking why some ways have certain tags.

In Finland they are also able to find where zones are ‘leaking’ and they report this back to the administrations, so it gets fixed.

So I’m not saying we should aim to map all of them, but I still want it to be possible and convenient to add those that have our interest.

So I’ve been working on the RoadSigns plugin to make sure it has data about the Belgian Road signs. The work is not done yet, but I think I was now able to add all the accompanying signs and all signs related to parking, of which there are surprisingly many! The way it works now, you’ll have to remove the tags it adds, for those objects they don’t apply to. I’ve made a few suggestions for improving the workflow, but it’s unlikely those will be implemented anytime soon, except if I get my hands ‘dirty’ and do it myself…

So the effect of the sign remains on the ways, and the (Belgian) code for the sign itself remains on the new node you created before using the plugin.

If you don’t check the tick box Traffic sign, that code won’t be added and you don’t have to remove any tags. The plugin then does what it was designed for, add the effect of the sign to the ways it applies to.

What I’m not sure of, since it was an enormous task that I gravely underestimated, is whether all the tags, that are applied as an effect are actually correct. So the plugin needs testing.

Or you can have a look at this wiki page, there may be obvious errors in it that jump out to you :-)

osm.wiki/Road_signs_in_Belgium/Road_signs_plugin

Using the plugin is a bit more convenient though, as you can actually see the signs, instead of those codes.

… See full entry

While looking at source code of JOSM plugins, I stumbled across the RoadSigns and the ScoutSigns plugins. ScoutSigns adds some source data added by people using Skobbler.

For each sign, it gives a litlle picture taken by that ‘reporter’. When adding the effect of those signs on the road it applies to, I was wondering if it would be possible to map the sign itself.

At the moment the RoadSigns plugin doesn’t support that yet. But that’s what an old hand like me needs it for. I know the tags which get applied to the way, mostly by heart by now. But those codes which are different from country to country, can’t be bothered to remember those.

So I was thinking it might be nice to be able to add a node next to the way, select it and select the way(s) it affects. Then click on the little extra icon on the top right of the Tags pane. Select a traffic sign and signs that accompany it. Then it should automagically do the right thing with both the ways and that isolated node next to the way.

I think it’s needed to extend the xml format. I created an example here:

osm.wiki/Road_signs_in_Belgium/Road_signs_plugin

Now I’ll have to get my hands dirty and start coding to make that happen, so I can hand a patch to the developer of the plugin.

But before doing that, I also want it to work for Belgian signs.

To do that, we also need icons.

They are here: osm.wiki/Road_signs_in_Belgium

Fortunately they are also on the German Wikipedia. I fetched all the SVG files as follows:

import wikipedia
import urllib.request
import re

filenameRE = re.compile(r'http://upload.wikimedia.org/wikipedia/commons/.+/.+/Belgian_road_sign_(\w\d+(\w)*.svg)')

wikipedia.set_lang('de')
wp = wikipedia.page("Bildtafel_der_Verkehrszeichen_in_Belgien")
print (dir(wp))

for url in wp.images:
    fn = filenameRE.match(url)
    if fn: fn = fn.group(1)
    print (url)
    print (fn)
    if fn: urllib.request.urlretrieve(url, fn)

… See full entry

I went totally out of my comfort zone today. I was reviewing some pictures I had submitted to Mapillary and found one, I wanted to add right away to OSM. So I tried the link to edit it with iD. The first thing that’s odd, is that the zoom level is far out and the location of the object is way off. (It ends up all the way to the right, under the vertical button bar). I’d have expected it to appear centered, zoom level 21.

The next thing that’s annoying, is that the Mapillary layer is not switched on automatically, so I need to know iD to know where to go to switch it on. That’s not entirely intuitive.

Anyway, time to try and create a Mapillary plugin for JOSM.

Then I wanted to upload what I just added. But I couldn’t figure out how to add tags to the changeset. In version 2 I managed to add a source, but that ended up on the object. Now this scrawny tree already has 3 versions and none of those changesets have the source information I wanted to add to it.

Jo

Gedurende het voorjaar en de zomer van 2014, ontdekte ik een paar interessante plaatsen in Heverleebos en Meerdaalwoud ten zuiden van Leuven. Mijn interesse werd gewekt door een aantal houten standbeelden/sculpturen gemaakt van dode bomen. Ik had deze wel eens eerder opgemerkt, maar had er tot dan nooit veel aandacht aan besteed.

Toen ik er dan uiteindelijk toch eens wat meer over ging opzoeken, ben ik begonnen aan een Artikel over Ad Wouters op Wikipedia. Wist ik veel hoeveel tijd en energie daar zou gaan insteken! Ik was het natuurlijk aan mezelf verplicht om dat te gaan vertalen… En uiteraard had ik ook een kaartje nodig om het te illustreren. Dit heb ik gemaakt met behulp van Maperitive:

• https://commons.wikimedia.org/wiki/File:Pad_van_Ad_op_OSM.png

• https://commons.wikimedia.org/wiki/File:%27t_Pad_van_Ad.svg

Deze kaart is enig in zijn soort. Er worden gegevens van het openbaar vervoer, haltes en routes (lichtblauw en roze) gecombineerd met knooppunten van het wandelknooppuntennetwerk (rood/oranje) en natuurlijk de OSM-data, plus de route die ik wilde beschrijven (blauw). Nog geen 10 jaar geleden was zoiets zelf maken simpelweg ondenkbaar.

En toen ontdekte ik Wikivoyage:

Pad van Ad op Wikivoyage

Ad’s Path on Wikivoyage

Het loont de moeite om beide taalversies te bekijken. De verschillen onderling zijn groot.

Op de Engelstalige Wikivoyage hebben ze een widget, waarop Openstreetmapdata wordt weergegeven. Het is mogelijk om te schakelen tussen lagen en er transparante lagen overheen te leggen, wat toelaat om de wandel- en fietsknooppuntennetwerken te tonen. Daarenboven is het mogelijk om klikbare POI’s in de tekst te vermelden, die dan op de dynamische kaart worden weergegeven.

… See full entry

During the spring and summer of 2014, I discovered some interesting places in the forest south of Leuven. My interest was peeked by some wooden statues/carvings made from dead trees. I had seen some of them before, but I hadn’t really been paying attention, and they didn’t register as a ‘collection’ until then.

When I did finally investigate further, it lead to the creation of an Article about Ad Wouters on Wikipedia. Of course, I needed to illustrate that with a map, so I created one with Maperitive:

• https://commons.wikimedia.org/wiki/File:Pad_van_Ad_op_OSM.png

• https://commons.wikimedia.org/wiki/File:%27t_Pad_van_Ad.svg

This map is unique in that it combines information about the public transport stops and routes (light blue/pink) and the walking node network (red/orange) and, of course OSM data, together with the route I was describing (blue). Only 10 years ago it would have been impossible to even consider attempting to create such a work.

Subsequently I discovered Wikivoyage:

Ad’s Path on Wikivoyage

On Wikivoyage they have a widget, which displays Openstreetmap data. One can switch layers and add overlays, allowing to show the cycle and the walking node network as well, but more importantly one can mention POIs in the text and have them appear on that dynamic map

It took some learning and stumbling. Each of these projects has their own rules and intricacies, which are completely different from OSM, but they are also different among what I consider a family of projects and even from one language version to another. At times they almost drove me mad, but now I’m quite happy with the results. I’m not sure if I’ll attempt something similar again any time soon though. Mapping for OSM has its own rewards.

Today I put template route relations for itineraries of De Lijn and TEC on Dropbox. Generating them takes 2 hours. And then another 2 hours when I noticed Python hadn’t actually compressed them. I’ll update them as fresh data from De Lijn and TEC comes in.

I also worked on a script which you can run after opening such a file in OSM.

I put some videos on Youtube to illustrate the process

(no sound was recorded and I still have to add subtitles to them, some day)

The script will download all the stops. They are not included in the files. Once the stops are downloaded the view zooms to their extent.

Then the script will use an Overpass Query to download all route relations with the same ref as identifier. At the moment all over Flanders. I hope I’ll find a way to limit it to the bbox you just zoomed to.

For all the nodes on the way, Overpass will download all the nodes which are within 30 meters (and all the ways and relations they belong to).

Now you’ll have to run the compare script again. I didn’t figure out yet, how to make the script wait until the download completes, so it’ll give an error message saying it didn’t find the master_routes it was looking for.

When run again, after the download is complete with not objects selected, it’ll compare the template routes with the already existing routes. It uses the master_route relations to match them. If the existing routes don’t belong to a master_route yet, you’ll have to use the template and add the matching existing routes manually. You can leave the template routes, they’ll be removed from the master as we progress. Select all the existing and the template routes and close the master route. Now run the compare script again.

You’ll end up with a relation which has some funny looking roles. Each letter-number (a0w1i3) combination represents as stop. Now it becomes easier to see which template belongs to which existing route.

… See full entry

People sometimes ask me: why add all these route relations to Openstreetmap, when De Lijn already has them as shape files?

For one thing, De Lijn cannot share them with us. A good example is the following:

Line sketch on Overpass for De Lijn 91

If you click through to one of the other lines, change the url so it has &style=wuppertal once again at the end. Also keep in mind not each and every line is already mapped, although we’re doing quite well in Oost-Vlaanderen.

Another example is a map I rendered where I’m combining the PT network (pink on the background), the walking nodes network (orange) and the route I wanted to talk about:

Pad_van_Ad_op_OSM.png

Only, say 10 years ago, creating such a map would have been an impossible endeavour. It’s important for the explanation on this site:

Ad’s Path (Pad van Ad)

To be able to show where the bus stops are and the walking node numbers are used in the directions, but not exactly used the way they were meant to be used… only to help point people in the right direction, instead of following them from node to node.

Incidentally you’ll have to have a look at the Dutch version of the article to see the map used. The English version uses Openstreetmap too, but in a more dynamic way.

Jo

I attended a meeting of the Peruvian community on Saturday. Somebody showed us the capabilities of Mapillary. Of course, I’m hooked already.

Soon the parts of the city and surroudings I frequently visit will be visible on Mapillary.

I’ve been waiting for a long time for a way to link back to pictures I took during my surveys. Wikimedia Commons is no good for this, all the worthwhile pictures I uploaded there got nominated for deletion. So I gave up on them.

Mapillary looks promising. If it were up to me they’d get a dedicated tag all for themselves:

mapillary=z0PmxYaUUDIzLYZSVLncsQ

But I also gave up on the voting process on the wiki. Even if a vote’s result are positive, that doesn’t mean or guarantee that that scheme will be rendered by the powers that be (yes, I’m talking about the ‘new’ public transport scheme).

So I’ll probably simply start using that tag and be done with it.

Jo

• De Nederlandstalige versie vind je hier
• La version francophonese trouve ici

This article is inspired by http://blog.openstreetmap.de/blog/2014/12/wochenaufgabe-kw-5152-oepnvfahrplanwechsel/ but adapted to how public transport is mapped in Belgium at the moment.

I also recorded an editing session in JOSM:

Add bus route in Antwerp Belgium

Apparently public transport companies do all the major changes to their time tables towards year’s end. At least this seems the case for both Germany and Belgium. So we have some changes to process in Belgium as well.

In Germany they have weekly assignments. I guess that, over here, cleaning up the mapping of public transport, will become a task, that takes somewhat longer than a week to accomplish.

… See full entry

I found a nice little feature in JOSM: q makes rectangles with 90 degree angles. They look perfect while still in JOSM. Then it gets rendered and I end up with parallellograms. What's up with that? (and more importantly: what can I do to prevent it?)

Jo

I'll be mapping in and around Cusco for the next 5 weeks.

Polyglot

Just bought a new datalogger/GPS mouse. It's a nifty little thing which has 44 channels. It's very precise. I can use the IPAQ to connect to it over bluethooth, but it can also log on its own. 330000 points.

So from now on I'll have two independent gpx tracks to work with. If only I could find a way to set its clock. All the rest, I already achieved. BT access from the IPAQ and from the portable, translating the log output with GPSbabel and uploading it with josm.

Just completed the (probably nonsensical) job of entering the layout of our botanical garden with Potlatch.

osm.org/edit.html?lat=50.879163&lon=4.701067&zoom=18