OpenStreetMap
I’ve just updated the contributor statistics for the whole year 2018.
As I pointed out here 2017 update blog post some the graphs look a bit odd due to the one-time effect of an initial large number maps.me users contributing as the editing feature was introduced.
This effect continued to decrease during 2018, but for completeness sake here are the updated graphs from the blog post for 2017 that continue to show continuing growth even without maps.me.
Now that Vespucci 12 is already nearing release (if you are helping with the translations, please have a look at them asap, as they are the major hold up), it’s time to think of what is coming next.
Vespucci has supported unconnected mapping since day one (nearly a decade ago) and that has improved over the years with addition of support for reading and saving files in normal OSM and JOSM xml format (the JOSM format stores information on local changes).
In version 10.1, early this year, we added support for MBTiles files for local on device background imagery sources, supporting building an imagery source while you are online on your desktop and avoiding having to download to Vespucci imagery cache manually which tended to be rather tiresome.
But despite all of this, the main problem remained that you need to keep OSM data files on device reasonably small because the contents would be read in to memory and while such areas can be quite large on a modern phone, things tends do to get slow. And as we all know when you make a selection in advance, Murphy comes in to play and you are surely are going to miss exactly the area that you suddenly stumbled in to.
To get around all of that I’ve been investigating a compact, indexed in one way of the other, on device format for raw OSM data for quite a while, and now have a solution that works even better than what I originally envisioned.
The fist beta/test version of Vespucci 12.0 is now available from the play store (together with the monthly update for the current release, 11.2.3).
The current way (pre version 12 that is) of adding new objects with a long press goes back to the work done by Jan Shejbal during GSOC 2012 (a couple of years before I had anything to do with the project). This has the advantage that it doesn’t need an extra menu, you can inspect where you are placing the object before actually creating it, and in general that it is fast. The downside is that it can be a bit fiddly at times and users need to be told about it in advance and as we all know RTFM is not very popular.
Simple action mode replaces the long click action on the screen with a menu driven way of creating new objects. Long clicks are disabled as long as the mode is active.
Tapping the large green floating button will show a menu. After you’ve selected one of the items, you will be asked to tap the screen at the location where you want to create the object, pan and zoom continues to work if you need to adjust the map view.
I just received a request to add a preset for amenity=language_school to my fork of the JOSM presets (that you should be using instead of the original :-)) Nothing to said against that, it is a reasonably popular feature with nearly a 1’000 uses.
Naturally a preset that is essentially just a stub isn’t really helpful, so as always I checked what additional properties should be added, and tagging the taught languages is an obvious attribute that is interesting.
Much to my dismay it seems as if in early 2016 it was infected with the “move values to key” plague. Instead of having two keys, lets say
language
and
language:main
containing a list of the languages, we now have more than two hundred potential keys for an in principle, simple attribute, making both data consumption, editing and creating a preset more than just difficult (as the keys can have at least three values plus unset, they can’t even be modelled with a checkbox in a preset).
At the time somebody further thought it was a good idea to retag existing language tags to the broken schema:
The beta should be available on googles playstore and from out github repo in a couple of days. No fancy pictures of new features as there are none.
As announced previously this version no longer supports Android versions prior to 4.0 on googles playstore due to restrictions that google is now imposing. It is however possible to either build your own APK with support for earlier Android versions and likely we will be distributing such a version via F-Droid and our github repository.
Do be able to do this we now have two build flavors:
Right now the current build mainly profits from bug fixes to the support libraries, for example the issue with sub-menus on tablets is resolved in the version we build the current flavor against.
This is not a commitment to supporting 2.3 indefinitely, but as we suspect that google will be increasing the minimum Android that practically can be supported in the future we expect that the legacy build will continue as a vehicle to support older versions.
On devices that only supply 32MB or less of heap to apps, memory usage of Vespucci had gotten to the point at which the app could potentially completely run out of available memory. To work around this a bit we’ve stopped loading the simplified country boundaries which saves roughly 4MB, this implies that any country specific features will not work. Further we recommend to, instead of the standard bundled preset, to load the untranslated one (see the on device Preset help page) which will save roughly another 3MB (mainly because we don’t build a translated search index then).
We released Vespucci version 11.0 at the beginning of July, 2 months later, the first beta of 11.1 is now available on googles play store and from our github repository..
Before going in to some details on 11.1, a quick word on the plans for the rest of the year. I hope that we can limit the testing period of 11.1 to the month of September and release early October.
My current plan is that 11.2 will have no new features and will solely be a vehicle to deal with googles forced update policy (see https://github.com/MarcusWolschon/osmeditor4android/issues/773 too). This also implies that there will be no update/bug fix releases of 11.1 after November 1st, so get on with testing :-) and that the google play version of the app is very unlikely to support pre-Android 4.0 devices any more.
Version 12, if all goes well including some interesting UI changes, will become available early 2019 in time for Vespuccis 10th anniversary. Back to 11.1.
The internal representation of presets has been overhauled allowing us to implement some new features without adding to already very byzantine code. The major changes for users are:
Work has proceeded nicely on the next major release of Vespucci and while there are still some rough edges and not quite everything has been added that I intended to, it definitely makes sense to test this version. THe beta APK is available from googles play store and our github repository.
The most obvious visual difference in version 11 is the layer control on the main map view.
Currently it is not possible to change the ordering or add more than one layer of a specific type, supported functionality:
First a note, it seems as if I never blogged about the 10.1 update, information on those changes can be found in the release notes on the Vespucci website.
The beta release of 10.2 that is now available in the beta channel on the google play store, or from the releases on github does not change an awful lot that is end user visible outside of a new upload UI, however there are two core changes that I want to touch on quickly.
Historically Vespucci has only supported using the on-device GPS location provider, or nothing at all. That meant that you were unable to get a rough location approximation on devices that didn’t have onboard GPS, or that had GPS disabled for example to reduce power requirements. The main reason for this is that on the one hand we wanted to avoid location information potentially tainted by your devices Android provider and avoid our users position being tracked by them.
We now support using so-called “network” location providers, that is location sources that derive your position from the mobile network, WLAN and other signals your device is receiving. If you’ve enabled such providers on your phone, more on that later, Vespucci will use all available providers for centering the map display on your position and for auto-downloads, tracks will still exclusively be generated from GPS data.
The change in opinion is mainly due to less and less people caring about such matters and at least google tracking in any case (see for example https://www.theverge.com/2017/11/21/16684818/google-location-tracking-cell-tower-data-android-os-firebase-privacy), further allowing such providers enables better indoor positioning which is a clear advantage.
I’ve updated, as every quarter, the statistics on our wiki.
One thing we can see on
is what looks like growth flattening out.
Splitting out the new contributors from maps.me and other sources gives us the following graph:
Just in time for “X*mas we’ve started rolling out Vespucci 10 “X”-
I discussed the new C-Mode and configurable validator previously in this diary entry, but there are quite a few other changes:
Even though Vespucci has been around for over 8 years, we used version numbers below 1 in a rather nerdy understatement fashion. I’ve been guilty of slightly weird numbering before .
The problem with this is that these days nobody understands if you are not at at least at version 3 after a couple of months and people may actually think something is wrong, so we’ve decided to do a Mozilla and jump to version 10.
While semantic versioning doesn’t really make a lot of sense for applications that don’t expose an API, we will be sticking with a major . minor . patch system for the internal numbering for now.
The preset search will now use the same list of synonyms that the iD editor does additionally to the internal preset search index. The new functionality is independent of preset translations and uses the same fuzzy matching as the index search. Additional synonyms should be added on transifex to the iD translations.
One of the reasons I based this on existing technology, even though the current system has its warts (we should probably simply gather synonyms completely separately from our translation platform) , is that OSM development already suffers enough from rampant NIH and there is no real added value in asking translators to add synonyms to two different systems.
You can now load (and save) tasks in a simplified Osmose JSON format. The format is not particularly forgiving and must follow the following example:
With the last release 0.9.9 running nice and stable and the monthly updates to it contain mostly updated presets and imagery configurations, it is time to give you a preview of what I’ve been working on for 0.9.10.
Vespucci has long highlighted objects that had fixme tags and streets that are missing name tags, the code for this goes back to 2010, long before that was considered award worthy.
Over the years we have added support for warnings from Osmose and OSM Notes, making it easy to spot issues that might need work. In 0.9.9 I added preliminary support for highlighting objects that haven’t been checked for a longer time.
But it is undoubtedly true that with OSM data becoming denser and denser and in many areas near complete it is non-trivial to find objects that need your attention..
The solution to this has two components:
A mode that only shows elements that have warnings.
A couple of weeks back I regenerated the data in the “Name Suggestion Index” from a current planet dump, adding a largish number of new entries. The index is used by iD and Vespucci to generate canonical spellings for well-known brands and to apply the correct presets at the same time.
Naturally the raw list contains a lot of nonsense, and that’s why there are two ways to reduce noise to an acceptable level: one, a list of (mis)spellings that are mapped to a canonical value https://github.com/osmlab/name-suggestion-index/blob/master/canonical.json and, two, a filter https://github.com/osmlab/name-suggestion-index/blob/master/filter.json that removes names that we don’t want, for example Bank for banks.
You may already have it on your device and this is old news, but yesterday we pushed the release version of Vespucci 0.9.9 to the google and amazon app stores.
The release notes list most of the changes including preliminary documentation on the opening hours editor.
I would specifically like to thank Mateusz Konieczny and Holger Jeromin for beta testing (Mateusz particularly for finding a last minute issue).
0.9.9 took a long time to get to this state, mainly due to a lot of under the hood work which should make upcoming releases much easier. I’ll be trying to step up the release frequency again and expect 0.9.10 including C-mode to be available by the end of the next quarter.
out there is trying to run Vespucci 0.7.0 which dates back to April 2011 on a Xiaomi Redmi Note 4 with Android 6.0.
While the app will run, 0.7.0 was released a while before 64bits for OSM element ids became necessary (early 2013), and trying to parse current data will lead to a crash.
With other words: please upgrade to a current version!
Thank you
Vor gut einem halben Jahr habe ich einen Blogpost zur Totalrevision des VGWR verfasst, nun ist es so weit: Anfangs Juli wird die revidierte Fassung in Kraft gesetzt. Da das BFS jetzt auch den Text dazu publiziert hat (revidiertes VGWR als PDF), kann man jetzt schon einige Aussagen dazu machen:
Offene Fragen:
PS: ab 1.1.2018 werden die Adressdaten des Kanton Zürichs zu sehr offenen Bedingungen verfügbar sein, so oder so wird also der Bestand an wirklich offenen Adressdaten für die Schweiz in absehbarer Zeit stark zunehmen.
Now despite the difficulties of garnering attention when your app doesn’t break things, Vespucci development has carried on and I’m pleased to announce that the first “official” beta release of 0.9.9 is available from googles play store and from our [github repository] (https://github.com/MarcusWolschon/osmeditor4android/releases).
While there are quite a few user visible changes, most of the work since the release of 0.9.8 has been under the hood, improving the maintainability, testing, documentation and code quality throughout the app, making it much easier to contribute to its development.
It should however be noted that Vespucci has a relatively large code base and contributing to the code (but not to the documentation, presets and other aspects of the app) remains fairly complex. See vespucci on OpenHub for more on that.
The release notes give an overview of what has changed, here just some highlights:
In November 2015 I produced a blog post on the relative sizes of our national communities and their development over time. When I updated the contributor stats on our wiki a couple of days back and looking at the massive impact the influx of first time users using maps.me have on the contributor numbers (not necessarily on anything else as I have pointed out in earlier posts), I thought it would be interesting to update the numbers and have a look at what has changed over slightly more than a year.
The most notable changes in the top 20 ranking by absolute community size are Russia overtaking France, Ukraine moving up 6 places and the Philippines moving in to the top 20 at the expense of the Czech Republic.
It is reasonable to assume that these changes are mainly due to the differences in the popularity of maps.me, No surprise and expected that it is most popular in Russia and the Ukraine, not so clear why it is so popular in the Philippines:
Three years back I started sending welcome messages to new mappers on behalf of SOSM, using a simple tool chain based on the RSS feed of new mappers from Pascal Neis.
The message is not customised as from the beginning it was seen as a replacement for the welcome message that had gone away with the web site redesign. The text (in 4 languages, no Romansh version though) is designed to welcome the new contributor and give pointers to the resources available nationally.
Has it worked? Difficult to say, as the whole point was a replacement of something that existed before, we didn’t expect drastic change and we didn’t get it.
And, no, I’m not stopping :-)
Two weeks back I was invited to Wherecamp Berlin, on the one hand to participate in the opening panel and on the other to have a short talk on the state of indoor mapping, more about that later.
Naturally the foreseeable impact of self-driving vehicles and machine learning in general was the, obvious, main topic of the opening panel. But outside of “things are going to change” I’m not sure that there was any other conclusion from a couple of oldish men without crystal balls.
From an OpenStreetMap point of view it was encouraging to see so many OSM based projects and operations presenting at the event mingling with essentially all the big names in geo-business.
Stefan from GraphHopper presenting at Wherecamp Berlin.
From a bit of a nerdy view, the first day presentations from Qualcomm, Broadcom and Google on what we will likely be getting in the way of GNSS support in upcoming mobile phones was quite interesting.
Einige Leser haben vielleicht einen leicht genervten Tweet von mir gesehen als ich diese Woche entdeckt habe, dass die Vernehmlassung über die Revision des VGWR an uns vorbeigegangen ist, ohne dass wir die Möglichkeit hatten unsere Meinung dazu anzubringen.
Das Gebäude- und Wohnungsregister ist ein schweizweiter Datensatz der sämtliche bewohnte Gebäude und Wohnungen in der Schweiz umfasst. Neben Merkmalen die für OpenStreetMap nicht interessant sind, enthalten die Daten auch einen vollständigen Adressdatensatz und zum Beispiel auch die Anzahl Stockwerke für jedes Gebäude. Die GWR Daten sind zwar nach der jetzigen Rechtslage nicht direkt in OSM nutzbar aber wir haben immerhin Zugriff auf einer daraus erzeugten Strassennamensliste den wir zur Qualitätskontrolle nutzen.
Aus den GWR-Daten generierte Adressen werden von SwissTopo auch auf map.admin.ch angeboten, auf welcher Rechtsgrundlage die Publikation beruht ist allerdings völlig unklar. Dies tangiert OSM insofern, dass in der Schweiz man keinen urheberrechtlichen Schutz so publizierter Daten erwarten kann, und der Nutzer somit auf magische Art und Weise wissen müsste, dass das aktuelle VGWR eine abschliessende Aufzählung der erlaubten Nutzungen enthält und diese Daten in keiner Art und Weise für OSM genützt werden dürfen. Eine ähnliche Problematik stellt sich auch mit öffentlich zugänglichen Datensätze dessen Nutzung im Geoinformationsgesetz geregelt wird, aber immerhin muss man da nicht ganz so obskure Kenntnisse der Rechtslage haben um sich richtig zu verhalten.
Der definitiver Text des revidierten VGWR, der Anfangs 2017 in Kraft treten soll, ist nicht verfügbar, deshalb bezieht sich diese Analyse auf den für die Vernehmlassung verteilten Entwurf. Dieser regelt viel administratives neu, zum Teil aus föderalistischer Sicht durchaus nicht unproblematisch, und erweitert das GWR um die nicht bewohnten Gebäude. Relevant für OSM sind vor allem aber die folgenden Punkte: