@redsteakraw my point was more that you are asking for an extremely complex editor, far far more complex than any other one, just alone handling all the edge cases will be a royal PITA.

I only mentioned Vespucci because you don’t have to have geometry editing enabled in Vespucci, you can even directly start it in tag editing mode and that is from a functional pov what you were proposing, but it doesn’t hide complexities of OSM modelling from the user and doesn’t want to.

The other point though is that there is only a very limited pool of mappers, and every additional app segments that small pool further, making only the absolute largest ones sustainable once the hype has died down and the only thing left is work. So my expectation that we will end up just with iD and maybe one nav app with some add on editing in the longer term.

@᚛ᚐᚋᚐᚅᚇᚐ᚜ 🏳️‍🌈 sure I have. The problem with that should be clear.

Further point that @DeBigC raises “… what is it that makes so many of the useful apps collapse?”.

It is the “many” in your question.

Conservatively estimated a conventional general purpose OSM editor will require at least 2 FTEs to maintain long term, something simpler more (providing a “simple” UI is -more- complex, not less). 1 FTE for a limited functionality app (though that is probably too low).

For the sake of this argument, let’s assume that a FTE is $200’000 annual total cost to their employer, or themselves, if not paid for the work.

Have a look at editor stats, annual costs per user using above assumptions for the top couple of entries are

General purpose editors

Restricted functionality editors

To be clear this is what you would have to assume in cost for continued maintenance to keep things current once the initial enthusiasm has worn off. Obviously most apps do not get so much attention and therefore slowly bit rot and fade away. Further dividing both user base and developer resources just makes matters worse.

Try thinking about the numbers as a yearly subscription you would have to pay to be able to continue to use the app.

PPS: in the end what we really want is for operators of POIs to add and maintain them themselves, leaving just the remaining gardening to OSM mappers. While the association with tech giants (Apple, FB, MS and so on) has its pros and cons, it definitively has led to more awareness in business communities and here we are seeing a clear interest in being present on OSM (yes I know FB et al don’t actually use OSM for POIs, but that can change).

Self maintenance is easy for example with https://osmybiz.osm.ch Obviously this could be even further simplified, but again what about using what we already have and improving that.

@DeBigC in general it is technically preferable to map POIs as individual nodes, in fact putting tags on buildings is only acceptable if it is the only POI associated with the building which most of the time it isn’t. Definitely you shouldn’t be basing your mapping on how a specific map rendering looks.

@redsteakraw the collection problem has been addressed, if not solved many times, going back to the mapzen POI collector (not related to the defunct mapzen company). The problem is more that people continue to ask for better hammers, but don’t actually want to do the hammering.

In practical terms Vespucci has shown outdated POIs for ages, functionality that was added to StreetComplete financed by the OSMF in the first microgrant round roughly a year ago. Not to mention that there has been numerous incarnations of outdated POI maps over the years.

And while an unpopular solution, in tag only mode in Vespucci (a mode which has existed over a decade), the effort for adding or editing a POI is minimal without having the complexity of being able to change geometry, but still being able to add new objects, and most important you can still see all existing POIs (which is were navigation apps tend to fail) .

But naturally both OsmAnd and organic maps / maps.me are valid solutions for POI collection if you are careful.

In summary there are plenty of tools available and if they were actually used, discussing refinements would be a valid point. As is, you are just asking for somebody to build you another hammer that will rust away in a corner.

PS: you can actually add POIs with voice commands in Vespucci, a experimental feature that has been revamped in version 17 to make it easier to use.

@Rovastar the fact is that most people immediately stop after a couple of edits, so you would expect most people that you contact with a welcome message to have already stopped or that they will do so around the time. Now and then I delay sending welcome messages by a bit 10’000 welcome messages it doesn’t make a difference in the net result.

@all It is hilarious is to spin conspiracy theories around the lack of retention around contributors coming from maps.me, particularly attributing it to “nasty” people. One of the constant problems we have had with maps.me editors was that in practical terms they can’t be contacted at all. There is no reason to believe that retention is particularly low from this source other than that it is a demographic for which this is intrinsically going to be the case.

It should be further pointed out that the overall low conversion rate from “trying OSM out” to a regular contributor has been a constant over the history of the project (and yes that was the case before you make changeset comments too). It is not something that we should fret over (this is a constant theme over all pastimes), we should simply make sure that the contributors are informed enough about the project that if they carry on they are already on the right track. That is something that we don’t seem to be very good at overall and the reason we send welcome messages locally here.

General note: the reason I even started with separating out maps.me numbers was because a large initial one time effect was to be expected and that was going to obscure underlying trends, which continue to indicate growth of reoccurring mappers. Again conspiracy theories are misplaced.

@G1asshouse wrt the numbers, it is simply maps.me demise obscuring the underlying continued growth of “organic” new mapper influx rate, see the two old blog posts. It wasn’t different last year either.

That growth is very slow, but that has always been the case, just as the number of active users has always grown very slowly. Seasonal effects and even individual news articles tend to lead to lots of variation, but there is clearly a bit of growth there, if anything it picked up a bit 2019-2020.

@Zverik in a world in which OSM had unlimited resources perhaps it would be possible to close our eyes to which contributor groups the project critically depends on and which not, but that isn’t reality.

In reality the project could easily survive and still be relevant with the 10% (or less) that contribute and maintain 90% of the data, the other way around clearly not. That doesn’t mean that the map isn’t better when we get input from those additional contributors, but it is also important to understand that many of them are not, as in really not, interested in any kind of long term relationship with OSM. RicoEletric has already covered most aspects of that.

And yes I believe we need to create low-impact, low-maintenance ways for such contributions to happen, that do not create extra load on the 10% of contributors that we really depend on, for example OSMyBiz. It can be argued that MAPS.ME was never that.

The thing is that roughly 2/3s of the annual contributors are new, see annual active contributors. And nearly all of them are short term to single edit contributors (and yes there is some correlation with tools and recruitment method, for example HOT) and while a larger pool of new mappers might lead to slightly more longer term contributors, conversion rates are so low that in practical terms even quite large changes in new contributor influx don’t make a big difference.

@G1asshouse in the 1st half of 2021 influx of new users from maps me / organic maps was still relatively “high” that is 7’881 vs 1’682 in the 2nd half, that is nearly exactly the difference you saw.

Taking a step back, the only metric that really matters for the project as a whole is that of reoccurring long term contributors. To anecdotally put that in perspective, the first 80’000 or so editors that used maps.me contributed less than I have in my OSM career, and I’m by far not a particularly large OSM contributor. The statement should seen less as a dismissal of the long tail of contributors, but more a statement of that the continued viability of the project lies elsewhere.

The question always was if the minor contributors even had a remote idea on what they were adding, there is definitely no indication that it was ever a source of regular contributors in any form.

As a vehicle routing application (I don’t think it ever though of itself as a map viewer) it remains around and organic maps continues to have the editing feature, but with a vastly smaller user base.

https://github.com/simonpoole/ConditionalRestrictionParser has been available and in use for half a decade.

I wouldn’t suggest that it’s a ‘reference implementation’ as the tag itself is woefully under specified and many things about it are not well thought out.

There hasn’t been a long standing question about mapper bump in 2016 except for people that willfully ignore what is going on outside of their bubble, see osm.org/user/SimonPoole/diary/43093

But is there something similiar on the open source market?

The problem with financing free ponies is that somebody has to do it (finance them). Please feel free to organize that, not just I am waiting with baited breath.

See osm.org/user/SimonPoole/diary/34856

The user I mentioned there had waited just under 10 years, so yours is older, but I’m sure there a numerous older ones.

Themed mapping apps are IMHO the emperors new clothes of OSM. As a movement we want our contributors to be effective as possible when they are doing the most expensive activity we have, that is surveying outside. There is probably a role for them for groups that aren’t really interested in OSM, but that’s about it.

I was thinking of something like 1F16 see https://en.wikipedia.org/wiki/Greek_Extended

@mmd I think most of these could easily be found (I’m sure we’ve got a lot of those in German language names too) because of the fixed mappings.

@bryceco given the small number of strings and that the community is likely small I wold agree that simply contacting the contributors in question with the list of problematic strings is the best idea.

@mmd I’m not convinced that mbsrtowcs does validation outside of ensuring that the input is well formed UTF-8, aka I don’t think it actually checks if the code points are assigned or not.

On the general issue: I’ve done some experimenting on input validation in Vespucci with regexps using unicode character classes and code blocks. That is validating that the input string is in the expected script and this seems to work reasonably well. Naturally you can really only output a warning and you need to allow emojis and so on, but it is probably better than nothing, for example you could warn if a name:mm tag contained non-Burmese characters and the same for name tag on an object located in Myanmar.

Isn’t it more correct to say that “OSM assumes that tags are encoded in valid UTF-8 but the API does not validate that this is actually the case”? This isn’t a surprise as validation of any kind outside of referential correctness has always been left to the editing apps.

My questions from a developer to the EWG would be, assuming that this will be a new requirement on editors

• is it reasonably possible to validate input strings against UTF-8
• is it reasonably possible to validate that input strings are using likely correct code points for the language/script in question?
• what libraries are available for the major programming languages that might support the above?

Obviously the use of characters not in the local script in names might throw a spanner in to validation (consider names with emojis in them), but maybe I’m missing something.

In general if the EWG considers this top priority then the 1st step should be to stop the bleeding, that is to add validation code to the editors so that the problem doesn’t get worse.