This is the logical way to get it, but isn’t usable from a script because the internal structure of the zip file isn’t easily predicted.

Maybe using a Github tarball and stripping the leading directory would do?

mkdir -p openstreetmap-carto && 
     curl -L https://api.github.com/repos/gravitystorm/openstreetmap-carto/tarball/v4.6.0 |
     tar xzf - -C openstreetmap-carto --strip=1

https://github.com/openstreetmap/openstreetmap-website/issues/1543 has some addntl details on the large number of anonymous notes last year.

I noticed that your demo loads vector tiles from tiles.mapbox.com in addition to the raster tiles from tile.openstreetmap.org (although you won’t see them). And then there’s the Mapbox API key, which I wouldn’t expect for such a demo.

Is this bounty is still relevant in its current form?

As we have found out in the meantime, the proper solution would be much more complex than just exposing the queue depth as a configurable parameter.

Anyone considering this topic should take a very close look at the detailed analysis posted by @apmon first: https://github.com/openstreetmap/mod_tile/pull/152#issuecomment-348805403

FYI: There’s some follow up discussion to this blog post on the osmf-talk mailing list: https://lists.openstreetmap.org/pipermail/osmf-talk/2017-December/thread.html

As a CoC would for sure affect all mailing lists, forums, etc. at some point in time, please consider moving this discussion to a place where everyone can participate, not only OSMF members. The talk mailing list might be a good start. Thank you.

Talking about efficiency. Recent measurements on a performance optimized branch indicate that a full day worth of queries can in fact be processed in a 25h timeframe on just ~ 2 CPU cores. This includes minutely updates, hourly area updates and gzip compression on the webserver.

Yet, the production instance at overpass-api.de frequently uses 6 out of 8 CPU cores, sometimes even 7 out of 8 cores according to munin. So, 4-5 cores are busy doing some work, which adds no real value. Even worse, users are getting “Too many queries” or “Timeout” error messages, although more resources could be made available for more productive purposes. It’s really only a matter of addressing the respective Github issues.

Most disturbing figure in this survey: less than 20% use OSM because they believe it’s the best map in their area (7 out of 37).

Schneller als achavi

… falls das Changeset im Amazon S3 Cache drin ist, ansonsten wird exakt dieselbe Query ausgeführt wie in Achavi, allerdings auf der Overpass Instanz von Mapbox. Da ist nicht ganz so viel los ;)

A new Endless Achavi demo is up, see this post for details.

It’s giving a different perspective to the question: is caching really worth it? Should we rather spend more time improving Overpass performance instead?

JOSM indeed supports downloading along a gpx track, but it would do so by downloading all data in a number of smaller bboxes rather than the data you’re only interested in.

Well, I guess this needs to be merged into the official Overpass API branch first. So far, this is only in my own personal branch and not part of the official release.

Zum Cachen der Changesets: so etwas gab es in ähnlicher Form schon vor 3 Jahren in Overpass API mit den Augmented Diffs, allerdings wurde es mit Version 0.7.50 eingestellt. Aus den Release Notes von damals:

“The Augmented Diffs are redesigned to be always generated on the fly. This is because the Augmented Diffs have been piling up on the server to almost a terabyte of data and we are running out of space.”

Die vorgestellte Lösung fängt aktuell irgendwo im Frühjahr 2017 an, noch ältere (oder auch sehr junge) Changesets müssen also nachwievor über Overpass angefragt werden (Mapbox hat eine eigene Instanz). Insgesamt hat das Cachen aber den großen Vorteil, Last von der Hauptinstanz zu nehmen. Wieviel Platz das Cachen aktuell aber wirklich beansprucht, ist leider nicht überliefert.

Das Thema ist insgesamt etwas umfangreicher und wird auch auf verschiedenen Github-Issues diskutiert (Overpass API, OSM main page und Achavi). Es gibt auch immer noch neue Ideen dazu, und Cachen ist vielleicht auch nur ein möglicher Ansatz. :)

@Adityo: why did you create a new thread for this very same topic on GIS SE. Please keep it all at one location.

Here’s another great question where ICU regular expressions come in handy:

https://help.openstreetmap.org/questions/57029/how-to-query-overpass-to-find-name-tags-with-chinese-characters

@ImreSamu :

and my problem with normal overpass “user” is that very hard to query the first v1:”user” of the object.

If the object currently still has object version 1, the query looks as follows:

way({{bbox}})(user:florinbadita_telenav)(if:version()==1); out geom meta;

(pending on the release of version() == 1, which is available for pre-release testing via http://overpass-turbo.eu/s/pLj)

Otherwise, if the object already has versions > 1, it’s indeed difficult to uncover the original creator. I described this issue here, but unfortunately, there’s no solution for it yet. Today, it would require so set the [date:...] to some value in the past and check the version for that particular point in time, which is not really convenient, especially if you look at a larger number of objects.

So, by checking the .osh.pbf file, you get direct access to all version 1 objects. That same information would be in Overpass API (back to September 2012). However, there’s no easy way to query for those object versions (for many different objects) as of today.

@Adityo: you can only check for a changeset number. It’s not possible to query by changeset metadata, such as some “#hotosm” tags. I don’t think this will change anytime soon. See https://github.com/drolbr/Overpass-API/issues/189

Hello friends of Overpass (btw: no camel casing in the name),

why can’t you check, if your ways with version 1 have been created in same changeset id as one of its nodes? If the way has been created as part of a split, its changeset id should be different from any of its nodes’ changeset id.

That should could probably be done quite easily via changeset() in an if: filter.

Also, I don’t quite get how the osmium filter on “v1” helps to solve the issue looking at the output above. Right now, the version filter used in the Overpass query does exactly the same already.

@RobJN: > Returning to the original issue (“Tickbox on sign-up page to accept push-notifications”), I’m not sure the PR solves this. I would prefer an option at new-user sign-up stage to select to subscribe to newsletters from the OSMF and from your local community. If you opt to get newsletters from your local community then you are agreeing to your email address being passed on to the Local Chapter that represents you region.

Unfortunately, the PR is not geared towards this. Primary target is allowing third parties to get your email address. Let’s take this oauth example:

http://osmlab.github.io/osm-auth/

With this PR in place you would get additional checkbox “see your e-mail”, if the third party requests it. Once approved, the demo app would also have access to your email (obviously this is not available on osm.org as of now).

For your new local community members, this would mean to manually send each new member such a link to a third party app to sign in (pretty much as it is today), and afterwards receive updates via push messages.

Bottom line: in terms of easy community management and keeping new members in the loop as you describe it, this PR may not be exactly what you’re looking for.

Push notifications is a dream for many of us.

It’s getting close. This PR would implement. May need some policy work. Showing support for the PR could help get this moving :) https://github.com/openstreetmap/openstreetmap-website/pull/1431

The current implementation as it stands really has privacy issues, as it would exhibit your personal email address if a third party requests it and you approve it. This won’t go away with some “policy work”.

I hope you realize that some people in the German forum found this really annoying and would immediately quit their OSM engagement! It breaks with a years long tradition of keeping the email address OSM internal, as mentioned in the privacy policy. For sure this will hit some people by surprise as they never expected something like this happen. If you’re a long term mailing list user, you might wonder, wtf? - still people are different and have different opinions and expectations on this topic.

Also, if you don’t authorize an App up-front, there will be no way to send a push notification at all. A user really needs to explicitly consent to it in the first place. Just randomly selecting some area and sending messages to all mappers will NOT work with this pull request on its own.

There are some ideas to mitigate the situation and have an approach that could work out for everyone without compromising privacy, and still offering the benefits of push notifications via email (or possibly some API). I’m not going to repeat that here, see the Github pull request for details.

Thanks for your time!

Have fun :)

Does this imply that an augmented diff may be updated after it has been published? Is there some mechanism to find out as a data consumer that there has been an update? I think for chsngeset analysis this is fine, I see some issues to keep a local db up to date with this approach. Would you agree to this?