The positive numbers would still fit perfectly well into uint64, so that wouldn’t be much of an issue here. osm2pgsql uses an internal node cache that derives some block numbers from object ids. If those ids happen to be too large as in this case, block numbers exceed internal limits and eventually the whole processing aborts. Renumbering makes sure there’s no conflicting ids while at the same time keeping object ids in a reasonable value range.

Positive ids in ms-ml-buildings-v0.3-positive.osc.bz2 appear to be way too large and would cause a segmentation fault or an assert error depending on which osm2pgsql version you’re using (https://github.com/openstreetmap/osm2pgsql/issues/965). Renumbering via osmium before running osm2pgsql is definitely required here.

Also, don’t use the corresponding osc file with negative ids. osm2pgsql will no longer support negative ids in the future and your workflow will simply break (https://github.com/openstreetmap/osm2pgsql/issues/1097).

Maybe you could take that to your Slack feedback channel as well for those folks not following your blog post.

Re negative ids and osm2pgsql: Are you aware of https://github.com/openstreetmap/osm2pgsql/issues/1097 ?

After your Easylist stunt 4 years ago, which banned our fundraising banners on osm.org, a lot of people probably put you on their Github block list. Get over it.

For context: https://github.com/openstreetmap/openstreetmap-website/issues/1333#issuecomment-255306223

My recollection of https://github.com/openstreetmap/mod_tile/pull/152 was that it was part of a discussion about whether what role OSMF and osm.org should play (should osm.org “compete with google” itself, or should it enable others to do so

I don’t recall that part of the discussion. Do you happen to have some more details?

To put it very simple, I see this issue as revisiting the topic of which tile gets rendered at which time, and applying any optimizions as needed. That in turn might help us serve more users by making more effective use of available hardware, or even use less hardware. I’m quite confident that the core parts of mod_tile that are now about 7 years old (and the original author described as “good enough at the time”) would definitely benefit from doing some rework, e.g. use more modern standard instead of some bespoke code, as @apmon outlined in pull request 152.

Does someone want money to work on mod_tile? Who? What’s the proposal? How much?

I can’t really offer any insights into the commercial part here.

Given that the mod_tile project is quite dead for so many years, you can’t assume that people will reach out to the board, in this case simply because there’s noone left. That doesn’t mean that the topic is irrelevant, though.

Essentially, there needs to be some way to bring important infrastructure parts back into active maintenance. That could mean that you need to actively reach out to the community and possibly find some contractor.

Here’s mine: “These software tools are very popular, and the people behind it have a track record of delivering things the OSM community benefits from”. Is that OK?

I have some doubts if a popularity contest will cut it here. How would you ever get some life back into those critical parts of our infrastructure, that haven’t seen any updates since 5 years, and have no maintainer anymore? As you might have guessed, I’m talking about mod_tile here. Others have pointed out serious issues in the queue implementation, that haven’t seen any in depth analysis or issue resolution at all since 3.5 years (https://github.com/openstreetmap/mod_tile/pull/152).

I don’t really care where/how osmf spends their money, or how strategic that investment is. I would judge those projects on a “critical infrastructure component that desperately needs some work” basis.

The only other bits in tier 1 that I’m aware of are Planet file generation and replication diffs, at least according to the OWG primary services definition. It really isn’t that huge a number of tools, as an unsuspecting reader might have guessed.

Minutely replication diffs still uses osmosis, which is soon to be replaced by osmdbt. For the hourly/daily diffs, we could either stay with osmosis or switch to osmium-tool. Depending on how that decision is going to be made, we might end up using libosmium (as part of osmdbt) and osmosis (for merging hourly/daily diffs), both of which again have a large overlap in relevant functionality and have been independently developed.

I find it a somewhat difficult to come up with a general rule for tier 1 here, except for the only rule is the exception, maybe?

Sorry - you lost me here. What different use cases are there? AFAIK these are developed for a single use on OSMF infrastructure. What other use cases do you have in mind? Things like opengeofiction?

Good question, it’s not related to any external projects, we’re only looking at osm.org topics here. Andy always wants to have a development environment that is easily approachble for new devs, hence the need to have a Rails port offering the full functionality. For performance reasons, we’re switching to CGImap for production. Both solutions have been independently designed (and their design details differ quite a bit), and serve different purposes. They’re only as much entangled, as they’re using the same database tables, and share the same functional requirements.

In any case this whole part of the discussion is besides the point of course since my point was that tier 1 usually does not call for diversity in different independently designed and developed solutions while the other tiers do

Yes, that’s exactly the point I disagree with - even for tier 1, a “one size fits it all” approach doesn’t work all the time.

But i would be very surprised if in this case the duplication of work was not at times put into question.

As I said, it’s a policy decision, so in theory you can put the double effort into question, but it doesn’t change anything.

possibly more related to the plan to throw one away strategy.

No, that isn’t correct either. Both worlds have their merits, it just depends on the use case you’re lookling at.

In tier 1 you naturally concentrate on exactly one toolchain to work reliably - replacing tools as they become outdated and incompatible but not usually developing alternatives for permanent use in parallel.

Just wanted to point out that your statement is inheritely incorrect in case of the Rails port and CGImap. Any development in CGImap needs to have its corresponding Rails development in place. Effectively, this doubles development efforts. We do this for policy reasons to have both worlds compatible at all all times, even though one of them isn’t used in production to some degree.

Also, I would recommend to try this sort of thing on the dev instance first and get familiar with how this API stuff works -> https://master.apis.dev.openstreetmap.org

There’s some real danger of creating a huge mess, if you don’t know what you’re doing. So don’t try this at home on the prod instance :)

For the rest of us: you can do the same thing in JOSM without any command line fiddling:

1. Download respective area
2. Change objects as needed
3. Hit the Upload button, upload dialog opens up.
   • Go to “Changesets” tab
   • Press the refresh button next to “Upload to an existing changeset”. This will populate the dropdown control with a list of currently open changesets
   • Click on “Upload to an existing changeset”, and pick your changeset in the dropdown list
   • Continue uploading changes like you’re used to

The following Github issue seems to have a bit more context and background information on that topic: https://github.com/fossgis/osmdata/issues/7

Take a close look at the following answer: https://help.openstreetmap.org/questions/20531/overpass-ql-nodes-and-ways-in-area

I got some feedback that https://s3-ap-northeast-1.amazonaws.com/overpass-db-ap-northeast-1/augmented-diffs/ is no longer being updated since quite some time. Is this a known issue? Is the URL still correct?

You can ask a moderator to hide it.

Query was the second one “huts, shelters, shalets, … “. In your query you use “nwr” which returns nodes, ways, and relations. However, for the ways, you only receive the “node ids”, not the actual nodes with lat/lon details.

togpx cannot create anything meaningful in this case, as there’s simply no geoemtry available. Please compare:

• way(34996405)[tourism=chalet];out;
• way(34996405)[tourism=chalet];out center;

I believe one of your example queries does not return any geometry information for ways, i.e. you’re probably missing out a few thousand objects. Maybe try this one here: https://overpass-turbo.eu/s/QHD

Needless to say that this rather senior mapper is also an administrator of the OpenStreetMapOrg Telegram group. I hope that’s not the new way of leading by example.