Now I see: Mindblowing! Thanks, Daniel, for sharing this - even with code. I’ll forward this to some channels.

Thanks for the nice blog post.

But Z curve is not forgotten :) see e.g. this discussion about GeoParquet partitioning

Regarding GeoParquet partitioning: I’m interested in evaluating the best spatial partitioning scheme for generating GeoParquet files in order to get the best performance when reading them within a bbox. => Which algorithm do you recommend? Z curve? My favourite is KD/KDB Tree. But there are also Quad Tree, Hilbert Curve, etc.

By the way, do you know this nice post?

Pour info : je suppose que tu connais WaterWayMap https://waterwaymap.org/#map=2/0/0 et « Routage avec des lignes à travers des polygones » https://blog.rustprooflabs.com/2022/10/pgrouting-lines-through-polygons .

I’ll also have a look at GeoParquet and GeoDesk when I find the time. They might contain additional ideas I overlooked.

Very good. I would definitely take a look at GeoParquet 1.1 https://geoparquet.org/ - and if necessary contribute to it.

I respect in-house developments like Oma, but the chances of a new format like GeoParquet catching on are much greater.

Take a look at the long list of GeoParquet (and Parquet) software. And see also, for example, these interesting discussions on the subject here: https://github.com/opengeospatial/geoparquet/discussions/251 .

Dear @kumakyoo. Interesting project. I haven’t yet understood all your requirements. But have you already looked at GeoParquet?

And regarding the time consuming styling: Soon the QGIS plugin “AIAMAS - AI-Assisted Map Styler” will also support line and area symbology. This plugin allows quick symbolization and automatically suggests fitting colors and symbols for newly loaded vector layers. Feedback welcome! https://plugins.qgis.org/plugins/AIAMAS/

Hey 2hu4u: You can share QGIS symbology using the QGIS Resource Sharing plugin https://qgis-contribution.github.io/QGIS-ResourceSharing/ .

I didn’t understand all your queries, but I often recommend the following output in combination with “NWR”:

...
//out body; >; out skel qt;
out center;   

I’d be very interested in this feature implemented as part of OSM.org website.

Seem’s that it’s Microcosms was renamed to communities https://github.com/openstreetmap/openstreetmap-website/pull/3717 .

But the project name really only is about the functionality in osm.org . So the project name will vanish one day when it is merged.

@OpenBrian : Can you give an estimate when the pull request will be ready to be merged?

Hi Mateusz: I’ll forward your comments to Timon, the maintainer of the OSM Wikidata Quality Checker, who is now working in my institute IFS at OST.

Hi Mateusz: The code repo has been published in the mentioned Wiki page osm.wiki/OpenStreetMap_Wikidata_Quality_Checker now also as “Project repository”.

Nice work! You know who’s the most famous armchair mapper? The Geographer living on the sixth planet in the novella The Little Prince by Antoine de Saint-Exupéry.

Yes. It would be nicest if he followed up now with actions after all the words.

Specifically, he and cartographers and db engineers could try to understand the pipeline for generating VT and then improve it (see e.g. here as starting points https://github.com/systemed/tilemaker/blob/master/docs/CONFIGURATION.md or here https://github.com/openmaptiles/openmaptiles/blob/master/layers/landuse/landuse.sql z13 - z6).

And if that is too much to expect - because of programming knowhow (Lua, SQL) - then I would be happy with a contribution on a scientific map generalization workshop or an ICC paper :-).

It tells us that OSM data schema is random/unstable/illogical and is getting worse.

Please stop these insinuations and false statements. As I said, there is hardly any scientific research left on the topic of data quality, because the OSM data is proven to be fit-for-use. See the 100+ papers on this: https://scholar.google.com/scholar?q=openstreetmap+quality

It is as diverse as people changing that schema without any common goal or strategy.

The possibility that people can change the schema is an advantage and even a necessity among other things to be able to cope with local situations. And the common goal is a digital representation of the world.

You ALWAYS have to convert it into something usable.

Exactly. And that’s because OSM has a key-value schema, which you typically want to convert it into a tabular (GIS) form. This already includes classification and aggregation: See not only the Geofabrik document (https://download.geofabrik.de/osm-data-in-gis-formats-free.pdf ) but also https://openmaptiles.org/docs/ or my OSMaxx schema documentation https://github.com/geometalab/osmaxx/blob/master/docs/osmaxx_data_schema.md .

Typification is one of generalisation operators (…) … It is one of generalisation operations which it would be completely impractical to do as a post-processing of data from vector tile on client side.

Agreed that post-processing based on Vector Tiles (VT, specifically Mapbox VT format) can be impractical - though that could be still an option if you want to scale up. See for example the OSM QA Tiles https://osmlab.github.io/osm-qa-tiles/ .

The typification you describe fits very well with the pipeline, procedures and tools needed to generate VT. See e.g. https://github.com/openmaptiles/openmaptiles-tools or https://github.com/geofabrik/openstreetmap-carto-vector-tiles/blob/master/README_VECTOR_TILES.md .

This seems to be a good place to engage yourself. And that what I meant when I said that I’d wish NMA and cartographers could do more.

To sum up and to clarify especially for you:

• OSM is a “scaleless” database, not a “cartographic” database.
• Automated generalization like you are looking for does not belong in OSM database. That is something downstream.
• When I say post-pocessing, I mean that OSM data must always be (post-)processed: first to tabular GIS data, then to further purposes like a (generalized) base map.

To make it clear again: Expert people like cartographers and geodesists often initially imply that OSM is of lower quality than “official” data.

However, research on geodata quality (completeness, logical consistency, positional accuracy, temporal quality, thematic accuracy and usability) has become uninteresting since everything points to OSM being fit-for-use.

Lewandowski & Specht (https://doi.org/10.1111/cobi.12481) go even further and state “Collectively, these data suggest that volunteer data are not consistently more variable than expert data”.

What is in fact could be enhanced in OSM are tools and map (visualizations) that indicate e.g. completeness. Here’s a shy attempt from some of my students https://eprints.ost.ch/id/eprint/940/ .

Correction: I wrote “Even cartography found an agreement of POI classes.” whereas I meant it obviously did not find a classification for POIs - even not a simplest common one: See my wiki here https://giswiki.hsr.ch/POI#POI-Kategorien . And it’s no need: It’s up to you to define the purpose of the derived map.

Summarising: you’re saying OSM is not for cartography.

No. I’m saying it’s inevitably homogeneous to a certain degree but still fit-for-use for cartography among many more usages.

to my knowledge most (all?) uses of OSM in professional way are transforming OSM data to some proper schema and then doing serious clean-up/postprocessing (and that is BEFORE doing any generalisation).

Exactly! That’s the workflow: data > postprocess > render. There is one (OSM) dataset of the world serving many purposes (base maps, routing, etc.). “Don’t map for the renderer”, i.e. don’t impose varying needs from downstream processes to the spatial data source.

What does that say about OSM data schema?

It tells us, that the OSM data schema is as diverse as reality is and it’s changing as reality is changing (see e.g. childcare). There is a continuum between a “stable” and centrally defined schema and a heterogeneous, decentrally defined schema, where NMA and OSM stay an both ends. The latter is much more up-to-date (meaning minutes versus years) containing 1001 POIs (versus some dozens) and which allows routing.

The real current challenges for both, NMA cartographers and OSM enthusiasts is to look for integration and synchronization (post-)processes.

Another interesting challenge would be to typify, as you suggested. Here, one could look into editor presets and semantic nets. I’d be happy to look at your open source repo about your work.

You wrote

there is currently no QA on the changes of tagging schema

This is a crowd sourced project and “changes of tagging schema” are heavily discussed.

schema is changed by a tiny group of people

No; Tags are changed sometimes changed by too many people.

most of whom have absolutely no understanding of the subject matter

Don’t be arrogant - put your arguments forward.

and at times moving to being less useful from cartographic perspective.

Processing OSM data for a cartographic product is like any other big data project: see the V’s von Big Data especially variety. If something is not new here, then it’s the scientific research which attributed OSM as definitely being “fit enough” to be even compared to “professional” data.

Cartography is pretty sensitive to proper/logic tagging schema - everything is interconnected: say for building typification on small scales we have to turn the building symbol towards the closest way (…) can that be a QA rule for how far a building must be from the road? what type of the building?

You speak of minimal distances to maintain readabiliy at a certain scale? That is entirely part of cartographic postprocessing.

Regarding classification and waterbody tagging: This is indeed again doable in OSM data processing. Many interesting questions to solve and publish here.

Pls. look at the papers about maps for the blind where - like with processing for Vector Tiles - it’s about generalization in extremis and to loosely cite two statements from there: There is no automated process available yet (except Mapy and in closed ArcGIS Pro). And there are no attempts to integrate data from different NMA’s. And regarding proper classification: Even cartography found an agreement of POI classes.

can you give a short description of idea: how do you typify buildings coming in a vector tile?

By tags and postprocessed attributes like “class” and “type” (or so) in OpenMapTiles.

Vector tiles is not a new technology.

Postprocessing OSM data to Vector Tiles or to any other map oriented db is still an area with much innovation potential to apply and extend cartographic map generalization (wich I know better than you suggest).

“Did any presentation of the ICC 2021 or a previous ICC have Vector Tiles as a topic (except “swisstopo Vector Tiles”)?” - yes of course.

I still can’t find much papers (actually none at ICC) about vector tiles and open source repositories - especially not vario-scale maps.

My main point is really not to bash NMAs but to say: Don’t expect more from OSM community than what the cartography did not achieve.

As a mapper it’s up to you to do interesting cartographic stuff with OSM. And reach out the NMA and cartographic researchers to tell them, what they could do more with using and integrating OSM with NMA data (especially regarding base maps, POIs, geocoding, and routing).

Tomas

I hope you aren’t a troll. You repeat that there would be a “total lack of any QA”. As a long time mapper you surely know an activity and a tool from here osm.wiki/Quality_assurance . So pls. explain yourself what you are referring to.

You wrote: > there is a small number of people who do not understand cartography and are systematically … > destroying data value (because of a total lack of any QA)

I still can’t follow you - pls. give specific examples to… 1. What has QA (of source data and/or tagging schema) to do with cartography? 2. Where was “data value” being destroyed in OSM?

if you have access to stable data it is hard to convince to contribute to OSM anything more than thematic/specific data.

Many NMA mappers think at the beginning that they can just dump their (allegedly) stable data into OSM. As a long time mapper you know that this is not possible - the QA of OSM says that data should not be inserted twice and the topology must be correct.

stabilise the base dataset

The OSM dataset has proven that it is stable enough to serve typical geospatial use cases including professional cartographic products.

Regarding open source projects: have you heard about CartAGen?

Yes. This exception confirms the rule (i.e. my statement above).

I think the magic part of cartography happens while creating those vector tiles, not while colouring them

You seem to underestimate the visualization possibilities of Vector Tiles.

But yes: Vector Tiles contain highly generalized graph-less data in the upper zoom levels: see e.g. the production process of OpenMapTiles. But nobody prevents you from performing model generalization and cartographic generalization on this data.

And the topic of this post was about lack of cartographic research/science/innovation

Yes. But you can find the people you need at the NMAs, universities and professional map publishers.

Did any presentation of the ICC 2021 or a previous ICC have Vector Tiles as a topic (except “swisstopo Vector Tiles”)? I can find presentations about this at State-of-the-Map and FOSS4G.

Many cartographers consider OSM as competition to NMA data, some deliberately denigrate OSM. But OSM is a complement.

Why should the OSM community get involved in such a congress? Hardly any cartographer or researcher using OSM has done an edit in OSM yet - as recommended by Prof. Muki Haklay. And I hardly know any cartographer who is involved in an open source project - at least not in Europe.

Regarding what cartographers are or are not doing with OSM. Besides the data there is currently no group(s) doing cartography research/inventions in OSM which cartographers could get involved into.

Do you have an explanation what prevents cartographers to contribute to the OSM carto style https://github.com/gravitystorm/openstreetmap-carto/blob/master/CONTRIBUTING.md or to publish a professional map style based on known vector tiles?

But my whole point is that besides data there is nothing being created, invented in OSM which you could present in Cartography conference.

Once again: My “castle dossier map” is an example presented at ICA2022 as well as the “swisstopo lightbase map”.

Oh: You are in the commission of generalisation and multiple representation? I know some researchers there. This reminds me how still visually (even paper) oriented many established (NMA) cartographers are thinking when speaking about “multiple representation”. Any properly desigend spatial database is “scaleless” and capable to serve to automated processes which represent mutliple representations.

My main point is, that cartographers - especially from NMAs - are indeed interested in (semi-)automation - but only under their full control and only concentrating on their own data.

There are shy initiatives coming up, like the MOU the US Chapter made with US-Gov’t. and smaller NMA cooperations. Then there is e.g. some research in maps for the blind with heavy use of OSM data, like in theses not-so-recent-anymore papers: “Automatic Derivation Of On Demand Tactile Maps For Visually Impaired People: First Experiments And Research Agenda” (2019). https://hal.uca.fr/hal-01980146/document and “Automatic (Tactile) Map Generation—A Systematic Literature Review” (2019). https://www.researchgate.net/publication/334072539_Automatic_Tactile_Map_Generation-A_Systematic_Literature_Review

P.S. I’m going to publish a study soon here about “OSM for administrations” osm.wiki/DE:Public-OSM_Partnership .