OpenStreetMap logo OpenStreetMap

baditaflorin's Diary

Recent diary entries

I have extracted all the turn_restriction that exist in the planet OSM file In total there are around 500.000 in the OSM Database

2% of them, or around 10.000 relations are broken, most of them because of other people that had added/ modified streets and deleted parts of the restriction relation.

A turn restriction is made of a TO - VIA - FROM relation That means that the relation should be of 3 different elements.

Found over 1000 relations that have a turn restriction relation made of at least 4 members, and going up to 100 members, when the absolute value is a 3 members relation

I have found 9700 turn_restriction relations that are invalid, because users deleted part of the relations, either in editors like ID that does not work people that they will break a relation, or other possibilities

You can see one example here osm.org/relation/1110481/history

I had made a editable Google SpreadSheet where you can see all of the 10000 broken relations, you can download them and report the ones that you had fixed

https://docs.google.com/spreadsheets/d/1ADbJl1QeSQmkfDadwEXtiYbDrYB8a1IT9FPU0ars7U8/edit#gid=0

Location: Center, Cluj-Napoca, Cluj Metropolitan Area, Cluj, Romania

Aim

From my point of view, if 1.000 OSM editors that use JOSM will learn at least one new shortcut that will allow them to save 5 minutes each day, we have 5000 minutes each day. Time that people can use to relax, or they can spend this extra 5000 minutes adding more things to the map.

JOSM Keyboard Shortcuts Cheat Sheet ## Print and Download You can download the full version ( 300 DPI ) from this dropbox link

LATER EDIT: SVG file

Happy Printing ## Request After you print it, p[ease send a photo with it. It would be glad to see in what parts of the world this CheatSheet will Travel.

Technical Details.

I have used Inkscape to design the JOSM Keyboard Shortcuts Cheat Sheet As a start, i have used my 5 year old attempt at a cheat-sheet for Inkscape https://openclipart.org/detail/93745/inkscape-keyboard-layoutv048-colored-path

Final Words

See full entry

Location: Center, Cluj-Napoca, Cluj Metropolitan Area, Cluj, Romania

You can see the Video Explaining the idea here http://youtu.be/YgzuX8juTW4

Text Transcript :

We can make the Turn Restriction tool more automated, by the order that we select the ways that will make out the Turn Restriction.

The first time we select the From The second time we select the Via The Third time we select the To

If you look now, the order is ok But you still need to type the 3 roles, losing ~10 seconds for each TR. My suggestion is to make the Turn Restriction autocomplete the FROM,VIA,TO fields if it detect a logical path between them

If you don`t wanna code that much: Put a experiential feature that allow the autocomplete function to exist, only on the logic :

What was selected first becomes “From”, second becomes “Via”, third becomes “To”. Without a logical check in place. Expert Users will still benefit from this function.

Steps involved in the actual process :

  • Move mouse to the “from” Box -Click on the “from” Box
  • Type at least 2 words, so you will get the “from” suggestion, and not “forward”

  • Move mouse to the “via” Box -Click on the “via” Box
  • Type at least the “v” so you will get the autocomplete suggestion for the “via”

  • Move mouse to the “to” Box -Click on the “to” Box
  • Type at least the “t” so you will get the autocomplete suggestion for the “to”

  • Move mouse to the ok button
  • Click the ok button

A special thanks to all JOSM developers that are making this great tool even greater.

Estatus del proyecto de importación de México

Posted by baditaflorin on 27 October 2015 in Spanish (Español). Last updated on 4 November 2015.

Estatus del proyecto de importación de México

Este documento explica el proceso que tuvimos que usar para importar más la mitad de las municipalidades en México.

  • admin_level=4 significa Provincia (Estado)

  • admin_level=6 significa Municipalidad

Estadísticas rápidas:

En números : Números de nodos/caminos/relaciones borradas

  • 500K / 2k / 500

Números de nodos/caminos/relaciones agregadas

  • 1000K / ~4k / ~1050

Número de horas dedicadas :

  • 200+

Número de municipalidades agregadas:

  • 1000+

Este anuncio está dividido en los pasos que se tuvieron que hacer para lograr la importación. Desde el inicio me gustaría mencionar que es un proceso de aprendizaje y todo el tiempo nos encontramos aprendiendo. Si encuentras algo que no consideras óptimo o si tienes una sugerencia de cómo hacerlo de manera distinta por favor avísanos.

Paso 0 : Transformación de Datos

El proceso está ilustrado en la página Wiki Mexico’s Administrative Divisions Import Project

Paso 1 : Pensar en el flujo del proceso

Debido a cuestiones de tiempo decidimos importar por el momento 21 de las 32 provincias

Tuvimos ayuda de las personas que hicieron la importación de Puerto Rico y me gustaría agradecerles el apoyo y ayuda que nos han dado para esta importación

http://wiki.openstreetmap.org/wiki/Puerto_Rico_Projects/Puerto_Rico_Legal_Boundaries_Import

La diferencia fue que tuvimos los datos un poco más complicados porque no empezamos con un mapa limpio, tuvimos un mapa que también tenía ya límites en admin_level=4 , admin_level=6, admin_level=7 8 9 10

Una de las dificultades técnicas que tuvimos al principio fue esta:

See full entry

Mexico boundaries import status.

Posted by baditaflorin on 26 October 2015 in English. Last updated on 4 November 2015.

This document explain the process that we had used to import more then half of the municipalities in Mexico.

  • admin_level=4 means Province ( State )
  • admin_level=6 means Municipality

Short stats :

In Numbers : Number of nodes/ways/relations deleted

  • 500K / 2k / 500

Number of nodes/ways/relations added

  • 1000K / ~4k / ~1050

Number of hours spend :

  • 200+

Number of municipalities added :

  • 1000+

The post is broken down into the steps that we had done to accomplish the import. From the start, i want to point out that this is a learning process, and we are learning all the time. If you find something thay you consider suboptimal, or you have a suggestion of how to do something in a different way, please do tell.

Step 0 : Data Transformation

The process is illustrated in the wiki page Mexico’s Administrative Divisions Import Project

Step 1 : Think at the workflow of the process

We had help from the people that did the Puerto Rico import, and I want to thank them for all the support and help that they had given to us for this import.

http://wiki.openstreetmap.org/wiki/Puerto_Rico_Projects/Puerto_Rico_Legal_Boundaries_Import

The difference was that we had the data a little more complicated, because we did not start with a clean map, we had a map that also had admin_level=4 , admin_level=6, admin_level=7 8 9 10 boundaries already in place.

One of the technical difficulties that we had in the beginning was this :

  • Find solution on how to deal with the ways that also have another relation in the way ( admin level 4 and 6, let`s say ). That means that we cannot delete them, because level 4 relation depends on the same way and when we will do the import it will create duplicated ways along the line If this is not done by script, we need to manually add around 500 boundaries to the new ways and recreate some relations. ( see the task at Step 4 that needed 70 hours )

See full entry

Location: General Francisco R. Murguía, Zacatecas, Mexico

When you want to do a import, and you start with a SHP file, there are 3 challenges that i had identified :

How do you convert and split the file, in such a way that a way will not be longer then 2000 points ? How do you remove the line duplicates after you explode the file ( at first all of the municipality is a polygon, that means that at common borders, there will actually be 2 ways.

How do you elevate the properties of the shp file, now the information is on the way, and you want to move this information into the relation.

I only get the data per continents, for some types of roads. for europe i only got them for 6 types, because osmosis is not able to load the whole Europe file, at least with a server that have 16 GB RAM and octacore processor. I had a deadline, and from my estimation would have took more then 2 days to load the whole europe, so i had used osmfilter and osmconvert to extract different types of highway types

text

POI nodes with over 100 tags

Posted by baditaflorin on 18 September 2015 in English.

Ever wandered how many POI over 100 tags ? Now you can know.

There are 33 nodes that have over 100 tags, mostly if not all being lighthouse

https://gist.github.com/baditaflorin/8a30a5f45c94f4d2bf8d

I am not able to save the list with the tags also, at least not in OpenJump ( SHP will truncate at 255, other extension does not work ) , QGIS ( Error after the forth value ) ( filled bug request ) Pgadmin (don`t know how to save directly so that i could open it

[Update] A list with all the nodes tags that have over 50 tags

https://gist.github.com/baditaflorin/53db974195c2d1e95745

image

question

My hypothesis is that we could reduce the planet file with 10-100 MB by only removing the unnecessary points that exist on the map.

I am trying to figure it out the correct postgis query to find out exactly this.

I am trying to compare the points of a linesting, and if the degrees between 3 adjacent points is 0 degrees, that means that the point in the middle can be deleted.

The only check in place that i see is to check that the point in the middle is not connected to a point and check that the hstore tag of that node is empty, meaning that there is no value added to the node ( pedestrian crossing, motorway_jucntion, exit_to, etc )

This is just a dump of some ideas and stats that you should not take for granted, the data its most of the time inaccurate , because i run 2 commands at the same time.

But anyhow, its a point of conversation First, i had filtered with osmconvert and osmfilter so i remained with a planet dump, composed of only the tag highway I am trying to import all the roads in the world, after i tried with a direct attempt and did not scale up, i had now spitted the world into each continent

Size = only the highway , saved as a osm.pbf

africa_highway ( 356 MB ) = 46 Minute to import into postgis Asia_highway ( 1318 MB ) = 213 Minute to import into postgis Australia_highway ( 259 MB ) = 14 Minute to import into postgis Central America ( 63 MB ) = 10 Minute to import into postgis

I am now in day 6 of waiting

There is a 16 GB RAM server, octacore environment, running ubuntu 14.10 ( don`t ask why )

I have a 317 GB file generated called copy********n

And there is a file called copy******w that is still being generated, growing by 10-20 GB per day , now is at 76 GB

And a file called copy*******wn at 30 GB

i am guessing that the n means nodes and w means ways, but how much should it grow, procentualy compared with the 317 GB n file ?

6 day stats

The command that i had used to give the command looks like this

sudo osmosis –rbf latest-planet.osm.pbf –wp host=localhost database=mydatabase user=*** password=****

Have you ever asked yourself what are the TOP 25 cities in the world, based on the number of POI in OpenStreetMap ?

To my surprise, the number 1 city is not a European one, but a city in Japan, that have 121.154 POI

I have calculated all the poi from the following category’s : amenity, leisure, shop, sport, tourism, man_made, office

I did not include the historic POI, because when i filtered the world planet, i had extracted the history tag, instead of historic

I had converter the ways into polygons, and then combined the 2 data-sets into one.

Then i had counted, based on the bbox of the city, the number of POI in each of the cities. I will give here the TOP 25 cities by the number of POI

Waiting for your opinions, comments, etc

Yokohama 121154 Paris 113165 Tokio 97078 Kawasaki 90576 Rio Grande 85238 London 84681 Saitama 75783 Berlin 74002 Birmingham 65093 Moskau 61579 München 54533 Essen 53935 Dusseldorf 52342 Stuttgart 50712 Madrid 48335 Vienna 47396 Toronto 47324 Köln 45018 Dortmund 44534 Lyon 44441 Hamburg 44334 Sete 41723 Milan 41317 Osaka 40756 Frankfurt 38128

Location: Center, Cluj-Napoca, Cluj Metropolitan Area, Cluj, Romania

OSM POI age of different cities around the world

Posted by baditaflorin on 25 September 2014 in English. Last updated on 26 September 2014.

Playing with Mapbox Studio and the ability to use the osm_id of a POI, i managed to make a historic map of the POI of 24 selected cities around the world

In a totally random way i will show some of the different type of cities that we have in OSM, in the terms of relevance to people that are interested in going in that city and using osm for POI

Most of the POI in Ankara were added in 2006 - 2008 Alt text

See full entry

After doing the Romanian OSM node density map, i wanted to do something more globally

The map was created by exporting using Qgis Print Composer in 4 different 24806x17716px , because of the limitations of QGIS that does not allow the export of bigger files yet.

The date was obtained using overpass-api, but for simplicity, if you want to do something similar, use the europe planet extract.

There where over 7 million shop,amenity, tourism and leisure POI in Europe that where combined in a single file using the Qgis Plugin > Vector > Data Management Tools > Merge ShapeFiles to one

Using Qgis plugin mmqgis i created a network of around 800.000 hexagons, and using the Qgis plugin “count in polygon” i was able to figure it how many POI are in each hexagon. To get to a more correct answer, i calculated the area for each of the 800.000 hexagons, that is ranging from about 18 square kilometers in south of Sweden to 28 sq km in Madrid, Spain and did a POI/area to get a more exact answer, POI per sq km

TOP 200 POI by number or POI

See full entry