Alugu n OpenStreetMap OpenStreetMap

Aɣmis n MiroJanosik

Inekcamen ineggura n uɣmis

Whitebikes update of OSM data from CSV export

Yuzen-it-id MiroJanosik af 1 October 2018 s English Last updated on 3 October 2018.

So I did update of the OSM data that we get as CSV export from WhiteBikes database. I had to take care of few troubles - export is not exactly CSV, data order differs, but at least they are consistent. I think it can’t be automated and changes should be reviewed, I did it by hand as I want to see which data modified. There are around 50 bike stands in the city.

So, lets see what did I have to do:

Prepare list of stands from database, in a format similar to previously retrieved data

  • download new data from whitebikes at * censored *. In case that website is saved as HTML and not as pure text try to use view source + select all - copy - paste into file.
  • It is in format of CSV (spreadsheet) that can be loaded into office if needed.
  • But it is in incorrect format, so it has to be adapted beforehand.
  • Replace all commas by dot. Replace all semicolons by comma.
  • After this change it will be CSV (Comma-separated-values) and it can be used in excel.
  • Though, there may be still some issues - semicolons are sometimes used also in text, not only as separators.

  • import into office spreadsheet editor (excel, calc) and make it nice and aligned by station number:
  • add new first column, fill it with numbers 1-100
  • set other columns for sorting, sort them by number (they may be reordered from database export)
  • align them with the numbers in first column, fill in empty lines for numbers that are not present
  • export into CSV again (see sample file export-2018-09-26-ordered-with-gaps.csv)

  • now, you can compare old and new csv with your favourite compare tool (Meld, WinDiff) and see it nicely aligned, see the differences that happened.

You may have to modify files to match some general differences (for example http to image changed to https, and such), (as example see sql-2017-03-26.csv and sql-2017-03-26-ordered-with-gaps-look-like-new.csv)

See full entry

Adig: Kalmárka, Nivy, Ružinov, District of Bratislava II, Bratislava, Region of Bratislava, 821 09, Slovakia

Toto nie je nový denník, iba slovenský preklad anglického denníka z pred 6 mesiacov osm.org/user/MiroJanosik/diary/39355

Prečo a ako

Na slovensku máme dostupný elektronický kataster. Poskytuje vektorové dáta budov a súpisné čísla budov (na Slovensku máme súpisné čísla Sk:Key:addr:conscriptionnumber ). Tieto dáta sú verejne dostupné a nejaké múdre hlavy z OSM komunity vytvorili JOSM plugin ktorým sa dajú importovať do OSM máp Sk:KaporSKAddress#Import_s.C3.BApisn.C3.BDch_.C4.8Disiel .

Rozhodol som sa že použijem tieto dostupné dáta a importovací nástroj a priradím ich do OSM mapy mojej obce. Potom tie dáta skontrolujem či sú pravdivé. Moje hlavné ciele boli:

  1. aby sa dal v OSM mape nájsť dom podľa čísla na mape. Už sa podľa správ v médiách ukázalo že to môže byť nápomocné pre sanitky v prípade núdze.
  2. aby moja obec mala aktuálne dáta v OSM - aby OSM reflektoval realitu - čo je cieľom OSM
  3. zvedavosť aké presné dáta sú v elektronickom katastri

Moja obec má približne 1600 obyvateľov a 632 domov plus zopár garáží, chatiek, a kôlní. Obec má približne 10 ulíc.

Rýchla časť

Vektory budov mojej obce sú v OSM už nejaký čas naimportované, a podobne aj pre väčšinu Slovenska. Avšak import súpisných čísel sa nedá plne automatizovať a sú tam chytáky. Jose mi pomohol s procesom importu (vďaka!) a mal som to hotové za menej ako pol hodiny; Toto bol môj prvý set dát nahraných do OSM.

Pomalá časť

Import ukázal že v asi 15 prípadoch majú rôzne domy rovnaké číslo. Rozhodol som sa prekontrolovať všetky domy v obci aby som zistil reálne čísla domov (a aj z dôvodov 1-3 hore).

Takže, vytlačil som si OSM mapu priblíženú dostatočne na to aby som na nej videl čísla domov. Chodil som po dedine a značil som si do papiera správne aj nesprávne čísla domov (mal som z toho 4 prechádzky po obci). Po každej prechádzke som spustil JOSM a upravil čísla domov ktoré potrebovali zmenu, odstránil som ‘poznámky z importu’ a pridal som parameter “source:conscriptionnumber”.

Moje zistenia

See full entry

Why and how

In Slovakia we have electronic cadastre available and it contains buildings in vector format with building conscription numbers (in our country it is conscription number Key:addr:conscriptionnumber ). These information are public and we have some smart OSM users that created a conversion possibility to get it into OSM maps KaporSKAddress#Conscription_numbers_import .

I have decided that I will use available method that takes cadastre house numbers and assign them to OSM map buildings in my town and then I will verify how correct these data are. Goal was:

  1. have houses with numbers searchable in OSM. It was found to be helpful also for ambulances in case of emergency to have searchable map!
  2. to keep OSM map up to date - to reflect reality which is goal of OSM
  3. by curiosity to see how precise are cadastre data

My town has approx. 1600 inhabitants and 632 houses plus some garages, cottages and huts. Town has approx. 10 streets.

Fast part

Building shapes are already imported into OSM for most of Slovakia for some time. Though, importing conscription numbers can’t be fully automated and it can be tricky. Jose helped me with import process (thank you!) and it took less than half a hour; This was the first commit.

Slow part

Import process shown that in about 15 cases different houses had the same house number. I decided to resurvey whole village to verify all the house numbers (because of reasons 1-3 above).

So I have printed out zoomed OSM map with visible building numbers on paper. I walked around the town and noted down if something was not correct (I have done this in 4 separated sessions). After each session I started JOSM and corrected houses that needed correction, removed ‘import notes’ from houses, added source:conscriptionnumber.

My findings

See full entry

Adig: Sološnica, District of Malacky, Region of Bratislava, 906 37, Slovakia

Starting with album of unofficial bicycle routes around Malacky. It can be found at http://rudava.mypage.sk , it is of course based on OSM data (and my paper notes) and drawn by hand in UMAP.

I’m expecting to slowly add new tracks as I find them and as I get contributions from other cyclist in the region. I found a local cyclist group “SCK Záhorák Malacky” which will have knowledge about local paths.

Adig: Malacky, District of Malacky, Region of Bratislava, Slovakia