Please help with Prishtina DXF Import
Käyttäjä h4ck3rm1k3 kirjotti tämän 24. Januarya 2010 kielellä EnglishI have here a dxf file of Prishtina in high quality.
http://www.archive.org/details/PrishtinaStreets
I have been able to convert it to lat/lon like this
cs2cs +proj=utm +zone=34T +ellps=WGS84 +datum=WGS84 +units=m -f \"%.7f\"
Problem is that most of the points are in relative position, there is a grid.
I have pushed a version of two nickels that can process alot of this :
To git@github.com:h4ck3rm1k3/TwoNickels.git
c8e4009..41f23f1 master -> master
It produces osm files with data in three groups :
1. points that are exact on the mark, (few)
2. points on the equator (0,0) relative to something
3. points that seem to be shifted over from utm34T
720496.25, 506444.9375 -> 22.987450699423831679, 4.5791053696411907481
I have created averages of good points versus the shifted ones and
came up with these constant shifts:
$n -= 187380;
$e += 4228672;
They are still 15km off of prishtina..
So I would like one of you to please help with the autocad and
transform all coordinates into some single form so I can convert them.
Or please advice on these points.
thanks,
mike
Discussion
Kommentti käyttäjältä Former OSM contributor 24. January 2010 klo 14.04
h4ck3rm1k3
Have you tried opening the DXF in AutoCAD and moving all the data with a known co-ordinate onto the 0,0 origin, then move the data again using that known co-ordinate with the @x co-ord,y co-ord input. Then re-export (DXFOUT) the data?
What sort of data within the DXF file are you pulling? Points or lines?
Kommentti käyttäjältä h4ck3rm1k3 25. January 2010 klo 11.19
Hi,
There are points, lines and streets.
I can open the file in qcad, but it is so slow that I cannot work with it.
There are some things not supported in DIME.
but Really, this goes over my head, I would need some exact instructions on how the coords are encoded in the dxf file so that I can decode them.
I dont have autocad or windows...
thanks,
mike
Kommentti käyttäjältä Former OSM contributor 25. January 2010 klo 20.11
Mike
I'm afraid that DXF file format is extremely long winded and text based. If you open it with an ASCII text editor you can see for yourself. The bit that you are interested in starts at the [ENTITIES] header then runs virtually to the end of the file depending upon which version of AutoCAD it was generated from.
Your DIME error may be related to BLOCK or GROUP entities which should be exploded and the file re-saved but as you say in your email to me you do not own AutoCAD. Again, that's something I would be willing to look at. Circles or arcs could also throw up errors. However I can not be sure as QCad is not a piece of software I am familiar with.
Years ago, I wrote GWBASIC programs to extract DXF data but things have gone rusty in the head. By the way, I haven't a clue what CS2CS is or does.
Zip the file and send it to me. I would be more than happy to take a look at the origin and find any information of use or what I can understand. I tried your link at the top and ... a page with nothing except obscure links and no DXF file.
Kommentti käyttäjältä h4ck3rm1k3 25. January 2010 klo 20.14
The file is here the whole time :
http://ia311536.us.archive.org/3/items/PrishtinaStreets/k_prishtina_com.dxf.zip
I am working on using my osm file from dime, converting that to pgsql and the will try and transform the data with qgis, all help appreciated.
mike
Kommentti käyttäjältä h4ck3rm1k3 25. January 2010 klo 20.18
using my TwoNickel to convert to osm,
http://github.com/h4ck3rm1k3/TwoNickels
using my patched version of osm2pgsql
(see patch here http://fmtyewtk.blogspot.com/2009/12/osm2pgsql-hack-for-importing-id-ways.html)
./osm2pgsql --utf8-sanitize --prefix prishtina --multi-geometry -P 5433 -m ~/cur/prishtina/K-Prishtina-Com.dxf.osm
It is now loading. Will then try and shift the data around manually with qgis.
Kommentti käyttäjältä h4ck3rm1k3 25. January 2010 klo 20.39
I have made some screenshots of the detail in prishtina, amazing!
http://osmopenlayers.blogspot.com/2010/01/working-on-prishtina-highres-data.html
Kommentti käyttäjältä h4ck3rm1k3 25. January 2010 klo 20.40
Upps, you have to view this with no style in firefox, for some reason the entry is messed up.
Kommentti käyttäjältä Former OSM contributor 26. January 2010 klo 15.04
Mike
I have the drawing open and see that thre are many different layers with services detail such as water and telephone. There is a big problem for anyone trying to extract data from this drawing, all the lines are short unconnected segments so there isn't any way to extract tracks from streets etc. Data for buildings is placed in the Boundaries layer. If you want to persist here is the co-ordinate of the Grid Cross in the Stadium X = 723814.43 Y = 513785.54 Z = 0.00. I think it's a trace over I'm afraid to have any useful data from it!
Kommentti käyttäjältä Former OSM contributor 27. January 2010 klo 08.11
I am leaving this comment again
--------------------------------------------------------------------------------
Comment from Andrew T Finney at Tue, 26 Jan 2010 15:04:24 +0000
Mike
I have the drawing open and see that thre are many different layers with services detail such as water and telephone. There is a big problem for anyone trying to extract data from this drawing, all the lines are short unconnected segments so there isn't any way to extract tracks from streets etc. Data for buildings is placed in the Boundaries layer. If you want to persist here is the co-ordinate of the Grid Cross in the Stadium X = 723814.43 Y = 513785.54 Z = 0.00. I think it's a trace over I'm afraid to have any useful data from it!
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 08.16
X= 723814.43 Y = 513785.54
This coord is off, with a normal projection you wont get any points in prishtina
cs2cs +proj=utm +zone=34T +ellps=WGS84 +datum=WGS84 +units=m -f \"%.7f\"
723814.43 513785.54
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 08.29
The stadium is here :
osm.org/?lat=42.662641&lon=21.157284&zoom=18&layers=B000FTF
that is utm :
north :4723364.5, east 512889.84
723814.43 -4723364.5= −3999550.07 (about -400k)
513785.54 -512889.84 = 895.7
zone 34t according to :
http://www.cellspark.com/UTM.html
Kommentti käyttäjältä Former OSM contributor 27. January 2010 klo 08.46
Do you want me to move the data to the correct position and re-export, remembering that the data is made up of unconnected short/dashed lines and will be next to useless for converting?
Here are some notes I made:
Drawing header is - Regional water supply Batllava GIS-unit, showing water supply from the lake to the city.
Units are in metres.
Again.....
Stadium reference grid cross is:
X = 723,814.43 Y = 513,785.54 Z = 0.00 (about 724 km east, 514 km north)
There is a reference
4
724
000 in x direction
at the same point for y direction
7
515
000
The map is split into rectangular grid 0.75 km x 0.5 km
Drawn or traced over using inserted tif files.
Contains
Surrounding land contours (although z directions are all zero)
District names
water points
sewer points
electricity point (pole transformers?)
boundaries parcels?
garden boundaries
house numbers
numbers for survey points
what look like house outlines
Possible problems
Rogue entities at origin 0,0
Road names are incorrect according to Openstreetmap's current layout and a.n.other.
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 08.52
That is great. Is there a way to get the coordinates shifted over prishtina? I can convert the dxf file. Also, these layers, I can split them from the resulting osm file.
The unconnected lines, we can do that manually.
mike
Kommentti käyttäjältä Former OSM contributor 27. January 2010 klo 09.37
Mike
Tell me the exact x and y co-ordinate for the cross in the stadium so there is no mix up.
Connecting the lines is a mammoth task take it from me, there are literally 10's of thousands. That's why in my opinion it would be quicker to trace over the jpgs I've exported from AutoCAD.
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 09.46
I have added these to my default.style :
node,way northing text linear
node,way easting text linear
node,way ac:northing text linear
node,way ac:easting text linear
node,way ac:color text linear
node,way ac:entityFlags text linear
node,way ac:layer_name text linear
node,way ac:part_of_poly text linear
node,way ac:type text linear
node,way type text linear
Invoke like this:
./osm2pgsql/osm2pgsql -S osm2pgsql/default.style --extra-attributes --utf8-sanitize --prefix prishtina --multi-geometry -P 5433 -m K-Prishtina-Com.dxf.osm
Then I can split by the layer attribute!
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 09.47
Andrew, good point. If you have highres jpegs, we can trace over them!
let me get you that cross. wait.
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 09.53
Here should be the cross on the stadium, the stadium is a bit off in osm:
osm.org/browse/node/587989678
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 09.59
Here is a list of layers extracted from the dxf:
0
0_korniza-rrjeti
1
2
3
4
5
7
antena
A-Photo-Kadastri
BL-I
BL-II
BL-III
BL-IV
BL-IX
BL-IX-1
BL-V
BL-VI
BL-VII
BL-VII-1
BL-VII-2
BL-VIII
BL-X
BL-XI
BL-XII
BL-XII-1
BL-XII-2
BL-XIII
BL-XIV
BL-XV
BOUNDARIES_COMMUNITY
BOUNDARIES_OTHER
BOUNDARIES_OTHERS
BOUNDARIES_PARCELS
Branches
BUILDINGS_1
BUILDINGS_2
BUILDINGS_çati
BUILDINGS_mur
BUILDINGS_ruin
Damm
description box
emer_te-tjera
GRID
GRID_TEXT
Harta 83
hydrant
image
insert pipe
Izohipsat-kryesore
Izohipsat-ndihmese
Janjeve-map
Keqikoll-map
Kufiri i Pr
Lakes and rivers
Layer1
Layer2
legend
LEGENDA
lidhja1
Liqeni
Lypjan-map
Mainpipe
multiple connection
multiple connetion
multiple-watermeter
name-district
Name district
NAME_OBJECT
NAME_OBJEKT
NAME_OTHER
NAME_PLACE
NAME_STREET
ndarje ndertesash
ndarje ndertese
ndarje-ndertese
ndertesa te reja
NETWORK_WATER
NUMBERS_HOUSE
NUMBERS_PARCELS
NUMBERS_SURVEY_POINTS
numrat
Obiliq-map
objekte
objektet
objekti
orientim manhole
orientim- manhole
orientim-manhole
pika-elekrtike
pika-gjeodezike
pika-kanalizimi
pika ndihmese
pika-uje
Pishinë
PLANKOPF
POINT_BOUNDARIES_COMMUNITY
POINT_DETAIL
POINT_DISTRING_HEATING
POINT_ELECTRICITY
POINT_OTHER
POINT_SEWER
POINT_SURVEY
POINT_TELECOMMUNICATION
POINT_WATER
Prishtina e gjere
Prishtina-map
Pr-map
Pumpstation
pus
pusetë
RAMJET
Reservoir
Rezervari
Rrethoja
rrjeti
rruga
rruge e re
rruge te reja
rruge-te reja
ruin
septik tank
SURVEY_NETWORK
symbol-manhole
Text
Text1
Text-izohipsat
trasa
tregues
Valves
valvul uji
vija ndihmese
vija te tjera
water connection
water-connection
water connetion
water-connetion
water-tank
well
Kommentti käyttäjältä Former OSM contributor 27. January 2010 klo 10.01
Mike
You'll have to excuse my ignorance of the co-ordinates, I've only been doing OSM for about 7 weeks. Can you convert and give me the nodes x-y co-ords that relate to metres? I don't want to get it wrong and redo everything. Once I've done the moving which will be this evening, I will need to know where to upload the file as it will not email. 120MB for DXF, 36MB for DWG, although I can split out the layers separately if that makes things easier? I know it will for email purposes.
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 10.06
using this tool :
http://home.hiwaay.net/~taylorc/toolbox/geography/geoutm.html
lat 42.6628371 , long 21.1567513
=>
x=512846.1776844547
y=4723386.444431229
zone=34
Kommentti käyttäjältä Former OSM contributor 27. January 2010 klo 10.21
Interogation of the stadium cross within AutoCAD now I've shifted everything is:
Command: id
Specify point: _nod of X = 512846.177684 Y = 4723386.444431 Z =
0.000000
Please ignore the slightly truncated decimal places. Let me know how you want me to proceed on my previous post?
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 10.23
you can post unlimited data for all eternity on archive.org
http://www.archive.org/account/login.createaccount.php
Kommentti käyttäjältä Former OSM contributor 27. January 2010 klo 10.49
I have tried uploading it and get a server IO error!
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 10.53
I never had a problem with archive.org, you can see some videos on how to use it here:
http://www.youtube.com/watch?v=0PE1b0FujyU
http://www.youtube.com/watch?v=AeKTHP6ty_Q
Kommentti käyttäjältä Former OSM contributor 27. January 2010 klo 11.01
I have registered and filled all the required boxes and also tried 'non-flash' upload. It may be the works server blocking outgoing. I will try it later on today. Mike, where are you in the world? I'm in North Staffordshire in the U.K.
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 11.02
I am in Offenbach germany,
you can use ftp directly.
mike
Kommentti käyttäjältä Former OSM contributor 27. January 2010 klo 11.33
Mike
Will your inbox accept a 12.6MB zip file? I can try emailing the street data to you, followed by the contour data. If so, let me have the email address.
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 21.30
Thank you very much Andrew,
http://www.archive.org/details/PrishtinaOpensourceMap
Kommentti käyttäjältä h4ck3rm1k3 27. January 2010 klo 21.31
http://picasaweb.google.com/Andrew.T.Finney/Prishtina#5431506018759127954
Kommentti käyttäjältä h4ck3rm1k3 28. January 2010 klo 00.12
Ok, I have uploaded the parts after being converted to osm
http://www.archive.org/details/PrishtinaStreetsOSM
the splitter script is also attached. I also shifted the points over to the right by 0.00016.
Kommentti käyttäjältä h4ck3rm1k3 28. January 2010 klo 07.19
It is uploaded now!
Kommentti käyttäjältä kenfast 29. January 2010 klo 02.07
I saw your request for help on the DIME mailing list. I took a look at the DXF file and it does have a lot of "extra" lines in it, as Andrew mentioned. It looks like the "thick" lines are actually rectangular blocks that are filled with small hatch lines. There may be a way to simplify the DXF. I have DXF geometry manipulation code that I have been working with over the last few years based on DIME. If I get to it I may try to write something to simplify the whole file. The source data is pretty amazing in terms of how much detail there is. I have been to Prishtina before, so I am curious.
Kommentti käyttäjältä h4ck3rm1k3 29. January 2010 klo 07.29
I have been able to filter out the streets from the osm file :
http://pastebin.com/f30aa460b
Just start with the layer :"BOUNDARIES.OTHER"
That contains most of the streets. There are other layers contains the string "rruges" meaning roads.
Kommentti käyttäjältä h4ck3rm1k3 29. January 2010 klo 07.47
Also, I want to point out that everybody is invited to help out with Kosovo and Albania. We have been getting alot of raw data and there is so much work to do.
There is so much work that we would need another 10 people working on it !
so if anyone has time and wants to help, all help is welcome!
Kommentti käyttäjältä h4ck3rm1k3 29. January 2010 klo 12.49
I have started a detailed page for Prishtina here:
osm.wiki/WikiProject_Prishtina#Data_Import
Kommentti käyttäjältä malenki 16. February 2010 klo 21.48
It would be very handy to have a table with the files listet (like at iMMAP Kosovo import) for best collaboration with other users.
Kommentti käyttäjältä h4ck3rm1k3 16. February 2010 klo 21.56
Ahh, yes. We dont have this data split up yet. Also we dont want to import all parts... do you have any ideas for splitting?