OpenStreetMap logo OpenStreetMap

Please help with Prishtina DXF Import

Posted by h4ck3rm1k3 on 24 January 2010 in English.

I have here a dxf file of Prishtina in high quality.
http://www.archive.org/details/PrishtinaStreets

I have been able to convert it to lat/lon like this
cs2cs +proj=utm  +zone=34T +ellps=WGS84 +datum=WGS84 +units=m -f \"%.7f\"

Problem is that most of the points are in relative position, there is a grid.

I have pushed a version of two nickels that can process alot of this :
To git@github.com:h4ck3rm1k3/TwoNickels.git
  c8e4009..41f23f1  master -> master

It produces osm files with data in three groups :

1. points that are exact on the mark, (few)
2. points on the equator (0,0) relative to something
3. points that seem to be shifted over from utm34T

        720496.25, 506444.9375 -> 22.987450699423831679, 4.5791053696411907481

I have created averages of good points versus the shifted ones and
came up with these constant shifts:
       $n -= 187380;
       $e += 4228672;

They are still 15km off of prishtina..

So I would like one of you to please help with the autocad and
transform all coordinates into some single form so I can convert them.
Or please advice on these points.

thanks,
mike

Location: Xhamia e Llapit, Tophane, Pristina, Municipality of Pristina, District of Prishtina, 10010, Kosovo
Email icon Bluesky Icon Facebook Icon LinkedIn Icon Mastodon Icon Telegram Icon X Icon

Discussion

Comment from Former OSM contributor on 24 January 2010 at 14:04

h4ck3rm1k3

Have you tried opening the DXF in AutoCAD and moving all the data with a known co-ordinate onto the 0,0 origin, then move the data again using that known co-ordinate with the @x co-ord,y co-ord input. Then re-export (DXFOUT) the data?

What sort of data within the DXF file are you pulling? Points or lines?

Comment from h4ck3rm1k3 on 25 January 2010 at 11:19

Hi,

There are points, lines and streets.
I can open the file in qcad, but it is so slow that I cannot work with it.
There are some things not supported in DIME.

but Really, this goes over my head, I would need some exact instructions on how the coords are encoded in the dxf file so that I can decode them.

I dont have autocad or windows...

thanks,
mike

Comment from Former OSM contributor on 25 January 2010 at 20:11

Mike

I'm afraid that DXF file format is extremely long winded and text based. If you open it with an ASCII text editor you can see for yourself. The bit that you are interested in starts at the [ENTITIES] header then runs virtually to the end of the file depending upon which version of AutoCAD it was generated from.

Your DIME error may be related to BLOCK or GROUP entities which should be exploded and the file re-saved but as you say in your email to me you do not own AutoCAD. Again, that's something I would be willing to look at. Circles or arcs could also throw up errors. However I can not be sure as QCad is not a piece of software I am familiar with.

Years ago, I wrote GWBASIC programs to extract DXF data but things have gone rusty in the head. By the way, I haven't a clue what CS2CS is or does.

Zip the file and send it to me. I would be more than happy to take a look at the origin and find any information of use or what I can understand. I tried your link at the top and ... a page with nothing except obscure links and no DXF file.

Comment from h4ck3rm1k3 on 25 January 2010 at 20:14

The file is here the whole time :
http://ia311536.us.archive.org/3/items/PrishtinaStreets/k_prishtina_com.dxf.zip

I am working on using my osm file from dime, converting that to pgsql and the will try and transform the data with qgis, all help appreciated.
mike

Comment from h4ck3rm1k3 on 25 January 2010 at 20:18

using my TwoNickel to convert to osm,
http://github.com/h4ck3rm1k3/TwoNickels

using my patched version of osm2pgsql
(see patch here http://fmtyewtk.blogspot.com/2009/12/osm2pgsql-hack-for-importing-id-ways.html)
./osm2pgsql --utf8-sanitize --prefix prishtina --multi-geometry -P 5433 -m ~/cur/prishtina/K-Prishtina-Com.dxf.osm

It is now loading. Will then try and shift the data around manually with qgis.

Comment from h4ck3rm1k3 on 25 January 2010 at 20:39

I have made some screenshots of the detail in prishtina, amazing!
http://osmopenlayers.blogspot.com/2010/01/working-on-prishtina-highres-data.html

Comment from h4ck3rm1k3 on 25 January 2010 at 20:40

Upps, you have to view this with no style in firefox, for some reason the entry is messed up.

Comment from Former OSM contributor on 26 January 2010 at 15:04

Mike

I have the drawing open and see that thre are many different layers with services detail such as water and telephone. There is a big problem for anyone trying to extract data from this drawing, all the lines are short unconnected segments so there isn't any way to extract tracks from streets etc. Data for buildings is placed in the Boundaries layer. If you want to persist here is the co-ordinate of the Grid Cross in the Stadium X = 723814.43 Y = 513785.54 Z = 0.00. I think it's a trace over I'm afraid to have any useful data from it!

Comment from Former OSM contributor on 27 January 2010 at 08:11

I am leaving this comment again

--------------------------------------------------------------------------------

Comment from Andrew T Finney at Tue, 26 Jan 2010 15:04:24 +0000
Mike

I have the drawing open and see that thre are many different layers with services detail such as water and telephone. There is a big problem for anyone trying to extract data from this drawing, all the lines are short unconnected segments so there isn't any way to extract tracks from streets etc. Data for buildings is placed in the Boundaries layer. If you want to persist here is the co-ordinate of the Grid Cross in the Stadium X = 723814.43 Y = 513785.54 Z = 0.00. I think it's a trace over I'm afraid to have any useful data from it!

Comment from h4ck3rm1k3 on 27 January 2010 at 08:16

X= 723814.43 Y = 513785.54

This coord is off, with a normal projection you wont get any points in prishtina
cs2cs +proj=utm +zone=34T +ellps=WGS84 +datum=WGS84 +units=m -f \"%.7f\"
723814.43 513785.54

Comment from h4ck3rm1k3 on 27 January 2010 at 08:29

The stadium is here :
osm.org/?lat=42.662641&lon=21.157284&zoom=18&layers=B000FTF

that is utm :
north :4723364.5, east 512889.84

723814.43 -4723364.5= −3999550.07 (about -400k)
513785.54 -512889.84 = 895.7

zone 34t according to :
http://www.cellspark.com/UTM.html

Comment from Former OSM contributor on 27 January 2010 at 08:46

Do you want me to move the data to the correct position and re-export, remembering that the data is made up of unconnected short/dashed lines and will be next to useless for converting?

Here are some notes I made:

Drawing header is - Regional water supply Batllava GIS-unit, showing water supply from the lake to the city.
Units are in metres.
Again.....
Stadium reference grid cross is:
X = 723,814.43 Y = 513,785.54 Z = 0.00 (about 724 km east, 514 km north)

There is a reference
4
724
000 in x direction

at the same point for y direction
7
515
000

The map is split into rectangular grid 0.75 km x 0.5 km

Drawn or traced over using inserted tif files.

Contains
Surrounding land contours (although z directions are all zero)
District names
water points
sewer points
electricity point (pole transformers?)
boundaries parcels?
garden boundaries
house numbers
numbers for survey points
what look like house outlines

Possible problems
Rogue entities at origin 0,0
Road names are incorrect according to Openstreetmap's current layout and a.n.other.

Comment from h4ck3rm1k3 on 27 January 2010 at 08:52

That is great. Is there a way to get the coordinates shifted over prishtina? I can convert the dxf file. Also, these layers, I can split them from the resulting osm file.

The unconnected lines, we can do that manually.
mike

Comment from Former OSM contributor on 27 January 2010 at 09:37

Mike

Tell me the exact x and y co-ordinate for the cross in the stadium so there is no mix up.

Connecting the lines is a mammoth task take it from me, there are literally 10's of thousands. That's why in my opinion it would be quicker to trace over the jpgs I've exported from AutoCAD.

Comment from h4ck3rm1k3 on 27 January 2010 at 09:46

I have added these to my default.style :

node,way northing text linear
node,way easting text linear
node,way ac:northing text linear
node,way ac:easting text linear
node,way ac:color text linear
node,way ac:entityFlags text linear
node,way ac:layer_name text linear
node,way ac:part_of_poly text linear
node,way ac:type text linear
node,way type text linear

Invoke like this:
./osm2pgsql/osm2pgsql -S osm2pgsql/default.style --extra-attributes --utf8-sanitize --prefix prishtina --multi-geometry -P 5433 -m K-Prishtina-Com.dxf.osm

Then I can split by the layer attribute!

Comment from h4ck3rm1k3 on 27 January 2010 at 09:47

Andrew, good point. If you have highres jpegs, we can trace over them!
let me get you that cross. wait.

Comment from h4ck3rm1k3 on 27 January 2010 at 09:53

Here should be the cross on the stadium, the stadium is a bit off in osm:
osm.org/browse/node/587989678

Comment from h4ck3rm1k3 on 27 January 2010 at 09:59

Here is a list of layers extracted from the dxf:

0
0_korniza-rrjeti
1
2
3
4
5
7
antena
A-Photo-Kadastri
BL-I
BL-II
BL-III
BL-IV
BL-IX
BL-IX-1
BL-V
BL-VI
BL-VII
BL-VII-1
BL-VII-2
BL-VIII
BL-X
BL-XI
BL-XII
BL-XII-1
BL-XII-2
BL-XIII
BL-XIV
BL-XV
BOUNDARIES_COMMUNITY
BOUNDARIES_OTHER
BOUNDARIES_OTHERS
BOUNDARIES_PARCELS
Branches
BUILDINGS_1
BUILDINGS_2
BUILDINGS_çati
BUILDINGS_mur
BUILDINGS_ruin
Damm
description box
emer_te-tjera
GRID
GRID_TEXT
Harta 83
hydrant
image
insert pipe
Izohipsat-kryesore
Izohipsat-ndihmese
Janjeve-map
Keqikoll-map
Kufiri i Pr
Lakes and rivers
Layer1
Layer2
legend
LEGENDA
lidhja1
Liqeni
Lypjan-map
Mainpipe
multiple connection
multiple connetion
multiple-watermeter
name-district
Name district
NAME_OBJECT
NAME_OBJEKT
NAME_OTHER
NAME_PLACE
NAME_STREET
ndarje ndertesash
ndarje ndertese
ndarje-ndertese
ndertesa te reja
NETWORK_WATER
NUMBERS_HOUSE
NUMBERS_PARCELS
NUMBERS_SURVEY_POINTS
numrat
Obiliq-map
objekte
objektet
objekti
orientim manhole
orientim- manhole
orientim-manhole
pika-elekrtike
pika-gjeodezike
pika-kanalizimi
pika ndihmese
pika-uje
Pishinë
PLANKOPF
POINT_BOUNDARIES_COMMUNITY
POINT_DETAIL
POINT_DISTRING_HEATING
POINT_ELECTRICITY
POINT_OTHER
POINT_SEWER
POINT_SURVEY
POINT_TELECOMMUNICATION
POINT_WATER
Prishtina e gjere
Prishtina-map
Pr-map
Pumpstation
pus
pusetë
RAMJET
Reservoir
Rezervari
Rrethoja
rrjeti
rruga
rruge e re
rruge te reja
rruge-te reja
ruin
septik tank
SURVEY_NETWORK
symbol-manhole
Text
Text1
Text-izohipsat
trasa
tregues
Valves
valvul uji
vija ndihmese
vija te tjera
water connection
water-connection
water connetion
water-connetion
water-tank
well

Comment from Former OSM contributor on 27 January 2010 at 10:01

Mike

You'll have to excuse my ignorance of the co-ordinates, I've only been doing OSM for about 7 weeks. Can you convert and give me the nodes x-y co-ords that relate to metres? I don't want to get it wrong and redo everything. Once I've done the moving which will be this evening, I will need to know where to upload the file as it will not email. 120MB for DXF, 36MB for DWG, although I can split out the layers separately if that makes things easier? I know it will for email purposes.

Comment from h4ck3rm1k3 on 27 January 2010 at 10:06

using this tool :
http://home.hiwaay.net/~taylorc/toolbox/geography/geoutm.html

lat 42.6628371 , long 21.1567513
=>
x=512846.1776844547
y=4723386.444431229
zone=34

Comment from Former OSM contributor on 27 January 2010 at 10:21

Interogation of the stadium cross within AutoCAD now I've shifted everything is:

Command: id
Specify point: _nod of X = 512846.177684 Y = 4723386.444431 Z =
0.000000

Please ignore the slightly truncated decimal places. Let me know how you want me to proceed on my previous post?

Comment from h4ck3rm1k3 on 27 January 2010 at 10:23

you can post unlimited data for all eternity on archive.org
http://www.archive.org/account/login.createaccount.php

Comment from Former OSM contributor on 27 January 2010 at 10:49

I have tried uploading it and get a server IO error!

Comment from h4ck3rm1k3 on 27 January 2010 at 10:53

I never had a problem with archive.org, you can see some videos on how to use it here:
http://www.youtube.com/watch?v=0PE1b0FujyU
http://www.youtube.com/watch?v=AeKTHP6ty_Q

Comment from Former OSM contributor on 27 January 2010 at 11:01

I have registered and filled all the required boxes and also tried 'non-flash' upload. It may be the works server blocking outgoing. I will try it later on today. Mike, where are you in the world? I'm in North Staffordshire in the U.K.

Comment from h4ck3rm1k3 on 27 January 2010 at 11:02

I am in Offenbach germany,
you can use ftp directly.
mike

Comment from Former OSM contributor on 27 January 2010 at 11:33

Mike

Will your inbox accept a 12.6MB zip file? I can try emailing the street data to you, followed by the contour data. If so, let me have the email address.

Comment from h4ck3rm1k3 on 28 January 2010 at 00:12

Ok, I have uploaded the parts after being converted to osm
http://www.archive.org/details/PrishtinaStreetsOSM

the splitter script is also attached. I also shifted the points over to the right by 0.00016.

Comment from h4ck3rm1k3 on 28 January 2010 at 07:19

It is uploaded now!

Comment from kenfast on 29 January 2010 at 02:07

I saw your request for help on the DIME mailing list. I took a look at the DXF file and it does have a lot of "extra" lines in it, as Andrew mentioned. It looks like the "thick" lines are actually rectangular blocks that are filled with small hatch lines. There may be a way to simplify the DXF. I have DXF geometry manipulation code that I have been working with over the last few years based on DIME. If I get to it I may try to write something to simplify the whole file. The source data is pretty amazing in terms of how much detail there is. I have been to Prishtina before, so I am curious.

Comment from h4ck3rm1k3 on 29 January 2010 at 07:29

I have been able to filter out the streets from the osm file :

http://pastebin.com/f30aa460b

Just start with the layer :"BOUNDARIES.OTHER"

That contains most of the streets. There are other layers contains the string "rruges" meaning roads.

Comment from h4ck3rm1k3 on 29 January 2010 at 07:47

Also, I want to point out that everybody is invited to help out with Kosovo and Albania. We have been getting alot of raw data and there is so much work to do.

There is so much work that we would need another 10 people working on it !

so if anyone has time and wants to help, all help is welcome!

Comment from h4ck3rm1k3 on 29 January 2010 at 12:49

I have started a detailed page for Prishtina here:
osm.wiki/WikiProject_Prishtina#Data_Import

Comment from malenki on 16 February 2010 at 21:48

It would be very handy to have a table with the files listet (like at iMMAP Kosovo import) for best collaboration with other users.

Comment from h4ck3rm1k3 on 16 February 2010 at 21:56

Ahh, yes. We dont have this data split up yet. Also we dont want to import all parts... do you have any ideas for splitting?

Log in to leave a comment