OK, since part 6 I have changed ship, which sort of solved the IT caos for my part (I managed to get approval for a full IT upgrade on the vessel I left), and came to a much better IT solution. Here I can access OSM with my own laptop, and even do some uploads. I focused on completing the building layer of the Planimetria import, and are now in the progress of uploading that. I will probably need one more day for that.

Well, the IT solution is better here, but the firewall are still too tight. There are a few features we would like to have liberated, such as e-mail and IM, but for the moment we can access our web-mail and use Meebo. I am not happy with the solution, but it is better than nothing.

I have started to ponder about next step in the import. There are plenty of layers to import, but another large layer such as the building layer must wait. I have the TXT_EDIFICACAO (building name), TXT_RUA (road name) and TXT_RODOVIA (highway name/ref) layers in mind when I talk about other large layers. Yes I know that it would have been good to import that together with the roads and the buildings. But mess in the splitting of the layers made that a cumbersome task, it is easier to get the data back from the server than to match the tiles of split layer data.

I am making notes of experience from every move here, and want a total rewrite of the splitting tools for the next import. I would also like to see a tool that can force "closed ways" on all objects in a file, something that would have made the building import a lot easier. i.e. if all ways with more than 3 nodes was wrapped to closed ways, than it would only be a matter of combining the areas (command shift J in JOSM@Mac). I have also installed a plugin that helps me create multipolygon relations with a mouse-click.

… See full entry

I am back at work, and arrived to IT caos. The computer in my office is not working at all, so I have to sneak in at other computers to do my work and daily read of webmics and other routine stuff. I have already sent request to the IT department to resolve the situation.

I am continue to work offline on the Vitoria import. There will be several files with layer data to sync with the server when I get the opertunity. I hope there are a computer or two with Java installed so I can do a few syncs.

Text layers will not be touched until I can work online again, meaning probably when I am back home in a months time or two. This is because the fact that I need to download underlying data to attach to the text, such as the streets for the street names, etc. I also need to find out how I can clean up the code to correct accented names, as my source have crumbled all the accents.

When I get back home I hope that I can get TwoNickles compiled to work properly on my computer, so that I don't have to ask for too much assistance. I also hope to get a splitter program that allows me to split nodes, ways and closed ways for each layer, as well as a script to automatically remove all blank spaces within layer names. That will help me a lot in future imports. I also would like to find out how I can use JOSM better to remove unwanted tags from the import data.

… See full entry

My wife finally got a working computer today, and that is good news for the import of Planimetria de Vitoria. That means I get a lot more time to work on extracting layers.

I am currently working on the building layer, which contains more than 77000 objects (i.e. buildings). I am dividing the import in batches depending on available time. The first batch became too big, so the next couple of batches are fixes to batch one. I will try to have as much as possible done by 10th of February, where I probably go offline again for some time.

There are some layers I can work on while offline, at the same time I will contact the GIS department of a few other municipals to start import process from more municipals. (Most interested in Guarapari and Divino de São Lorenço).

I am working with extracting more information from the PLanimetria de Vitoria import. There are too many layers to keep you updated for each layer, but I am making progress. At the moment, most ways, parking spaces, beaches, and coastline should be updated, though some more work might be needed on a few layers.

If it continues in this speed I might be completed with the import (not the cleaning of the data though) by the easter. I am not making much effort on the cleaning of Vitoria, as I would need to go places and check stuff, and Vitoria is a little outside my daily roaming area.

When this import is completed, I will look into getting the same sort of dataset for Guarapari, here I will be able to do the cleaning as well.

Today I finally got to upload first part of Planimetria de Vitoria import, the layers imported are VIA_* (all roads) CICLAVIA (cycleways) and various footway layers (CAMINHO* *TRILHA CALCADA).
There are still a lot of layers around, with street names (still not imported), buildings, bus stops, gas stations, parking spaces, walls, parks, squares, green areas, beaches. When the complete import is done, than Vitória will be complete. The next will than be to maintain the data (and I will find another town to import).
At the moment, another user have taken over to clean the roads, the import have resulted in a few duplicated roads (basically all existent road data in Vitória), so there is some need to go over the data.
When I get more time, I will try to work on the next layers.

I am working on the import of the Planimetria of Vitoria, a high resolution data set with multiple layers. Thanks to h4ck3rm1k3 I have now a perl filter/chunker, and I have made a chuck-fixer in case a node is missing from one chunk.

Mike's filter is designed so that it can take a list of layer names, and chunk up all ways and poligons containing those layers, that way I can make separate chunks for cycleways, or unclude both unpaved and paved ways in the same layer.
I have also substracted a layer list.

To help me in the tagging, I made a separate preset file for my JOSM, where I have an dedicated button for adding the source=Planimetria de Vitoria tag, that all the imported data will contain.

Ruff estimate says I might have done all roads, footways, cycleways and paths within wednesday morning, which sort of is my deadline, I will also see if I can made a hotel layer, chuch layer and school layer before that, but the way layer have priority.

I found one possible problem though, it seems there is a slight offset between my data and that which is already in based on GPS tracks. I will check if this is a fixed offset, in that case I will align the entire changeset accordingly before upload, if not, than I will need to investigate a bit in how to solve this.

Some time ago, vdepizzol got hold of a AutoCAD .dwg file with PD data of high quality for the entire municipo/city of Vitória, the state capital of Espírito Santo state of Brazil and asked me for help. None of us have access to CAD software, and we where a little stuck.

After pondering on this for too long time I got the idea to contact h4ck3rm1k3, who didn't have access to AutoCAD either, but via his contact net managed to convert it to .dxf, and from there passed it through a program he is making, managed to convert the poligons of the CAD files into poligons on an OSM-xml file. The output archive was HUGE, bzip compressed to 27MB, so uncompressed is probably around 2-300MB, chunked it up into 3 smaller chunks, and I dived in to start working with it.

After drawing close to all the roads in the first chunk, I got stuck, the next chunks didn't load in JOSM, so I opened the xml files in Xcode, and started to clear out the errors. At the same time I asked vdepizzol for assistance in rechunking the larger file. I would like to have it in uncompressed osm-xml chunks of less than 5MB each (yeah, will be a hell of a lot of files), so that I can load areas around where I am working to get a flawless background image to trace upon.

I figured out that making automated import scripts of this would be difficult as the data shows poligons of what is outside the roads. (A future house import from this data should be easy, but the roads are tough).

Anyway, when the roads are in the database we need to do an on the ground verification of the data, to find out what is accessable by cars, what is to be defined as footways and cycleways, any oneways, access restrictions, etc. I think we should be able to import the street names from the data.

I have tried to put into system the tags that need more work on the wiki. In the work of marking stub tags I noticed that some tags that was missing in English had pages in other languages, and even a couple that was redirecting directly to other language pages. I have been able to compile a list of tags that either are missing in English (or extremely short) where other language pages exists.

There are actually 4 languages that have these pages, Finnish, German, Russian, and Italian. I did not check wether these pages are stubs aswell or if they have descriptions.

Since I am limited to the following three languages: English, Norwegian and Portuguese, any translations between these languages I can help with, and during this process I also translated (or rather gave a fuller description) to the special Brazillian key:cep that was only partly described in Portuguese.

The list:
Tag:amenity=hunting_stand - German keywords describing? the tag in English page
Tag:historic=battlefield - Russian
Tag:historic=manor - German
Tag:leisure=water_park - German
Tag:power=minor_line - German and Finnish
Tag:power=pole - German and Finnish
Tag:shop=confectionary - Russian
Tag:wood=coniferous/deciduous/mixed - Russian
Tag:boundary=water_protection_area - German
Tag:historic=monastary - German
Tag:natural=stone - German
Tag:power=cable_distribution_cabinet - German
Tag:leisure=dog_park - Italian

If everybody helps getting these pages to English, and try to keep at least an English language page of every keys and tags to use, than it is much easier to translate the documentation to other languages. Remember that there probably are people understanding (and able to translate to and from) English in every language, while not every community can find people understanding other languages.

… See full entry

While being at work, far from where I can obtain a single GPS track, the Brazilian community have adopted the project TIGER 250, and adjusted it for Brazilian conditions. The project was named Brasil 250 Cidades which means Brazil 250 Cities. One of the important elements with the project is a distance table with the route length between key Brazilian cities, with color codes identifying their status (based on calculated length, not reality).
So what have this to do with me? Simple, as I cannot go out and drive down unmapped roads, I can try to figure out whats breaking routes, and if there are high resolution Yahoo images where such routes are broken, than I can try to fix it. The result? One (or more) green routes on the distance table.
Long term goals? Making the best routable map of Brazil.
In the long run, OSM is turning into the best routable map of the world!

The capture card I bought for my surveillance was, according to vendor, not compatible with Windows, so they had formated the computer (without consulting me, only my wife), and installed Windows XP. Since I have as much experience setting up a Windows Server as maintaining an engine (almost nothing, I know how to change air filters and oil, so I can set up a simple ftp or web service), there are no way I can set up additional services now. (If you wander what I am talking about, ready my two previous entries).

At first I was set to reformat the entire server and install Linux again, than I thought about Linux on a Stick. Now I do not know what to do. I want to set it up on Linux, but since the vendor insists it will not work in other than Windows, I am not sure what to do. At the moment there is a completely passive wlan card, that was intended for remote maintenance. I want it to work in Linux, but the vendor said that the card supported for Linux was 4 times more expensive.

The wlan card inside is not supported by linux, so I use ndiswrapper to get it up and running, and I have read similar drivers are available for video caption, so I want to try it out.

But maybe I will let it all wait until I can afford buying a new computer. The system now is a really outdated Pentium4 system, with IDE hard drives. A 80GB IDE harddisk costed the same as an 350GB S-ATA, so the future is definitely to change the disks to S-ATA.

Well, that means that the WMS/TMS system, and the map updates will have to wait, how long is still not decided.

Yes, I managed some detours today. Gathered a few more tracks that will be entered soon. A few more roads in Guarapari, though that is getting increasingly difficult, Novo Guarapari, and Meaipe. The funds have been low the last few weeks, so there have been little chance to explore, and my wife's computer have been with problems,the result, she occupy my computer. The result of all this, little editing. A very little editing.

Hopefully the problem of my wife's computer should be fixed over the weekend. At the same time I hope that the computer I have made disponible for my maid(nanny/servant/cleaner) and the server for the alarm/surveillance should be operational as well.

I am thinking of making some sort of service on the server to generate updated maps of Espirito Santo for use on various GPS units, and maybe a TMS/WMS service for map data to import. I have some PD maps in PDF and .dwg formats, and havn't found any workable sofware for Mac or Linux, so I have to dive into think tank for solving the import.

If I get this import going, Vitória and Guarapari will quickly be completed. After that I will head for Divino São Lorenço, and the municipals in between. In time the whole of Espirito Santo will be imported this way, one municipal at a time. I have at least one more on the team for this import, maybe more will join when we have the ball rolling.

Guarapari is coming along quite well. Or rather it have come to a point when I have to make plans and detours to gather more GPS tracks. It is almost impossible for me to drive somewhere and get on a new road without starting to make detours. Centro is now as complete as I can get it without gathering street level data on foot, same also for Parque Areia Preta, Muquiçaba, and Praia do Morro. There might be a few roads in the other suburbs, but it is hard to say as the maps I have to compare with are of a bad quality (old data, inaccurate data, or plainly wrong). The other suburbs are places I generally don't go, either because they are very far off the roads I use, or because they are somewhat dangerous to enter.

For gathering data further off, we plan for weekend trips, we have a few in the view, but no concrete plans at the moment. But we will most likely go to Devino São Lorenço, Mimisa do Sul and Saquareem in the near futurem that might tie in a few more roads.

I am trying to figure out how to import data from AutoCAD .dwg files. At the moment I am sitting on a very up to date .dwg drawing of Vitoria, and I guess I can get the same type of data out of Vila Velha and Guarapari too. I will start a project to gather this type of data from all municipals in Espirito Santo as soon as I know how to get it from the .dwg to osm. I will focus at one municipal at a time (and since it is 84 of them, it might take some years to get through, though I have one other on the team at the moment. We might need more hands on the team.

Last Saturday my wife and I, and some friends, decided to go to Venda Nova do Imigrante to eat polenta and listen to rock. But we headed out with a slight miscalculation. As the traffic into Guarapari was terrible (this weekend is holiday in Brazil), we decided not to fill up the tank in Guarapari. My plan was to fill in Viana before we start climbing the mountain, but could not find any suitable petrol stations along the main road, and no obvious signs to one.

"Well, I should have enough to get to Marechal Floriana, so thats no problem" I thought, and I was correct in that, though the warning lamp for low fuel level lit about the time I passed the entrance to Domingo Martins. In Marechal I turned left into the first fuel station I saw, which was closed (bummer), and asked where the next one was.
"About 43km further, or you can go back do Domingo Martins (5km)" a guy said.
So I hurried back to Domingo Martins, where I found two fuel stations closed. The situation was starting to get critical. Could I go 50 km to get the one they where talking about? We stopped infront of Policia Civil and asked for help. It turned out that we knew the same people, the officer on watch at policia civil and us, and they helped out as best they could (2 liters of gasoline transfered from one vehicle to out car) and called up the road to find out if there was another fuel station in Marechal. He came back telling us that there is a 24h post just 3 km from where we turned around. The fuel from the police held until there, and we got the tank topped up. Now we could continue to Venda Nova, and eat polenta.

I will for certain put in the fuel stations, and a few more roads in Venda Nova, Domingo Martins, and I have also a lot of more tracks for Guarapari. There is also a new road bridge in Vitoria, and I have tracks ready to correct that road.

On the way from Guaçui to Guarapari I try to avoid BR-101 without too much detours. I have found a route I like, but I have to follow BR-101 for a short while.

Yesterday, shortly after entering BR-101 between Cachoeira and Rio Novo, traffic stopped promtly and I could see emergency flashes far up ahead. Shortly after I stopped a HGV almost rammed me in the rear, so I decided to zig-zag a little up the file of stopped trucks to get further from the end of the line. Shortly after I started ahead tree other trucks almost collided with that trucks in the rear.

After waiting some time, a slow movement started on the queue (without any traffic coming against us, and after a few minutes we passed the accident site. A car had been totally crushed by an oncoming truck, and there was without doubt deaths. The queue for the other direction was completely halted, and after passing that we started to signal to oncoming cars for them to slow down in time not to make matters worse. A little after oncoming cars signaled us as well, so we thought it might be an accident further ahead, so when we passed an intersection leading to an alternative route we left BR-101, the result is one track from Rio Novo to Marataiz.

Tomorrow I will drive down to Macae for a 4 day course. On Friday we probably continue to Rio de Janeiro to visit ITAMARATY and the General Consul of Norway for various paper work.

The days in Macae will let us explore a little there. The through road I added 2 weeks ago will be connected to the rest of the road network, at least one hotel will be added and a little more. I do not know how many amenities I will be able to add, as I will be on the course from 0700 to 1800 (or later) every day.

It is a possibility that I will go to the neighboring cities, such as Rio das Ostras for a night meal.

We have still not decided on where/how to sleep over in the vicinity of Rio de Janeiro, but most likely Niteroi/Sao Goncalves/Mage, as finding a good pousada in Rio when having no directions is difficult, and hotel is expensive.

My wife have started practicing to get her drivers license. That means I have to find roads with little transit. Let my wife drive a bit with instructions from me, and get back in the house.

Finding these roads makes us discover some roads I would never find else. Today we headed to the contorno, but took the right hand direction instead of the left hand. The right hand direction is a motorway, and my wife didn't want to practice in 110km/h, so we headed up a dirt road instead, all the way up to BR-101 and back.

Tomorrow we will look for another road to explore.

Last night I had to park the car a little away from the house, because of *^&% tourists parking full the street, and while we where away preparing for the party some visitors parked in our drive way (I wouldn't have been able to fit our car in that gap anyway, but I could have parked in the garrage). I forgot my GPS on the dash, switched on, with the charger cable in place. When returning this morning the car was as hot as a owen, the display of the GPS contorted, and I was afraid it was f***ed (technical term). I switched off the GPS and put it in the gloves department for some time and cooled the car down with the A/C. Later it worked as normal. Remember to hide the GPS from the sun when you park for the night.

I have collected some new trackpoints. Some have been edited and uploaded, some will come in the near future.

1) BR-101 - Macae - Arraial do Cabo (Main roads already entered)
2) Guarapari - Piuma - Iconha (The road have been entered by somebody else, but the intersections have been updated)
3) Guaçui (Visited some places I havn't been before, new roads will be entered)
4) Cachoeira do Itapimerim (A few new roads will be entered)
5) Guarapari (Diversions because of road work give a few more tracks
6) Vila Velha (My son was hungry, we had to find a supermarket, new road to be entered)

Have also noted a few facts to be entered, such as maxweight, maxspeed, lanes, turning restrictions, etc.

More to come

I have heard that all Brazilian (federal, stately and municipally) survey and statistical data is by law in the public domain. I havn't seen this law, but are willing to put it to the test.

The Brazilian community have already started on a process of importing IBGE data. This is the large data set from the Brazilian Geographical and Statistical Institute. Some have also contacted a few municipals to get survey data files donated to prepare imports of municipals.

I will of corse try to get the same, will start with Guarapari, and after that take a few other municipals where I feel I do the mapping. But I have a bigger idea. Something that can turn into a extremely huge import.

I would like to get my hands on the hydrographic surveys done by the Brazilian navy devition for hydrographic survey. If I can get this, than all navigational bodies of water in Brazil can be imported, along with all ports, lighthouses, buoys, coastline, beaches, water depths, seabed materials, and much much more.

First step will be to get the survey datas, that might take quite some time as I will need to find out where to send the requests.
Next step will be to analize the data to find out how I can get the possition data out of it, and how things are coded.
Than all possitions needs to be converted to WGS84.
Than I need to make import files in OSM format based upon the various datas, and split into chunks.
Than each chunk of each import type must be cleaned in order to remove all duplicates.
When that is done, I can run it through bulk_import.pl or some other bulk import program.

I hope that some of the cleaning can be done in some sort of graphical interface, such as JOSM.

If somebody have experinces, scripts, tools, or hints that can help me along, feel free to leave them here, or contact me in a private message.

Ok, the plates are on the car, and we have had our first weekend trip, up to Devino Sao Lorenco, Guacui, Dores de Rio Preto and Espera Feliz. Some of the people we visited lives up (up to now) unmapped roads, and we had a sidetrip to Caiana for a local festival, so a few new tracks was gathered.

One of the reasons to choose Dores de Rio Preto is that we used the weekend to baptize our son.

I have also started on tracking more around Guarapari, today I had a trip up to Camarugi (had hoped that I could pass there and back to civilization another way, but that didn't work out). Camarugi seems to be a cur-de-sac on the map.

The problem now is time. I have to prioritize time with my son, as I will go away for 6 weeks of work later this week. That makes editing of tracks, and copying tracks from the GPS difficult, as it hardly can be combined with spending time with a 1 year old kid.