OpenStreetMap
I am on a roll mapping pedestrian crossings (or ‘crosswalks’ as Americans tend to call them.
First I download a sliver of the map that covers a major road in JOSM:
I think you could also use ‘download along way’ in JOSM if the road is not nice and straight, but around here they usually are.
Then I pan along the way and add crossing nodes using Shift-R to quickly copy pedestrian crossing tags from the previous node.
I have been on a bit of a MapRoulette binge lately. MapRoulette 2 is coming along nicely and we are at the point where we can start working on the front end. This is where a lot of your suggestions come in. If you have more ideas about how MapRoulette should (not) work, please take a moment to go to that PiratePad and add them. Thanks!
MapRoulette 1 is still very much alive however! Let’s look at what has been happening.
Last week, I posted new Sidewalk Mapping challenges (Tampa, Salt Lake City, your city?) to help OSM become a better map for getting around as a pedestrian safely in cities in the United States. Given that someone on foot on U.S. streets was hit by a car about every 8 minutes in the past decade, we could use better maps to help prevent accidents.
Speaking of pedestrian safety! Dr. Stefan Keller, a long time OSM enthusiast and founder of the Geometalab at the Hochschule für Technik, Rapperswil launched a really cool initiative to detect missing crosswalks based on analysis of both OSM data and aerial imagery. The results are making their way to MapRoulette, too.
This diary also appears on the ImproveOSM blog. Follow ImproveOSM there or on Twitter to stay informed of everything we do with ImproveOSM.
See also Wille’s post about this (in Portuguese)
We launched ImproveOSM about 6 months ago as a way to turn the vast amounts of GPS data that Scout users give us into useful and actionable hints mappers can use to add turn restrictions, missing roads as well as wrong or missing one-way streets. The response has been incredible – since we launched, more than 26 thousand hints have been processed, leading to more than 16 thousand improvements to the map worldwide. I think that is a fantastic result, and we will keep working to make ImproveOSM better based on your feedback.
Initially, we just used our own GPS data to generate the hints. But there is no reason why we couldn’t process any GPS data we can get from other sources. So I was really excited when long time Brazil mapper Wille Marcel got in touch with a cool idea. He worked with the Brazilian Environment Ministry, which collects GPS data of the vehicles that work in environmental monitoring. Most of the data are in rural areas where OSM is much less complete. So this was a perfect fit for ImproveOSM’s missing roads tool.
After getting the proper permissions from the agency, Wille sent us the GPS data and we started analyzing it.
This post also appears on the ImproveOSM blog
United States cities are built for cars, with very few exceptions. From where I am sitting right now, I see this:
Cars zooming by incessantly at 70kph.
Finding your way in an urban space that is designed this way is tricky - and often dangerous - if you are walking or bicycling. Sidewalks are often not present, crossing streets can be very dangerous or even impossible. OSM has great tagging for bike lanes and sidewalks, but I find that these crucial tags are often missing on ways that need them most: the four or six lane urban arterials that you see in the picture above.
As I was sitting here asking myself how on earth I would get back to my hotel (which is 10 minutes away) safely, I thought to myself: ‘we can fix this problem and make the world a bit safer for those who can’t or won’t drive.’
MapRoulette to the rescue!
It has been a few weeks since I wrote about the public beta release of Cygnus, the Telenav conflation engine for OSM data. Since then, I have since been approached by a few folks who wanted to take it for a spin. One of them is long time OSM contributor MikeN. He is preparing an import for Holt and Atchison counties in the U.S. state of Missouri. We worked together on scaling some technical hurdles. Here’s a report of what we (well mostly he) did.
Mike obtained the source data from Holt and Atchison counties from their official GIS:
I obtained an updated road network from their official GIS, extracted and translated the tags, and followed up with a review against current aerials as well as checking for connectivity, glomming like road segments, and simplifying geometry. The final goal is to obtain permission to import, go through the import process steps, and merge new data onto the existing OSM data.
The next step was to convert the data into the OSM PBF format that Cygnus requires. This is when Mike got in touch with me to work through some technical difficulties:
Since Cygnus required the PBF format, I used Osmosis to convert. This failed because the nodes did not “have a version attribute as OSM 0.6 are required to have”. I have learned from Martijn that OsmConvert works without a version attribute, and was able to verify this on my second county.
The next catch was that PBF doesn’t accept negative node numbers. The simple workaround is to just use a text editor to remove the minus sign from
id='-andref='-. This seems a bit dangerous - would that file upload if accidentally selected? If so, many low numbered objects would be corrupted around the world. Hopefully, the conversion from OSM to PBF can be moved to the Cygnus chain so that it can accept zipped OSM since most users will start with .OSM data.
I am happy to announce that Improve OSM, the suite of open source Telenav tools that help us fix OSM based on vast amounts of GPS data, is completely redesigned and expanded. We made the web site much easier to use. We combined the existing JOSM plugins into one new plugin. And as of today, we have an entirely new category of fixes: missing turn restrictions. I will talk about all of these changes in this diary entry. A lot to cover so let’s get started!
We redesigned the Improve OSM web site completely. Instead of separate maps for each type of error, all errors are now displayed on one map. You can turn each type on and off in the layer panel. You can also control the filters for each type there.
What’s even better (I think) is that you can now perform the entire fixing workflow from the web tool, including resolving the error as fixed or invalid. (You used to need the JOSM plugin to change the status on items.)
Missing Roads is doing great! Since we launched the plugin a little over two months ago, almost 14 thousand Missing Road tiles were closed.
A little over a quarter of those were marked as invalid. We have been looking carefully at those cases, and it turned out the two most occurring invalid tiles were ones involving foot / bike paths and water. So we created filters for those. This should help you use the Missing Roads plugin much more efficiently!
Here is the filter in action for water tiles in the web tool:
Update After talking to a lot of people about the future of MapRoulette and getting a lot of great feedback, I decided to abandon the Django effort I wrote about here. The New MapRoulette is coming together nicely! See its progress on Github.
I recently started looking into Django, the seminal Python web application framework. I like what I see much more that I thought I would, so much so that I am starting to rewrite MapRoulette as a Django application.
One thing I want to accomplish with a next version of MapRoulette is to make creating challenges much, much easier. For example, you should be able to define a challenge using an Overpass query. (You would be surprised how many annoying errors in OSM can be exposed with a pretty simple Overpass query! For example, here is a query that gives you all highway=tertiary that have no name in my home state Utah - 425 ways.)
So I needed a way for a Django application to access the Overpass API and display and store the results. That should be simple enough to accomplish using the Overpass API Python wrapper project I started a while ago. And it was!
I wrote about Cygnus, our effort to create an OSM-specific conflation engine, a few months ago. We developed it specifically to aid the import of INEGI road data in Mexico we are preparing together with the community in Mexico. But when used with care, I think it can be very useful in other import scenarios as well. This is why we decided to make it into a web tool anyone can use.
Before I go into details, I should offer a few words of caution.
Firstly, this tool is an early beta. Right now, it only conflates roads, nothing else. We have tested the tool internally, but only on a limited amount of cases, all using open INEGI data from Mexico. With this public beta, I hope to gather more feedback to help us make improvements to it.
You can be more than one kind of geek at the same time, I guess. First and foremost, I am a map geek. This started when I got my first atlas for my 6th birthday. Then came computers, with the arrival of the Commodore 64 I shared with my brothers. Shortly after that came trains. I still have a few thousand slides of stations, trains and railyards in my basement. Also, German and Dutch timetables dating back to 1985.
One of my later railway photos. Berlin Lehrter Stadtbahnhof, summer 1996. This was demolished not too long after to make room for Berlin HBf. Source: Flickr
I announced our second tool based on Scout GPS data, Traffic Flow Direction, a few days ago. I didn’t spend a lot of time on explaining how it works. This post will hopefully make up for that!
The goal of this plugin, and the accompanying web tool, is to make it easy to find and correct OSM ways that we think are missing a oneway tag, based on billions of GPS points from Scout users. Here is what it looks like in JOSM:
I will walk you through installation, basic operation and some mapping tips in the next paragraphs. Happy mapping!
This is a JOSM plugin so, installation works like any JOSM plugin. Make sure your JOSM is up to date first. Then go to JOSM preferences. Select the Plugins tab and look for the TrafficFlowDirection entry:
Last month, we released the Missing Roads plugin and web tool. This quickly became a popular pastime for quite a few mappers - one month later, more than 20% of all the Missing Road tiles have already been resolved :)
For the darker and colder November days – well, if you are on the Northern hemisphere at least – we thought we would cook up something new to keep us all busy. We ran another analysis on our GPS data to uncover ways that probably do not have the right directionality. Either they should be oneway=yes and they are not, or they are oneway=yes but in the wrong direction.
Here is an example from Karlsruhe, Germany:
The orange arrow points in the direction we think traffic on that street flows based on what we know from our Scout GPS data.
I have long wanted to try and do some audio mapping. Especially since I moved to the US and started spending more time in a car.
When you are driving, there is not a lot of ways you can record what you see in a way that makes it easy to map later. One way is to use Mapillary, but the sheer amount of information can be overwhelming. A picture every 5 seconds means 180 images to go through on a short, 15 minute drive. It also means handling over 300 MB of image data. And that’s only for a 15 minute drive.
So audio mapping. I have this tiny recorder that weighs almost nothing, has built in space for almost 70 hours (!) of recording and runs weeks on a set of AAA batteries:
A few weeks ago, I started something new for the Salt Lake City area OpenStreetMap group: Flash Map Mobs! The idea is to have a group of mappers descend on an under-mapped area and add lots of shops, restaurants etc. in a short amount of time. We do them after work, say from 5-6. Sometimes we get a drink or dinner afterwards.
We have done four so far and I wanted to share some results.
The first one! I decided to meet at Sugar House Coffee because I go there a lot. This place can get a bit crowded and loud however. This may turn people off so perhaps not perfect as a meeting place. The Mob itself (then still called ‘After Work Map&Meet’) was fun. We shared experiences and arrived at Pushpin OSM as the ideal iOS mobile editor for this kind of mapping.
Last week, we released the Missing Roads JOSM plugin, together with a web tool, the plugin source code, and a manual. More than 700 mappers have already given it a try, and nearly 10% of the missing roads tiles are already resolved. That is fantastic progress!
We have received a lot of great feedback from you and we are looking into every single suggestion. You may have noticed that we have already updated the web app adding a location hash. This means you can now share the URL at any time and it will include the current map view.
We also received a few requests to get access to the data powering the plugin and app. This sounded like a great idea to us! We started publishing daily dumps of the tile data. You can download and use these any way you see fit.
This is what the data looks like:
The most fun I have with OSM is making the map better. Adding nodes and ways and tags. Seeing how my puttering around with JOSM or iD leads to a prettier map that is more useful.
Getting folks together in Salt Lake City where I live to have some OSM fun together. For example. A few weeks ago, I started a new thing called the Flash Map Mob. We do them every other week now in SLC and together we map something like 100 shops and restaurants every time.
The map becomes more useful and folks are having fun. That is what OSM is to me!
I don’t need to sit on any board to do these things. Heck, I can probably do more of them when I am not on the board! So why do I do this? What value do I add being on the OSM US Chapter board?
I’ll tell you and I’ll be brief.
OSM is still very good at adding contributors.
(The graphs come from osmstats.)
What we’re not so good at is actually getting folks to go out and map. Look at the flat daily active mappers graph for the US:
Earlier this week, we released the Missing Roads project. Using its tools, every OSM mapper can now easily find roads that are not yet in OSM, based on our years of collected global GPS data from Scout users.
A lot of you have already had some fun with the JOSM plugin and the web tool. I received interesting reports about important roads you have been able to add. But also some questions about how to optimally make use of the Missing Roads tools. So I thought I might write up a Missing Roads Manual of sorts.
Missing Roads consists of two tools: a web tool and a JOSM plugin.
The web tool is a convenient way to locate missing roads in an area. You can quickly get a sense of the distribution of the missing roads data.
If you zoom in far enough, you can also see the individual tiles.
You will notice that there are different colors for both the traces and the tiles themselves.
New roads are built and opened for traffic around the world every single day. In many places, these are added to OSM by watchful mappers right away. Not everywhere though. There are still many places where there are few local mappers, and new construction goes unseen for a while. Available aerial imagery can be pretty outdated, so armchair mappers are not always able to help out either.
Lots of people drive on these new roads from day 1. And we’re in luck - a bunch of them are usually Scout users :) This means that they contribute GPS traces of the missing roads to us. Lots. Of. Traces. Enough for us to come up with a good guess about where new roads might be that are not on OSM yet.
We have done just that! I am excited to announce the Missing Roads project, opening up the aggregated, processed GPS data pointing at missing roads as a convenient JOSM plugin.
K1wi’s recent diary showed us how the use of destination is catching up with exit_to in the US for exit sign tagging:
I am excited about this! You may recall that I am very much in favor of destination on the _link way to tag exit signs rather than exit_to on the junction node.
I’ve come up with an idea to squash the remaining exit_to-tagged nodes in the US and move the exit sign info to a destination tag.
No scripting. No funny stuff.
A MapRoulette Challenge!
This is what it would look like:
Looking for new ways to get people out to map, I started a new initiative in our local Salt Lake City OpenStreetMap group: The Flash Map Mob. Inspired by the Flash Mob, the idea is to descend on a local commercial area, spread out and map all the businesses in an hour or less. We tyically meet at a coffee shop, divide up the area, and go out and map! Most people use their smart phones or tablets with apps like Pushpin, Vespucci or OSMAnd. This way, everything you map gets added to OSM right away and there is no work to be done after. But you could also use pen and paper, or GPS + camera.
Here we are checking out Pushpin on an iPad:
Here are the results of the first Flash Map Mob we did a few weeks ago, where we mapped 80+ businesses with three people: