OpenStreetMap logo OpenStreetMap

CjMalone's Diary

Recent diary entries

Not everybody cares, but we do. We care a lot.

Posted by CjMalone on 18 November 2023 in English. Last updated on 21 November 2023.

We care about having the best data. We care about having best the map. We care about having the best mapping community.

“We” is doing a lot here. Honestly, I don’t care about the render. Sure I’ve wanted some things rendered before, sure I’ve been emotional when it didn’t magically happen. But I’ve only though about rendering for a few minutes. People care a lot more then me, in retrospect, I’m sorry. I should have trusted you.

Honestly, I don’t care about the intricacies of tagging/”schemas”. Sure I’ve gotten emotional on the mailing list, and I’ve even talked about “you” (the list) behind your back. But you’ve thought about this more than me, I should have trusted you. Sorry.

I have thought a lot about the quality of our data, it’s not perfect. About the usability of our data, there are issues. About the accessibility of our data, it could easier. But actually we are doing pretty good. We are even doing better than the commercial datasets.

So trust the people that care the most, support us, encourage us, give us data and tooling, but most importantly, don’t get in our way. We’ve got this.

POI Dataset

Posted by CjMalone on 23 August 2021 in English.

I’ve been working on collecting datasets for various chains across the UK, and the globe. Yesterday I changed the repo to open, so say hello to KAFF Places. It’s mainly got retail chains, but it also some hotel chains, pharmacies and the like. You can see the list here.

This data is first party from their respective websites, interestingly enough there are a lot more quality issues that come with that than I expected. I have reached out to some of them to point out issues, and I’d love to work with all of them to improve their data.

I’m waving any database rights I may have, but because this is first party data each provider holds copyright and/or database rights. I currently do have some rules on which sites I collect data from, the most important rule is that they are publishing the data in Linked Data or Microdata. I personally believe once somebody explicitly publishes data in a machine readable format they want the data to be used. However the law may disagree. This shouldn’t be used as a source for OSM.

Some of this data is already being used by Robert Whittaker in Chain Reaction to help monitor the quality of OSM coverage. I’d also like to see this data being used as an overlay on top of OSM data to increase coverage in under mapped areas, in projects like OsmAnd and Daylight Map Distribution.

Mapping allotments

Posted by CjMalone on 25 November 2020 in English.

I’ve taken on the task of mapping allotments throughout Great Britain. I think there is a similarity between allotment users and OSM contributors, even if it’s not immediately obvious. Allotments are quite often community owned or some form of cooperative, but even the commercially owned ones have roots back to the Diggers when private land was reclaimed for public good and food production.

I’ve been thinking more about the overlap between Open Data, Free Software and the common land lately. I’m starting to think it’s all the same thing, a rebellion against private property and ownership. It’s about common good, and helping your neighbour. It’s about the future and your legacy. It’s about leaving the world slightly better than you found it.

I’ve created a MapRoulette Challange based on a subset of OS Open Greenspace. For some I can add little more than a rectangle with landuse=allotments, some I can see a few more details like paths and tracks. 1 2 3

Quarterly Project with StreetComplete

Posted by CjMalone on 18 August 2020 in English.

One of, if not the, easiest way to contribute to the current Quarterly Project is through StreetComplete. For anyone not aware StreetComplete is an editor on Android, it’s designed to be incredibly simple and help introduce new users to contributing to OSM. It’s how I started regularly contributing.

Some of it’s questions (quests) displayed to users are directly related to the current Quarterly Project like asking if a given road has a cycle path, and categorising types of bicycle parking. So if you were exclusively interested in the Quarterly Project you could disable the other quests and focus on the ones above.

However some of the other quests indirectly help OSM cycling data, such as road surface, whether the way is lit, names, one way, speed limits, crossings and pavements. They are all incredibly useful to downstream users of OSM cycling data. I think I recommend starting with all of the quests but disabling some high traffic ones, such as building types, you can, and should, come back to them, but if the amount of quests is overwhelming there is no harm hiding them until later.

Freedom of Information requests

Posted by CjMalone on 14 August 2020 in English.

I’ve started sending some Freedom of Information requests! This is a new world for me, but seems to be going good and has helped a lot.

The first one I sent was for public phone boxes after BT refused to give me a list. It was successful, and I’m now surveying the locations, as well as the ones already in OSM. The two lists differ quite a lot, and interestingly neither is accurate. It makes me wonder if BTs own list is accurate.

So I’ve been taking a lot of photos of phone boxes for Mapillary and that makes me feel like a trainspotter. I promise I’ve not go a weird thing for phone boxes, it’s just an area of OSM that needed some love. Maybe I’ve got a weird thing for OSM…

My next request was for defibs, they still haven’t responded, but I can’t imagine a reason why they wont provide them. Once they do I’ll start surveying and documenting them. I’ve been adding local defibs to OSM for over a year now so it will be interesting to see which ones I’ve missed.

I think anyone else in the UK should consider using Freedom of Information requests, WhatDoTheyKnow is a great tool and makes it super easy. It probably goes without saying but anything you receive from FOIs is not compatible with OSM, you’ll need to get another source (go out and survey).

Where's the nearest payphone?

Posted by CjMalone on 27 July 2020 in English.

I’ve recently decided to survey the payphone/phone boxes/amenity=telephone in OSM. To my surprise, they don’t exist any more, well 12 out of the 17 surveyed don’t.

It seems BT is in wave of removing them, publicly they are just “restructuring to save costs”, but in reality it seems they are just removing all phone boxes.

They don’t publish locations of payphones but if you contact them they have to give you a list of any in consultation to be removed, or any recent consultations. They’ll give you a PDF with a postcode and a vague-ish location description. You’ll then need to go out and survey them, if they’re gone, delete them, if they still exist add some metadata tags, ensure its operator tag is correct, add it’s incoming phone number.

I’m going to try and survey the 65 remaining payphones in OSM in my county, based on my last survey, it might only be 20 that actually exist! (75%)

StreetComplete may be helping in this area soon, Tobias Zwick is working on map maintenance through the app so I’ll defiantly be recommending this and bus stops to the list of verification quests.

OpenStreetMap and Linux are pretty similar in a lot of ways. The licences are both copyleft, require attribution, and allow modification and re distribution. Both projects have a foundation to look over them, but are relatively hands of on day to day operations. Both foundations have corporate sponsorship to fund them.

However when it comes to edits, the projects differ. Linux is now hugely dominated by paid developers, OSM is almost exclusively volunteers. One reason for this may be because most downstream users of OSM use it exclusively as a base map, they use there POI data on top and then almost laughably link to Google Maps for directions. They are only using a tiny portion of OSM. (And they are missing out!)

Lots of Linux contributors are paid employees of a business, usually tasked with fixing or implements a specific thing. As far as I know only Amazon does this in OSM, paying employees to fix routing issues. Sometimes this is closer to having a benefactor, where the business pays the employee to work on what ever they want and the benefactor trusts it’s in the best interest over all. Mark Shuttleworth ran Ubuntu as a loss for a long time, they contributed to the whole ecosystem during that time. I guess Mapbox might have the finances to be able to be a benefactor, but other than them there doesn’t seem to be a lot of money around.

Some developers are founded directly through other people on platforms like Patreon. I don’t think this is a massive in the kernel space, although a few high profile developers for other projects do it. You would then get the freedom to spend your time how you wanted, working on projects that interest you. This is similar to having a benefactor although you have to spend a significant amount of time and effort marketing yourself, and being your own PR. I don’t know if OSM has a big enough crowd to support this.

See full entry

Asda stores county wide

Posted by CjMalone on 29 April 2020 in English.

As some people have noticed last week I edit lots of Asda stores across the UK. I have been adding branch specific contact:website as well as brand data (brand=Asda brand:wikidata=Q297410 brand:wikipedia=en:Asda). The stores that didn’t already have address data, now should have data provided by FHRS Open Data.

Across the way I’ve made some mistakes, when I started I was also adding brand social media (contact:facebook=https://www.facebook.com/asda/ contact:twitter=@asda). trigpoint pointed out this is wrong because some (most?) Asda branches have there own social media accounts that would be more accurate, and user agents can still get the brand accounts via wikidata.

Throughout this project I have been prioritising adding the the specific webpage rather than a generic homepage. This has multiple benefits.

  • The most obvious being users get precise data when they click the link.
  • Another benefit is that the Asda website supports microdata, to publish metadata.
    • User agents can use this data to get more up to date or missing data than OSM provides, even though relatively few Asda stores have contact:phone that data is now available and can be used. The same goes for opening_hours, addresses and even the GeoCoordinates.
    • Mappers can use it similarly to improve OSM.
    • Data scientists can now easily compare OSM data to Asda data.

While doing this I was careful not to use data from the Asda website, due to copyright concerns. So I ignored opening_hours and contact:phone, this data is valuable and I would love for it to be in OSM but I didn’t want to compromise OSM. For addresses I used another source, FHRS Open Data, it is an amazing data set and in my opinion every OSM mapper in the UK should know about it. Using it in OSM has some awesome benefits.

See full entry

Here in the UK, and probably most of the world, schools are closing, which means parents and students are looking for contact information for the schools. If any of them look for that data through OSM.org or downstream clients it is really useful to have that data up to date. So if anyone has a bit of time to kill, please help.

Robert Whittaker generates reports about schools, it’s a great tool to use. I don’t have time to do a proper write up, my work hasn’t closed, and is incredibly busy.

But vaguely,

  • Pick an area
  • Go to the table of schools matched by location
  • The first link is to edubase/Get Information About Schools.
  • Copy the contact details to OSM

It would be nice to also add the id as ref:edubase and the address, but I think the contact details should be prioritised at the moment. Other contact links like Facebook and Twitter would also be great but take longer to find as they aren’t part of Edubase.

I’ve done the PO and SO postcodes so far, I’ll be doing more as I can.

Bus stops in the UK

Posted by CjMalone on 23 January 2020 in English.

As with everything in OSM, quality varies massively by location. Around my area, and lots of the UK, the state of bus stops has been neglected. Around a decade ago there was the NaPTAN import, with a few nit picks I believe it was a good thing. OSM needed bus stops, NaPTAN had bus stops, the answer is obvious, import.

However we have let that data go stale and get out dated. A surprising amount of the bus stops, haven’t been touched in the 10 years+- since the import. They still have naptan:verified=no. They have old names. Take a look at osm.org/user/NaPTAN/history, pick a changeset, pick a node. Was it last edited about 10 years ago by NaPTAN? That’s an issue. Maybe somebody realigned it based on satellite imagery, that’s still an issue.

The NaPTAN import was a static thing. A one time event. But the NaPTAN data isn’t, they record and publish when a bus stop is changed, it’s name changed, it gets deleted, moved. Why can’t we keep on top of this data? I know OSM is very against automatic edits, for one reason or another. So I’ll just flat out ignore that, even though I disagree.

So we should be manually keeping in sync with NaPTAN, one way to do this is to download the NaPTAN data for a region and sort it by newest modified date. Compare it to the OSM data, does it have a new name? Has it location changed? Has it plain been removed due to route changes? Do we need re survey the bus stop? To download NaPTAN see that page.

Please, everyone who reads this, do an overpass query in your local area, the bus stops need your help.

Not verified:

node["highway"~"bus_stop"]["naptan:verified"="no"];

No NaPTAN ref:

nwr["highway"~"bus_stop"][!"naptan:AtcoCode"];
nwr["public_transport"="platform"]["bus"="yes"][!"naptan:AtcoCode"];

See full entry