OpenStreetMap logo OpenStreetMap

pnorman's Diary

Recent diary entries

OSMF Board candidates: Joseph Reeves

Posted by pnorman on 28 November 2015 in English.

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I’ve added annotations in italics where I felt they would be useful.

Joseph Reeves * Location: UK * OSM User IknowJoseph * Manifesto

Questions

Where do you currently participate in the OSMF?

I admit that I have not contributed to any OSMF Working Groups. I greatly appreciate the work they do, however, and would only work to encourage participation in them. I am happy to join any WG that I can be an asset in. I have done work to promote OSM outside of more formalised methods and would continue to encourage others to do the same

Which contributions to OSM should I consider for my decision beyond your data edits at OSM?

Do you use OSM at work for business purposes?

First of all, please don’t judge my, or anyone else’s, suitability to serve on the Board by their map edits! I’d argue that a previous preoccupation with mapping skills has been to the detriment of the OSMF and has contributed to a cultural problem that is inhibiting diversity and community involvement. In my opinion crucial Board skills involve communication, planning and organisational skills. Most relevant to my OSMF Board application, I have served on the HOT Board. I have been a long term contributor to HOT as a Member and have participated in a number of Working Groups and activities, including three trips to Indonesia. An example of my writing is available on the HOT Blog. I have spoken about OSM at numerous events including an Oxford University seminar series that can be watched online here, and TechCamp Sarajevo 2014.

Conflicts of Interest

See full entry

OSMF Board candidates: Mikel Maron

Posted by pnorman on 28 November 2015 in English. Last updated on 2 December 2015.

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I’ve added annotations in italics where I felt they would be useful.

Mikel Maron

Questions

Where do you currently participate in the OSMF?

In the past, I have served on almost every working group, except Operations. Currently, I am on the SotM WG, and anticipate joining the LCWG again. Most every working group needs more help, and I will promote and recruit heavily for more participation from OSMF members and the broader community

Which contributions to OSM should I consider for my decision beyond your data edits at OSM?

  • I mapped Brighton, UK
  • Facilitated mapping in places from the West Bank, to Nigeria, to Swaziland
  • I co-founded Map Kibera
  • I co-founded Huanitarian OpenStreetMap Team
  • I’m on https://github.com/mikelmaron. I’ve contributed to OSM website, got the first version of OSM on Garmin and the iPhone, worked on the OSM Tasking Manager. Most recently, I’ve been working on analysis and metrics of OSM.
  • I’ve written numerous tutorials, spoke at tons of conferences, and represented OSM projects in venues from community meetings in slums, to international political gatherings. Important point … editing and programming are critical contributions to OSM, but there are other skills we need on the Board.

Do you use OSM at work for business purposes?

I work at Mapbox on the data team. My focus is on leveraging work here for the widest possible benefit to OSM. I’ve previously worked at the State Department, as a Presidential Innovation Fellow, on MapGive. Prior to that, I consulted on OSM related projects through GroundTruth Initiative.

Conflicts of Interest

See full entry

OSMF Board candidates: Guido Stein

Posted by pnorman on 28 November 2015 in English.

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I’ve added annotations in italics where I felt they would be useful.

Guido Stein

  • Location: USA
  • OSM User gsteinmon
  • Manifesto quoted below

As someone new to the community I hope that I can share my prespective as a newer member in your efforts to grow the community as well as in your efforts to have better outreach with newer members. I have years of experience running and founding local community groups as well as technical experience with geospatial technologies. I am stepping up for your consideration in order to help where I can with the community growth and development.

Questions

Where do you currently participate in the OSMF?

I am still very new to this community and have not started working on any working group, but plan to join some working groups within the next couple of months.

Which contributions to OSM should I consider for my decision beyond your data edits at OSM?

As a newer member I have contributed by helping to grow the Boston GeoSpatial community and supporting local mapping events with the OSM Boston Meetup Group. I use my own community group, AvidGeo, to help build awareness and attendance of OSM Mapping parties locally. I will be working with the 2017 FOSS4G Conference to also help bring attention to the OSM community.

Do you use OSM at work for business purposes?

I am a GIS Analyst and use OSM data as a base layer in some of my work, but my work does no currently include any OSM editing activities.

Conflicts of Interest

No, I do not have an conflicts of interest. I work for Applied Geographics, Inc. and am the founder/lead organizer of AvidGeo. I am also on the organizing committee of the 2017 FOSS4G. None of these positions have a fiscal or other overlap with the duties of the OSM board.

Who should the OSMF serve?

Has not responded.

Role of the board

See full entry

OSMF Board candidates: Peter Barth

Posted by pnorman on 28 November 2015 in English.

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I’ve added annotations in italics where I felt they would be useful.

Peter Barth

Questions

Where do you currently participate in the OSMF?

I joined the Data Working Group about half a year ago, and have been helping to resolve disputes between mappers since.

Which contributions to OSM should I consider for my decision beyond your data edits at OSM?

Besides my work as a mapper and Data Working Group member, I have: * created and contributed to Open Source software related to OpenStreetMap, especially related to 3D rendering * mentored students doing OSM work during Google Summer of Code projects and theses * cooperated with government, industry and charity organizations on a local level * given speeches at various conferences, and contributed content to WeeklyOSM and the German Wochennotiz * introduced new users to OpenStreetMap through events, talks, and tutoring.

I believe this gives me a broad perspective on various aspects of the OSM project an community, and I hope to contribute my experience as part of the OSM board.

Do you use OSM at work for business purposes?

No. While I try to promote OSM at my university through bachelor and master theses, I actually have no work related affiliation to OSM.

Conflicts of Interest

No. See above.

Who should the OSMF serve?

Serving the needs of map users is an integral part of the usefulness of our project, but to me, the OSM community consists of mappers and software developers. And these groups are who I’d try to represent.

Role of the board

See full entry

OSMF Board candidates: Ilya Zverev

Posted by pnorman on 28 November 2015 in English.

In preparation for the 2015 OSMF board election I am gathering basic info and question responses by candidate, to help people be better informed about their choices.

I’ve added annotations in italics where I felt they would be useful.

Ilya Zverev

Questions

Where do you currently participate in the OSMF?

First, most work is done outside OSMF, even by OSMF members. People write blogs by themselves, explain legal issues without being on LWG, and code without EWG help. There are 5 groups currently active: OWG and DWG, because they have the only power there is in this community: access to servers and a banhammer. And LWG, LCWG and SotMWG, which do organizational stuff few people have knowledge or experience or time to do.

I am not currently on any working groups, though formerly I’ve been on EWG (promoting API changes) and on the Membership WG, which didn’t set off. Which does not mean, obviously, that I’m not doing any work WGs do: for instance, I do communications work in the Russian community, editing a news blog and hosting a weekly OSM podcast. During my involvement in OSM, I tried most things WGs do: explained licensing, wrote code, admined servers, tried to reason with vandals, organized conferences, considered opening a russian local chapter.

Why I don’t sign up to any of the groups? Some, I don’t feel qualified enough (e.g. LWG or LCWG), others are too taxing (I envy these steel-nerved heroes at DWG) or require 24h commitment (OWG, we can’t thank you enough). And some working groups don’t seem useful: why join CWG or EWG, when all the help you need is in IRC channels and blogs?

See full entry

OSMF Board election canidate positions

Posted by pnorman on 28 November 2015 in English.

The OSMF is having a board election, with voting opening this weekend. The questions to candidates are arranged by question, but I wanted to see the responses of each candidate together, so I rearranged them, and sorted the questions into an order I think makes some sense.

Because this is going to end up being long with 11 candidates, I’m splitting into multiple diary posts. I’ll link them all together after publishing, but I can’t do that yet.

Candidates

Guido, Ryan, Gonzalo, and Douglas have not responded to most of the questions.

Mikel and Joseph have not responded to some of the questions.

Questions

The questions are reproduced below.

Where do you currently participate in the OSMF?

Most OSMF participation and work is done through working groups. What working groups are you on, and if you’re currently not participating in any, why not?

Which contributions to OSM should I consider for my decision beyond your data edits at OSM?

Please give me a short list of your contributions to OSM (both editing and programming). Do you have edit via multiple OSM accounts which are not listed at Foundation/AGM15/Election_to_Board? Which OSM related software and websites do/did you develope? Please give me a short list and/or a link to your Github profile.

Do you use OSM at work for business purposes?

Do you use OSM at work for business purposes? Do you earn money by developing OSM-related software/websites? Do you earn money by editing at OSM (aka paid mapping)?

Conflicts of Interest

Do you work for a company or are involved with organizations that are working with OSM in some form? If yes how do you plan to handle situations where the interests of this company/organization and the interests of those you would represent on the board diverge?

Who should the OSMF serve?

See full entry

OpenStreetMap Carto Complexity

Posted by pnorman on 16 November 2015 in English.

This is a repost from my blog

I often refer to OpenStreetMap Carto as the largest most complex open multi-contributor map style, but what does that mean?

Broken down, it means

  • It’s the largest open stylesheet. If you measure in code size, features rendered, or complexity, nothing else is close;

  • It’s the largest multi-contributor map style that doesn’t have a company dictating what is worked on. This means we get merge conflicts. They got so bad we changed the technology we use to define layers to make them solvable; and

  • It’s the largest style using OpenStreetMap data. Some proprietary styles like OpenCycleMap, MapQuest Open, and Mapbox Streets are complex, but none of them render the range of features we do.

This complexity didn’t come about out of nowhere. It’s been building since contributions shot up in October 2014. This is when we introduced YAML layer definitions, making the style much easier to edit and streamlined the feature merge process.

See full entry

Repost of https://lists.openstreetmap.org/pipermail/osmf-talk/2015-November/003452.html

This is the announcement of this year’s Annual General Meeting. The official announcement has been sent to member’s emails already, but more people may notice this.

The PDF attached to those emails had the wrong month for cut-off for nomination and start of voting. This PDF has corrected information

The 9th Annual General Meeting of the OpenStreetMap Foundation will be held online in the IRC chat room #osmf-gm on the IRC network irc.oftc.net, at 1600 UTC on Saturday, 05 December 2015. You do not need to attend the meeting to vote.

The agenda is included in the the PDF and can also be viewed here: http://wiki.osmfoundation.org/wiki/Annual_General_Meetings/15

Information about email voting will be sent in due course, and voting will open one week before the AGM.

I will be sending out additional information on standing for the board election.Board nominations close two weeks before the AGM. Someone else or myself will send information on the special resolution.

Only regular members can vote on the special resolution or be on the board. If you wish to change your membership status from associate member to regular member, please don’t wait until the last minute.

On behalf of the Board, I am looking forward to our Annual General Meeting

More OpenStreetMap Futures

Posted by pnorman on 31 October 2015 in English.

Repost from my blog

Andy recently blogged the developer numbers from his OpenStreetMap Futures talk at SOTM US.

Wanting to play with the numbers myself, I took the osm100 code and added in additional projects. The original list of repos came from a list of “Core Software” from the Engineering Working Group, and since then some of the software has been replaced, and there’s other older software which used to be core, but isn’t.

The big changes were

  • JOSM plugins

    These serve as an easier entry into JOSM programming, and have far more committers than JOSM core

  • Taginfo

    Taginfo is now a core part of OSM, being used for auto-complete in iD

  • Osmium and Libosmium

    Osmium and Libosmium run behind the scenes for a lot of OSM software. This probably didn’t add many more people, as most developers wouldn’t start here.

  • Old Mapnik Stylesheets

    These had a lot of cartography work until 2009, and are important for historical data. The contributors are likely to be different too.

  • OpenStreetMap Carto

    OpenStreetMap Carto is a large, active, multi-contributor project, and with many unique contributors.

See full entry

New osm2pgsql long-term release

Posted by pnorman on 18 July 2015 in English.

A new long-term version of osm2pgsql has been released, 0.88.0. This includes the work done in the 0.87.x development series and the porting to C++.

Like all versions, 0.88.0 can be obtained from https://github.com/openstreetmap/osm2pgsql

Potentially breaking changes

Major new features

  • A new backend has been added, the “multi” backend. This allows multiple tables which can each contain different types of features. More documentation is available at https://github.com/openstreetmap/osm2pgsql/blob/master/docs/multi.md.

    One potential use for this feature is matching an existing schema for data, to allow OSM data to be a drop-in replacement.

  • In-memory pending tracking instead of in-database, with significant performance gains.

  • Rendering tables are ordered by GeoHash when created, resulting in significant performance improvements.

  • z_logic has been improved, taking into account more recent work across multiple styles. https://github.com/openstreetmap/osm2pgsql/pull/374 has more information.

Other changes

  • The node storage has been improved, and out of order nodes and nodes at 0,0 should now always be handled correctly

  • A new test suite with unit tests

  • Many bug-fixes and cleanups

Known issues

  • Append mode is not supported with non-slim. This is suported in 0.89.0-dev, but is frequently a user error. Typically, when a user does this they should instead merge input files with osmosis/osmconvert and use –create

What’s next?

See full entry

We are preparing a new osm2pgsql stable series release, 0.88.0. This is based off of the work done in 0.87 development series with porting to C++.

Help is needed in doing a last round of tests before release, in particular the z_order logic. 0.88.0-RC1 can be obtained from https://github.com/openstreetmap/osm2pgsql

If upgrading from a 0.86.0 or earlier created database, the schema migrations in https://github.com/openstreetmap/osm2pgsql/blob/master/docs/migrations.md are required.

Major changes

Osm2pgsql is now C++ and requires the Boost libraries.

A new backend has been added, the “multi” backend. This allows multiple tables which can each contain different types of features. More documentation is available at https://github.com/openstreetmap/osm2pgsql/blob/master/docs/multi.md

In-database pending way tracking has been replaced with in-memory tracking, offering significant performance gains. This requires a schema migration for old databases.

z_order logic has been improved, taking into account recent work across multiple styles. https://github.com/openstreetmap/osm2pgsql/pull/374 has more information. This requires a schema migration for old databases.

Osm2pgsql 0.87.4 release

Posted by pnorman on 11 July 2015 in English.

old engineering plan storage

Osm2pgsql 0.87.4 has been released. This development release is focused on improving the node cache and pending way status storage.

Lockfree Queue removal

The boost::lockfree::queue implementation of the pending way queue has been removed, leaving the std::stack based implementation which used to be available with the --without-lockfree configuration flag. The stack implementation was found to use substantially less RAM. This should allow the --cache value to be increased and drastically speed up import speeds, particularly with full planet imports and machines with 16-32GB of RAM and mechanical hard drives.

The boost::lockfree::queue implementation used to require Boost 1.53 or later, and this difference has been removed.

Any package maintainers packaging the 0.87.x series should move to 0.87.4 and remove any usage of --without-lockfree from their build scripts

Node cache cleanups

See full entry

Osm2pgsql 0.87.3 release

Posted by pnorman on 30 April 2015 in English.

Osm2pgsql 0.87.3 has been released. This development release primarily fixes bugs, but some of the bug fixes make other features usable.

Included is a bug fix for the lockfree queue implementation. Anyone using versions 0.87.0 to 0.87.3-dev, parallel processing, Boost 1.53 or newer, and not using –without-lockfree should immediately upgrade or stop using parallel processing. No data corruption issues have been observed, but the lockfree implementation may have been buggy on all systems.

There have been various fixes with moving hand-written C structures to C++ standard library equivalents and other code cleanups. The main user-facing changes are

  • The multi-backend should now be functional, with an example which creates separate tables for bus nodes, highways, and buildings

  • –without-lockfree is no longer needed on OS X, BSD and some Linux distributions and architectures. This should simplify downstream build scripts for multi-architecture builds and improve speed on any OS that required the option before.

  • nodecachereader should now work with node IDs > 2^31. This is a separate utility program, and obviously isn’t used much

  • Nominatim-related performance improvements

  • Many autoconf macros have been updated. This should ease configuration on non-standard systems.

This may be the last tagged release that does not require C++11. We have no current PRs which will require C++11, but would be willing to accept them.

A full list of commits is at https://github.com/openstreetmap/osm2pgsql/compare/0.87.2…0.87.3

As always, bugs can be raised at https://github.com/openstreetmap/osm2pgsql/issues. I’m particularly interested if package maintainers have concerns. If osm2pgsql isn’t packaged for your OS and you want to do so and have questions about the osm2pgsql side, please ask them too.

Many thanks to those who have contributed code to this and previous releases.

Paul Norman
On behalf of the osm2pgsql maintainers

Building Osm2pgsql for Testing

Posted by pnorman on 9 January 2015 in English.

This is a copy of a blog post on my site.

Recently I needed to run a bunch of osm2pgsql tests in a virtual machine, so optimized the process. You would not normally do development in a VM, but it’s useful for testing. It’s particularly useful for me because my old Ubuntu server uses a development version of PostGIS which doesn’t work with the testsuite.

I was using VirtualBox to run the VMs, and started with a base Ubuntu 14.04 server install on a virtual drive of at least 30GB. The flat-nodes tests require a lot of space.

Starting with the base, I updated the OS with

sudo apt-get update && sudo apt-get dist-upgrade && sudo shutdown -r now

This gave me a VM I could clone and use for other projects. Starting from a cloned VM, I installed the dependencies. Copying and pasting into VirtualBox can be a pain, so a bit of clever shell expansion minimizes the text I need to type

sudo apt-get install libtool g++ protobuf-c-compiler postgresql-9.3-postgis-2.1 \
  lib{boost{,-system,-filesystem,-thread},xml2,geos++,pq,bz2,proj,protobuf-c0,lua5.2}-dev \
  python-psycopg2

The osm2pgsql testsuite requires a custom tablespace for some of the tests, as well as normal PostgreSQL setup.

sudo -u postgres createuser -s $USER
sudo mkdir -p /tmp/psql-tablespace
sudo chown postgres.postgres /tmp/psql-tablespace
psql -c "CREATE TABLESPACE tablespacetest LOCATION '/tmp/psql-tablespace'" -d postgres

To get osm2pgsql, I was cloning my own git repo instead of the normal one, but if you wanted to get the source, build the latest version, and run the testsuite, I did it with

git clone https://github.com/openstreetmap/osm2pgsql.git
cd osm2pgsql
./autogen.sh
./configure
make –j4
make check

make check tends to take awhile to run, as it has to do several imports and create a lot of databases for testing. It also has to write a 20GB flat nodes file.

Langley Imagery

Posted by pnorman on 3 November 2014 in English.

Note: Repost of http://paulnorman.ca/blog/2014/11/langley-imagery/

I just got the recent Langley 2014 imagery from their Open Data program loaded onto my server, and I’m impressed. The new imagery has at least as good spatial accuracy, while having better resolution, colours, and being more recent.

Comparison between old and new imagery

In the next few days, I want to release the new layers.

Unfortunately, while Langley is using the PDDL license, other cities in the region are using custom licenses, meaning I have to enquire with each one individually, taking significant time. If they were all using standard licenses, I would have rebuilt my BC Mosaic layer by now and this post would be about updating it to include new sources.

Location: Aldergrove Village, Aldergrove, Township of Langley, Metro Vancouver Regional District, British Columbia, V4W 2Z0, Canada

OpenStreetMap Carto v2.22.0

Posted by pnorman on 6 October 2014 in English.

Labels on Shelf

OpenStreetMap Carto v2.22.0 has been released. This release focuses on labels.

The biggest change is a rewrite of landcover labelling. A landcover label is text connected to a background colour or pattern rendering, and not connected to an icon. This has been demoed extensively, and well received. It was also sent to the mailing list. The big changes are making colours better connected to the background, rendering labels on some features where they weren’t before, and sizing labels based on area.

The last deserves a better explanation. Previously, the selection and choice of what labels to render didn’t include area. It now does, avoiding placing labels on features that are only a few pixels in area at low and middle zooms, and selecting font size based on feature area. This results in a much more sensible label placement, more readable labels, better selection of what labels to place, and in many cases, more labels without impairing readability.

See full entry

Central Park mapping party

Posted by pnorman on 4 June 2014 in English. Last updated on 5 June 2014.

I am planning on hosting a mapping event in Central Park (osm.org/way/23165846), in Burnaby, BC.

My tentative date is Saturday, June 14th at noon, but if people want a different time I could shift it.

The park is a major park in the region, but under-mapped, with potentially not all of the trails.

Some of the things I’d like to get mapped are

  • More appropriate tags for the trails. highway=track is probably not accurate
  • The surface of trails
  • Locations of fitness equipment in the park
  • Bathrooms
  • Picnic areas
  • Other park infrastructure
  • Anything missing or interesting

At this time of year, it should be nice sunny weather, and the shade from the trees should be welcome.

I intend to bring my mapping kit (camera, GPS, etc), as well as field papers type printouts on larger paper for us to mark up. I’m not planning on bringing a laptop, or if I do, it’s staying in the trunk.

After mapping, we can go somewhere nearby for food. I’d also like to discuss holding regular events.

If you’re interested, please let me know so that I know other people will be coming and to attend on time myself! I also need to have printouts made in advance.

(Cross-post of https://lists.openstreetmap.org/pipermail/talk-ca/2014-June/006166.html)

Edit: Date change to 14th

Location: Garden Village, Burnaby, Metro Vancouver Regional District, British Columbia, V5G 3G5, Canada

Organizational mapping policy

Posted by pnorman on 14 May 2014 in English.

This is a cross-post of https://lists.openstreetmap.org/pipermail/talk/2014-May/069772.html


We have more and more organizations and businesses mapping in OSM. Multiple organizations have been conducting paid editing in Europe and the US. This generally comes to light after complaints are made - with the company usually not identifying who they are, what their goals are, and what they want, beforehand. There have also been difficulties determining what has been mapped on behalf of an organization.

We will likely see more of this type of editing in the future, and while not necessarily bad, there are differences between it and normal editing. Recent events in a project similar to OpenStreetMap - Wikipedia - have demonstrated that the participation of organizations in data editing can occasionally lead to misunderstandings or disharmony in the project, particularly where a lack of transparency is involved.

For this reason the DWG is considering if it is necessary to issue guidelines for organizational editing. Some previous discussion is at http://lists.osm.org/pipermail/osmf-talk/2013-November/002344.html

There are some activities we do not want to cover in the guidelines

  • Unorganized editing by employees, e.g. a shop owner adding their shop or nearby details to the map

  • Editors mapping in response to a contest or similar where the contest organizer does not have the power to require them to edit

  • Individuals who, on their own accord, decide to participate in an organised effort or challenge, like local mapping parties, Mapathons, HOT projects, etc

Some possible guideline requirements could involve

See full entry

State of the Map kickoff/morning notes

Posted by pnorman on 19 April 2014 in English.

On the schedule for the Friday and on Saturday morning was the kick off party, registration, let’s map!/OpenStreetMap in your organization sessions, and coffee break.

Kick off party

Thanks to a delayed flight I ended up going to the Mapbox Garage. The entrance is on the rear, so the taxi driver was confused getting there. Since I was there early, I helped with the setup, and schlepping the alcohol over from the store nearby.

The party got too loud, but I was able to catch up with a few people. We ended up running out of bottled water, and the taps were inconvenient. I walked back with a few others staying at the Washington Plaza hotel.

Saturday morning

Registration

Registration went smoothly. Conference t-shirt just said State of the Map, not State of the Map US, which seemed odd. I think most of the conference people handling registration were paid staff, which was a difference.

Welcome

There was a welcome talk which I thought was rather good at the time, but seems it wasn’t very memorable, because I can’t actually remember much of what was said!

The conference had over 500 people check in at registration, a big growth from the 2009 SOTM-US which had about 50 people.

First session, coffee break

I ended up talking with the people from Amazon for all of this session, so missed the talks. I’m hopeful that they’ll be able to commit some EC2 credits to do some dev work and performance testing. Of course, I was the same way after previous conferences and nothing came of it.

It doesn’t sound like they’re willing to commit to helping any osm.org infrastructure with resources in any ongoing manner.