Repost of https://lists.openstreetmap.org/pipermail/osmf-talk/2015-November/003452.html

This is the announcement of this year’s Annual General Meeting. The official announcement has been sent to member’s emails already, but more people may notice this.

The PDF attached to those emails had the wrong month for cut-off for nomination and start of voting. This PDF has corrected information

The 9th Annual General Meeting of the OpenStreetMap Foundation will be held online in the IRC chat room #osmf-gm on the IRC network irc.oftc.net, at 1600 UTC on Saturday, 05 December 2015. You do not need to attend the meeting to vote.

The agenda is included in the the PDF and can also be viewed here: http://wiki.osmfoundation.org/wiki/Annual_General_Meetings/15

Information about email voting will be sent in due course, and voting will open one week before the AGM.

I will be sending out additional information on standing for the board election.Board nominations close two weeks before the AGM. Someone else or myself will send information on the special resolution.

Only regular members can vote on the special resolution or be on the board. If you wish to change your membership status from associate member to regular member, please don’t wait until the last minute.

On behalf of the Board, I am looking forward to our Annual General Meeting

Repost from my blog

Andy recently blogged the developer numbers from his OpenStreetMap Futures talk at SOTM US.

Wanting to play with the numbers myself, I took the osm100 code and added in additional projects. The original list of repos came from a list of “Core Software” from the Engineering Working Group, and since then some of the software has been replaced, and there’s other older software which used to be core, but isn’t.

The big changes were

• JOSM plugins

  These serve as an easier entry into JOSM programming, and have far more committers than JOSM core

• Taginfo

  Taginfo is now a core part of OSM, being used for auto-complete in iD

• Osmium and Libosmium

  Osmium and Libosmium run behind the scenes for a lot of OSM software. This probably didn’t add many more people, as most developers wouldn’t start here.

• Old Mapnik Stylesheets

  These had a lot of cartography work until 2009, and are important for historical data. The contributors are likely to be different too.

• OpenStreetMap Carto

  OpenStreetMap Carto is a large, active, multi-contributor project, and with many unique contributors.

… See full entry

OpenStreetMap Carto 2.33.0 has been released. This release focuses on cartographic style improvements, but the release notes also include 2.32.0.

The biggest changes are

… See full entry

A new long-term version of osm2pgsql has been released, 0.88.0. This includes the work done in the 0.87.x development series and the porting to C++.

Like all versions, 0.88.0 can be obtained from https://github.com/openstreetmap/osm2pgsql

Potentially breaking changes

• A database reload is recommended. If not, the schema migrations in https://github.com/openstreetmap/osm2pgsql/blob/master/docs/migrations.md are required.

• Osm2pgsql is now C++ and requires the Boost libraries. Build scripts and package dependencies may need adjustment

Major new features

• A new backend has been added, the “multi” backend. This allows multiple tables which can each contain different types of features. More documentation is available at https://github.com/openstreetmap/osm2pgsql/blob/master/docs/multi.md.

  One potential use for this feature is matching an existing schema for data, to allow OSM data to be a drop-in replacement.

• In-memory pending tracking instead of in-database, with significant performance gains.

• Rendering tables are ordered by GeoHash when created, resulting in significant performance improvements.

• z_logic has been improved, taking into account more recent work across multiple styles. https://github.com/openstreetmap/osm2pgsql/pull/374 has more information.

Other changes

• The node storage has been improved, and out of order nodes and nodes at 0,0 should now always be handled correctly

• A new test suite with unit tests

• Many bug-fixes and cleanups

Known issues

• Append mode is not supported with non-slim. This is suported in 0.89.0-dev, but is frequently a user error. Typically, when a user does this they should instead merge input files with osmosis/osmconvert and use –create

What’s next?

… See full entry

We are preparing a new osm2pgsql stable series release, 0.88.0. This is based off of the work done in 0.87 development series with porting to C++.

Help is needed in doing a last round of tests before release, in particular the z_order logic. 0.88.0-RC1 can be obtained from https://github.com/openstreetmap/osm2pgsql

If upgrading from a 0.86.0 or earlier created database, the schema migrations in https://github.com/openstreetmap/osm2pgsql/blob/master/docs/migrations.md are required.

Major changes

Osm2pgsql is now C++ and requires the Boost libraries.

A new backend has been added, the “multi” backend. This allows multiple tables which can each contain different types of features. More documentation is available at https://github.com/openstreetmap/osm2pgsql/blob/master/docs/multi.md

In-database pending way tracking has been replaced with in-memory tracking, offering significant performance gains. This requires a schema migration for old databases.

z_order logic has been improved, taking into account recent work across multiple styles. https://github.com/openstreetmap/osm2pgsql/pull/374 has more information. This requires a schema migration for old databases.

Osm2pgsql 0.87.4 has been released. This development release is focused on improving the node cache and pending way status storage.

Lockfree Queue removal

The boost::lockfree::queue implementation of the pending way queue has been removed, leaving the std::stack based implementation which used to be available with the --without-lockfree configuration flag. The stack implementation was found to use substantially less RAM. This should allow the --cache value to be increased and drastically speed up import speeds, particularly with full planet imports and machines with 16-32GB of RAM and mechanical hard drives.

The boost::lockfree::queue implementation used to require Boost 1.53 or later, and this difference has been removed.

Any package maintainers packaging the 0.87.x series should move to 0.87.4 and remove any usage of --without-lockfree from their build scripts

Node cache cleanups

… See full entry

Osm2pgsql 0.87.3 has been released. This development release primarily fixes bugs, but some of the bug fixes make other features usable.

Included is a bug fix for the lockfree queue implementation. Anyone using versions 0.87.0 to 0.87.3-dev, parallel processing, Boost 1.53 or newer, and not using –without-lockfree should immediately upgrade or stop using parallel processing. No data corruption issues have been observed, but the lockfree implementation may have been buggy on all systems.

There have been various fixes with moving hand-written C structures to C++ standard library equivalents and other code cleanups. The main user-facing changes are

• The multi-backend should now be functional, with an example which creates separate tables for bus nodes, highways, and buildings

• –without-lockfree is no longer needed on OS X, BSD and some Linux distributions and architectures. This should simplify downstream build scripts for multi-architecture builds and improve speed on any OS that required the option before.

• nodecachereader should now work with node IDs > 2^31. This is a separate utility program, and obviously isn’t used much

• Nominatim-related performance improvements

• Many autoconf macros have been updated. This should ease configuration on non-standard systems.

This may be the last tagged release that does not require C++11. We have no current PRs which will require C++11, but would be willing to accept them.

A full list of commits is at https://github.com/openstreetmap/osm2pgsql/compare/0.87.2…0.87.3

As always, bugs can be raised at https://github.com/openstreetmap/osm2pgsql/issues. I’m particularly interested if package maintainers have concerns. If osm2pgsql isn’t packaged for your OS and you want to do so and have questions about the osm2pgsql side, please ask them too.

Many thanks to those who have contributed code to this and previous releases.

Paul Norman
On behalf of the osm2pgsql maintainers

This is a copy of a blog post on my site.

Recently I needed to run a bunch of osm2pgsql tests in a virtual machine, so optimized the process. You would not normally do development in a VM, but it’s useful for testing. It’s particularly useful for me because my old Ubuntu server uses a development version of PostGIS which doesn’t work with the testsuite.

I was using VirtualBox to run the VMs, and started with a base Ubuntu 14.04 server install on a virtual drive of at least 30GB. The flat-nodes tests require a lot of space.

Starting with the base, I updated the OS with

sudo apt-get update && sudo apt-get dist-upgrade && sudo shutdown -r now

This gave me a VM I could clone and use for other projects. Starting from a cloned VM, I installed the dependencies. Copying and pasting into VirtualBox can be a pain, so a bit of clever shell expansion minimizes the text I need to type

sudo apt-get install libtool g++ protobuf-c-compiler postgresql-9.3-postgis-2.1 \
  lib{boost{,-system,-filesystem,-thread},xml2,geos++,pq,bz2,proj,protobuf-c0,lua5.2}-dev \
  python-psycopg2

The osm2pgsql testsuite requires a custom tablespace for some of the tests, as well as normal PostgreSQL setup.

sudo -u postgres createuser -s $USER
sudo mkdir -p /tmp/psql-tablespace
sudo chown postgres.postgres /tmp/psql-tablespace
psql -c "CREATE TABLESPACE tablespacetest LOCATION '/tmp/psql-tablespace'" -d postgres

To get osm2pgsql, I was cloning my own git repo instead of the normal one, but if you wanted to get the source, build the latest version, and run the testsuite, I did it with

git clone https://github.com/openstreetmap/osm2pgsql.git
cd osm2pgsql
./autogen.sh
./configure
make –j4
make check

make check tends to take awhile to run, as it has to do several imports and create a lot of databases for testing. It also has to write a 20GB flat nodes file.

Note: Repost of http://paulnorman.ca/blog/2014/11/langley-imagery/

I just got the recent Langley 2014 imagery from their Open Data program loaded onto my server, and I’m impressed. The new imagery has at least as good spatial accuracy, while having better resolution, colours, and being more recent.

In the next few days, I want to release the new layers.

Unfortunately, while Langley is using the PDDL license, other cities in the region are using custom licenses, meaning I have to enquire with each one individually, taking significant time. If they were all using standard licenses, I would have rebuilt my BC Mosaic layer by now and this post would be about updating it to include new sources.

OpenStreetMap Carto v2.22.0 has been released. This release focuses on labels.

The biggest change is a rewrite of landcover labelling. A landcover label is text connected to a background colour or pattern rendering, and not connected to an icon. This has been demoed extensively, and well received. It was also sent to the mailing list. The big changes are making colours better connected to the background, rendering labels on some features where they weren’t before, and sizing labels based on area.

The last deserves a better explanation. Previously, the selection and choice of what labels to render didn’t include area. It now does, avoiding placing labels on features that are only a few pixels in area at low and middle zooms, and selecting font size based on feature area. This results in a much more sensible label placement, more readable labels, better selection of what labels to place, and in many cases, more labels without impairing readability.

… See full entry

I am planning on hosting a mapping event in Central Park (osm.org/way/23165846), in Burnaby, BC.

My tentative date is Saturday, June 14th at noon, but if people want a different time I could shift it.

The park is a major park in the region, but under-mapped, with potentially not all of the trails.

Some of the things I’d like to get mapped are

• More appropriate tags for the trails. highway=track is probably not accurate
• The surface of trails
• Locations of fitness equipment in the park
• Bathrooms
• Picnic areas
• Other park infrastructure
• Anything missing or interesting

At this time of year, it should be nice sunny weather, and the shade from the trees should be welcome.

I intend to bring my mapping kit (camera, GPS, etc), as well as field papers type printouts on larger paper for us to mark up. I’m not planning on bringing a laptop, or if I do, it’s staying in the trunk.

After mapping, we can go somewhere nearby for food. I’d also like to discuss holding regular events.

If you’re interested, please let me know so that I know other people will be coming and to attend on time myself! I also need to have printouts made in advance.

(Cross-post of https://lists.openstreetmap.org/pipermail/talk-ca/2014-June/006166.html)

Edit: Date change to 14th

This is a cross-post of https://lists.openstreetmap.org/pipermail/talk/2014-May/069772.html

---

We have more and more organizations and businesses mapping in OSM. Multiple organizations have been conducting paid editing in Europe and the US. This generally comes to light after complaints are made - with the company usually not identifying who they are, what their goals are, and what they want, beforehand. There have also been difficulties determining what has been mapped on behalf of an organization.

We will likely see more of this type of editing in the future, and while not necessarily bad, there are differences between it and normal editing. Recent events in a project similar to OpenStreetMap - Wikipedia - have demonstrated that the participation of organizations in data editing can occasionally lead to misunderstandings or disharmony in the project, particularly where a lack of transparency is involved.

For this reason the DWG is considering if it is necessary to issue guidelines for organizational editing. Some previous discussion is at http://lists.osm.org/pipermail/osmf-talk/2013-November/002344.html

There are some activities we do not want to cover in the guidelines

• Unorganized editing by employees, e.g. a shop owner adding their shop or nearby details to the map

• Editors mapping in response to a contest or similar where the contest organizer does not have the power to require them to edit

• Individuals who, on their own accord, decide to participate in an organised effort or challenge, like local mapping parties, Mapathons, HOT projects, etc

Some possible guideline requirements could involve

… See full entry

On the schedule for the Friday and on Saturday morning was the kick off party, registration, let’s map!/OpenStreetMap in your organization sessions, and coffee break.

Kick off party

Thanks to a delayed flight I ended up going to the Mapbox Garage. The entrance is on the rear, so the taxi driver was confused getting there. Since I was there early, I helped with the setup, and schlepping the alcohol over from the store nearby.

The party got too loud, but I was able to catch up with a few people. We ended up running out of bottled water, and the taps were inconvenient. I walked back with a few others staying at the Washington Plaza hotel.

Saturday morning

Registration

Registration went smoothly. Conference t-shirt just said State of the Map, not State of the Map US, which seemed odd. I think most of the conference people handling registration were paid staff, which was a difference.

Welcome

There was a welcome talk which I thought was rather good at the time, but seems it wasn’t very memorable, because I can’t actually remember much of what was said!

The conference had over 500 people check in at registration, a big growth from the 2009 SOTM-US which had about 50 people.

First session, coffee break

I ended up talking with the people from Amazon for all of this session, so missed the talks. I’m hopeful that they’ll be able to commit some EC2 credits to do some dev work and performance testing. Of course, I was the same way after previous conferences and nothing came of it.

It doesn’t sound like they’re willing to commit to helping any osm.org infrastructure with resources in any ongoing manner.

Right now the main OpenStreetMap.org stylesheet uses Unifont as a fallback font to render Chinese, Japanese and Korean (CJK) characters, as well as any other characters not present in the DejaVu font. Unifont is mainly designed to support all characters, and is not designed to look good.

I’m looking at Droid Sans Fallback, a free font developed for Android, and evaluating if it would be a better fallback font than Unifont. Because I don’t read Chinese, Japanese or Korean, I could use help.

I have prepared a demo at http://tile.paulnorman.ca/demo/fonts.html with three layers: conventional OSM.org, tiles without any fallback font, and tiles using Droid Fallback as a fallback font.

What I would like is for people to look at the difference between the conventional OSM.org and Droid Fallback tiles and see which is easier to read for the CJK glyphs. The tiles without any fallback font can be used to find areas where DejaVu doesn’t have glyphs and the fallback font is being used.

Some examples

Japanese cities: http://tile.paulnorman.ca/demo/fonts.html#9/35.443/138.247

Japanese train stations: http://tile.paulnorman.ca/demo/fonts.html#16/36.415/139.325

Korean cities: http://tile.paulnorman.ca/demo/fonts.html#9/37.25/127.22

Chinese tourist attraction: http://tile.paulnorman.ca/demo/fonts.html#15/39.94/116.48

Please keep in mind that

• My server is not nearly as powerful as tile.osm.org, so renders slower and has less cached data
• Only Asia is loaded on my server
• The data is a couple of days old and isn’t being updated

I would like some feedback on if Unifont or Droid Sans Fallback looks better. Please keep in mind that I don’t read the languages being rendered.

I had to write some osm2pgsql command lines for benchmarking recently. This is how it feels

Just a preview of what I’m working on right now

Today I surveyed both the new Westbound Port Mann Bridge and the north portion of the SFPR. Either of these would of been a lengthy trip in itself, combined they took two hours of driving to get traces and I still didn’t get everything surveyed.

I used my camera to take photos with an interval of 8 seconds and my eTrex 20 sampling at 1 Hz, then correlated everything in JOSM.

I got off at 152 Street and went along 104 Avenue to cross over at 160th. I drove around the south side of the forest on the slope before heading over to join up with the SFPR, 104 Avenue and 176 Street Extension intersection. Little did I know how many times I would be going through here.

I then headed south to Highway 15, 96 Avenue and Golden Ears Way, heading down Golden Ears to do a U-turn in a side street. I then went straight on to the SFPR through the 104 Avenue intersection. From here it was an easy drive on new quiet pavement with no chance to turn off or turn around until King Road and 138 Street. I was struck by how few cars there were on the road. There were occasional trucks but I didn’t see another car going my direction. The view was rather nice in parts

… See full entry

After some technical and legal work, I now have a number of imagery layers hosted on a rented server.

These layers cover from Vancouver to Hope in 20cm or better and Lions Bay to Pemberton as 40cm or better. 10cm imagery is available for Vancouver, Richmond, Ladner, West Delta, the North Shore, Whistler and Surrey.

Most of the imagery was taken in 2009, but Surrey is from 2011.

I’ve prepared a page with links to the imagery and Potlatch2 and JOSM URLs at http://imagery.paulnorman.ca/tiles/about.html

Some layers may be slow initially loading at high zooms as they may need to fetch from a remote server.

An example of two layers from there and Bing is http://imagery.paulnorman.ca/tiles/surreyexample.png. Starting in the top left going clockwise the layers are Surrey2011, DataBC bc_gvrd_east_2009 and Bing.

Note: This post is a duplicate of a talk-ca mailing list post

After nearly a month on a 4-drive RAID0 array…

time ~/osm/osmosis-0.40.1/bin/osmosis --rb planet-120401.osm.pbf --lp interval=60 --wd database=osm user=openstreetmap password=openstreetmap allowIncorrectSchemaVersion=yes 2>&1 | tee apidb.log
5-May-2012 4:27:02 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.40.1
5-May-2012 4:27:02 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
5-May-2012 4:27:02 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.

...

1-Jun-2012 8:36:48 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline complete.
1-Jun-2012 8:36:48 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Total execution time: 2347785558 milliseconds.

real    39129m46.249s
user    1418m52.368s
sys     415m9.705s

Surrey, BC is holding an Open Data Hackathon on Sunday November 20th. More
details are at http://www.surrey.ca/city-services/10036.aspx

I will be attending and looking at writing some new conversion scripts.

Does anyone have suggestions for any printed OSM materials to bring?