Mexico released a huge amount of open data not too long ago.

Huge: Mexico's statistical institute INEGI goes open data @INEGI_INFORMA (via @rodowi — Alex Barth (@lxbarth)

A lot of this data is geospatial, so I say yummie! Alex Barth wrote about this data, that comes from the Mexican national statistical agency INEGI, on his diary before, with a nifty map to show how rich this data is:

(My mediocre animated GIF skills really don’t do it justice - check out Alex’s blog post to see an interactive map.)

So now the question becomes: how do we get some (or all?) of this data into OSM? This is not straightforward - OSM already has rich data in many places in Mexico we would definitely want to keep.

Here at the Telenav OSM team, we have come up with an answer to this question. We call it Cygnus - The Bringer of Balance. Let me explain in a few visuals what Cygnus does.

Consider this area in the Aguascalientes region. There is some OSM data there:

If we look at the Bing aerial images, we can clearly see that there’s an entire village there that is not mapped though!

But INEGI has most if not all the roads in this village in their now open dataset Conjunto de Datos Vectoriales de Carreteras y Vialidades Urbanas.

After converting the original data attributes mapped to OSM-appropriate tagging, saving the result as an OSM file, and loading it into JOSM, it looks like this:

OK, that is nice, but we still have two separate layers that are unconnected, and even if we merge them, we will still have to manually resolve duplicate ways and connections between the ways from OSM and those from the INEGI data.

This is where Cygnus comes in - a new conflation technology we developed at Telenav specifically to tackle this.

Cygnus, as the Bringer of Balance, takes as its input a base OSM file in PBF format, as well as what we call an ‘enhancing’ file, also in PBF format. It will then compare and conflate the two inputs and output one JOSM XML file that can be merged with OSM base data straight away. This detail shows the merged layers with all the INEGI ‘imported’ ways connected to the pre-existing OSM way:

Even though Cygnus does a pretty amazing job merging OSM data with an ‘enhancing’ layer, you will still need to check the result before you upload. Take this example here:

The highway=secondary was the pre-existing OSM data, and the highway=residential; oneway=yes came from INEGI data. It is clear from Bing imagery that the two ways should be connected, yet they are not. Cygnus has a (tweakable) distance threshold it uses when it decides if two ways should be connected or not. In this case, the INEGI way was too far away, so it remained disconnected.

There are a few other things to consider when you work with a Cygnus-produced change file:

• Cygnus will never degrade existing OSM data
• When both OSM and the enhancing layer have the same way, Cygnus will always keep the original OSM geometry, but it will optionally import name and other useful tags.

We are currently working on an import plan for INEGI data that will make heavy use of this new technology. More about this very soon! In the mean time, watch our SOTM US talk on the INEGI data and OSM in Mexico.