Custom Maps using OpenStreetMap and D3.js

You need some geo data in your project? D3 has you covered - here’s how.

I recently needed to extract some geographic data from OpenStreetMap and use it in browser-based applications. OpenStreetMap is by far the largest and most used openly licensed mapping data collection on the web. It has a surprisingly good data quality and a lot of different means of querying its vast database. I will show you how you can do the same and render this data using D3.js.

Let’s start by querying OpenStreetMap to get the raw data we need. There are various ways of doing this, but one of the simplest ways is to use the Overpass API. How you use this API is by submitting a query written in an admittedly hard-to-learn query language to a REST endpoint. What we get back is either an XML or JSON response. I recommend you use JSON since it is easier to read and has tons of good tooling around it. The response size may vary wildly depending on how large of an area you query and how many points, paths, and relations are in the response. To make writing these sometimes complicated queries easier there is a small web application called Overpass Turbo. You can write your queries here and get visual feedback of what the results will look like. So let us query some data - let’s query all data needed to render the Albula mountain railway line:

[out:json][timeout:25];
rel[name="Albulabahn"];
(._;>;);
out;

This query will visualize all points, ways and of the Albula line relation:

Visual representation of the GeoJson returned from the Overpass API

Visual representation of the GeoJson returned from the Overpass API

We now have a query that gives us everything we need to render a map of the Albula line. Let’s use a little typescript code to render an SVG that we can then further process into a stylized map:

const response = await fetch("https://overpass-api.de/api/interpreter", {
  body: `data=${encodeURIComponent(`[out:json][timeout:25];
  rel[name="Albulabahn"];
  /*added by auto repair*/
  (._;>;);
  /*end of auto repair*/
  out;`)}`,
  method: "post",
});

const result = (await response.json()) as OSMResponse;

We now have an OSMResponse structure. This structure contains relations, ways, and points that we need to render the map. Let’s do some filtering and indexing next so we have ways and points:

const nodes = result.elements.filter((x): x is Point => x.type === "node");
const ways = result.elements.filter((x): x is Way => x.type === "way");

We can now use d3.js and the geojson library to preprocess and then render the data. We first reduce all the ways to LineStrings:

const wayElements = ways.map((x) => {
    const wayNodes = x.nodes.map(n => nodes.find(y => y.id === n)).filter(x => x !== undefined).map(p => [p.lon, p.lat] as Position);

    return <GeoJSON.LineString>{type: "LineString", coordinates: wayNodes}
}).reduce((acc, next) => <GeometryCollection>{geometries: acc.geometries.concat(next)}, <GeoJSON.GeometryCollection>{geometries: []});

We now have all the ways in the relationship as LineString. Next, we need to make D3 render this data. For this, we need configure a projection. A projection is how a geographical point (which is a point on this, almost, sphere we call earth) onto a two-dimensional plane we call a map. There are many projections you can use and quite a few funny ones too! But these small-scale we are working with here, it does not matter much what projection you choose - so I went with the Mercator projection. We also need to find our viewport. This means, that we need to define a rectangle that contains all our points - or at least the points we want to draw in our SVG:

const projection = d3.geoMercator().fitExtent([[10,10],[clientRect.width - 10, clientRect.height - 10]], {type: "LineString", coordinates: nodes.map(x => [x.lon, x.lat])})
const pathFactory = d3.geoPath().projection(projection);

As can be read in the script above, we can use the fitExtent function to have D3 figure out a viewpoint that is certain to contain all our LineStrings. The last step remaining is to draw the SVG again using D3:

svg.selectAll('path').data(wayElements.geometries).enter().append('path').attr('d', d => `${pathFactory(d)}`);

We are almost there. What we have now is an SVG that we can either use as-is or we can further stylize it since it does not look like much yet. We can do this either using typescript code to further customize our SVG and make it more presentable. I just copies the SVG into a file and opened it in Inkscape, modified the viewport a bit, added a background, and cleaned up the paths I did not want and this is how it turned out:

Stylized OSM map of the Albula line between Preda and Berguen

Stylized OSM map of the Albula line between Preda and Berguen

I liked it a lot, so I did the same for the Gotthard line:

And since I was on a roll, I modified my Overpass query a bit and drew the entire track system of the Zuerich central station:

Stylized OSM map of the track system of the freight station of Altstaetten and the main station in Zuerich

Stylized OSM map of the track system of the freight station of Altstaetten and the main station in Zuerich

Now, this query returned quite a lot of data and the image has ~150'000 paths and though that is not much by mapping-data-size standards, it did put my browser to work quite a bit. Inkscape struggled as well with the sheer number of paths it had to render. And if you have ever asked yourself why there is no SVG (or any other vector-file-formatted) Mapping application out there - we now have the reason why. If you want to render maps at a larger scale you are much better served by using a combination of Mapnik and PyOsmium to rasterize the maps you want and serving them using a tile server.