29:null 2a:["$","title",null,{"children":"How to make a vectortile-basemap | Geodan research | Observable"}] 2f:T46a,the "BasisRegistratie Grootschalige Topografie", roughly translatable as the "Base Registration of Large Scale Topography". All assets have a location as a point, line or polygon where it should be noted that the whole country is covered without gaps in polygons, as you can see in the picture.

Here you see an example of an instructional picture on how these polygons should be mapped.

As you can imagine, this dataset is rich and maybe too rich when all you want to see is the outline of a city. Therefore, there is another nationwide dataset available called 30:T8ee,I will not explain how topojson and its functionality works but I will show you the SQL code that comes with it. Since the query is rather long, I will show it in three steps. The first step is creating a big Featurecollection in Geojson, since we need to feed it to javascript. (I should mention here that dealing with geojson became easier in newer postgis versions) WITH geojson AS ( SELECT json_build_object( 'type', 'FeatureCollection', 'features', json_agg( json_build_object( 'type', 'Feature', 'geometry', ST_AsGeoJSON(ST_ForceRHR(geom))::json, 'properties', ('{"layer":"' || layer || '","class":"'|| class || '","color":"' || color||'","relh":"' || relh||'","mergekey":"' || mergekey || '"}')::jsonb ) ) )::jsonb as geojson, count(geom) as numfeats FROM bgt ) The second step is where the magic happens and we call several wrappers to javascript functions that will simplify our polygons without breaking the network topology. ,features AS ( SELECT plv8.d3_mergetopology( plv8.d3_simplifytopology( plv8.d3_totopojson(geojson, 1e8), 5 ), 'mergekey') as geojsonfeature FROM geojson WHERE numfeats > 0 ) The last step puts it back from Geojson to postgis geometries again. SELECT (geojsonfeature->'properties'->>'class')::text as class, (geojsonfeature->'properties'->>'color')::text as color, (geojsonfeature->'properties'->>'relh')::integer as relh, ST_SubDivide( (ST_Dump( ST_CollectionExtract( ST_MakeValid(st_setsrid(ST_geomFromGeoJson(geojsonfeature->>'geometry'),28992)), 3 ) )).geom )::Geometry(Polygon,28992) as geom FROM features; If this all seems like a tedious task, that is because it is... But we still think it is worth to have polygons without gaps in between. You can try the functionality yourself by using the extension we published: https://github.com/Geodan/plv8_geo 31:Tca5,## Ready to serve Our data has now been reclassified, merged, simplified and subdivided and is ready to use. The result is a table full of geometries for different zoomlevels. Since pg_tileserv would by default simply serve *all* geometries, regardless of zoomlevel, we have to find a way to deal with that. Luckily, pg_tileserv has a feature that is both powerful and simple called 'function layers'. When you create functions in your database that follow a certain layout, they will automatically be picked up by pg_tileserv as a layer and show up in your layers overview. You can write these functions in every language that is available to postgres, so I started with a 'simple' SQL function: CREATE FUNCTION basiskaart.getmvt(z int,x int, y int) RETURNS bytea LANGUAGE 'sql' AS $BODY$ WITH bounds AS (SELECT ST_TileEnvelope(z, x, y) AS geom), mvtgeom AS ( SELECT ST_AsMVTGeom(ST_Transform(t.geom,3857), bounds.geom) AS "geom", "color","zmin","zmax","relh" FROM basiskaart.polygons_zlevels t, bounds WHERE ST_Intersects(t.geom, ST_Transform(bounds.geom,28992)) AND z BETWEEN t.zmin AND t.zmax ) SELECT ST_AsMVT(mvtgeom, 'basiskaart.getmvt') FROM mvtgeom; $BODY$; Despite the relative simplicity of the functions it turned out to be rather slow though. This is likely due to the fact that the planner has difficulty in incorporating the indexes in the plan. Where writing the function in 'plpgsql' it turns out to run quicker. CREATE OR REPLACE FUNCTION basiskaart.getmvt(z int,x int, y int) RETURNS bytea LANGUAGE 'plpgsql' AS $BODY$ DECLARE result bytea; BEGIN EXECUTE ' WITH bounds AS (SELECT ST_TileEnvelope($1,$2,$3) AS geom), mvtgeom AS ( SELECT ST_AsMVTGeom(ST_Transform(t.geom,3857), bounds.geom) AS "geom", "color","class","zmin","zmax","relh" FROM basiskaart.polygons_zlevels t, bounds WHERE t.geom && ST_Transform(bounds.geom,28992) AND $1 > t.zmin AND $1 <= t.zmax ) SELECT ST_AsMVT(mvtgeom, ''basiskaart.getmvt'') FROM mvtgeom; ' INTO result USING z,x, y; RETURN result; END $BODY$; Another reason that that the plpgsql function is faster could be that it gets cached which is cheaper the next time you run it. In the end there is a small disadvantage of more boilerplate in plpgsql but it saves you a ton of headache. As a bonus tip: we had problems with the ST_Intersects function crashing on some *very* invalid geometries. This was solved by only using the bounding box intersection (*&&*). On top of solving this problem, it may have also sped up the query a little since there is no need to check for a real intersection since ST_AsMVTGeom will ignore any overshoots.32:T714, viewof buildingsmap = { const container = html`

`; let resolve; container.value = new Promise(_ => resolve = _); yield container; // Give the container dimensions. const map = new maplibre.Map({ container: container, zoom: 16, center: [4.913, 52.342], pitch: 60, maxPitch: 60, style: 'https://saturnus.geodan.nl/mapbox-viewer/styles/freetilehosting/positron.json', scrollZoom: false }); map.on('load', function () { map.addLayer({ "id": "building3D", "type": "fill-extrusion", "source": { type: 'vector', tiles:["https://research1.geodan.nl/anneb.bag13_lod1213_2d_levering/{z}/{x}/{y}.pbf"], bounds: [3.25524871110589,50.7347266517351,7.24079584680245,53.5258520631408], minzoom: 13, maxzoom: 18 }, "source-layer": "anneb.bag13_lod1213_2d_levering", "minzoom": 13, "maxzoom": 24, "paint": { "fill-extrusion-color": ["case", ["<",["get","bouwjaar"],1001],"black", ["<",["get","bouwjaar"],1700],"#7f0000", ["<",["get","bouwjaar"],1800],"#b30000", ["<",["get","bouwjaar"],1900],"#d7301f", ["<",["get","bouwjaar"],1930],"#ef6548", ["<",["get","bouwjaar"],1960],"#fc8d59", ["<",["get","bouwjaar"],1975],"#fdbb84", ["<",["get","bouwjaar"],1990],"#fdd49e", ["<",["get","bouwjaar"],2000],"#fee8c8", ["<=",["get","bouwjaar"],2025],"#fff7ec", "#fff7ec"], "fill-extrusion-height": ["-",["get","dd_h_dak_max"],["get","h_maaiveld"]], "fill-extrusion-base": 0, "fill-extrusion-opacity": 0.95 } }); }); resolve(map); // Expose the Map instance as the view’s value. invalidation.then(() => map.remove()); } 33:T956,## Conclusions We came to the end of our journey, only some conclusions and recommendations to go: Postgis already was a very convenient tool for storing and handling your data, but now it also turns out a very performant service to serve your vectortiles. I found it very easy to deploy my fresly created data from postgis into a vectortile service. Actualy, there is hardly any deployment needed, just point your pg_tileserv to the right database and you are done. On top of that the performance is phenomenal. You may have noticed the example above is still a bit laggy but that has to do with the rest of the 3D datasets, not with the vectortiles. All in all we will keep using this setup for our future mapping needs and will likely also explore the use of pg_featureserv which has nice examples with for instance routing. ### Recommendations Vectortiles open up a whole new world of possibilities, especially when it comes down to interactivity in the viewer. There are a couple of things that might serve us even better when incorporated in the standard: * Serve multiple projections Although webmercator is the official projection of the vector tile specification and it is also possible to do this on the client-side, it would be great if ST_AsMVTGeom would serve more than only webmercator. * Serve arrays of data in vector tile Currently you need to store your data as 'attributes' fields, with one numerical value or string per attribute. It would be interesting to store arrays as attribute since it would give you the option to serve things like timeseries or multispectral data. * Keep the Z coordinate in the MVT How cool would it be if you can serve real three dimensional houses instead of the extruded blocks. This is way beyond the official specification because it would imply rewriting the client but for the back-end technically not a big change to keep the Z coordinate in your vertex output. A point of concern might be the simplification that is not that simple in three dimensional geometries but in many cases it would be ok to ignore the z-value in the simplification and just output whatever is left after removing vertices.34:T430,## Fin Some places I got inspiration from for this presentation: * https://research.geodan.nl/all-dutch-3d-buildings-in-1-9gb/ A very nice post of my colleague Anne Blankert and the basis of our current work, on how he did all the buildings in 3D with pg_tileserv * https://nieneb.github.io/styling_vector_tiles/ An inspiring FOSS4G presentation of Niene Boeijen with lots of amazing examples in styling vector tiles. * https://callforpapers.2021.foss4g.org/foss4g2021/talk/AZB8QK/ A talk from Stefan Brand I unfortunately misseed last FOSS4G on his experience with vectortile services. * https://www.youtube.com/watch?v=TXPtocZWr78 Extensive presentation on pg_tileserv and pg_featureserv from its author, Paul Ramsey, full of fun examples I would have never come up with myself.35:T521,threedeeconfig = { return { id: 'maquette', dracoDecoderPath: "https://www.gstatic.com/draco/versioned/decoders/1.4.1/", tilesets: [ { "id": "GebouwenNL", "url": "https://beta.geodan.nl/data/buildingtiles-dev/buildingtiles_nl_compressed_3857/tileset.json", "horizonClip": true, "horizonFactor": 200 } , { "id": "Niveau0", "url": "https://fileserv.beta.geodan.nl/i3dm/dev/nl_niveau_0/tileset.json", "horizonClip": true, "horizonFactor": 200 }, { "id": "Niveau1", "url": "https://fileserv.beta.geodan.nl/i3dm/dev/nl_niveau_1/tileset.json", "horizonClip": true, "horizonFactor": 200 }, { "id": "Niveau2", "url": "https://fileserv.beta.geodan.nl/i3dm/dev/nl_niveau_2/tileset.json", "horizonClip": true, "horizonFactor": 200 }, { "id": "Niveau3", "url": "https://fileserv.beta.geodan.nl/i3dm/dev/nl_niveau_3/tileset.json", "horizonClip": true, "horizonFactor": 200 } ] } }36:[{"id":"046c60bffb1a9fbb22368314e219052f859cb3ac1bcb449339008f412d4d6c71a95f8fe8239130e756f269f49a56439ae34e611e7c54274185df97c72f5952d0","href":"https://static.observableusercontent.com/files/046c60bffb1a9fbb22368314e219052f859cb3ac1bcb449339008f412d4d6c71a95f8fe8239130e756f269f49a56439ae34e611e7c54274185df97c72f5952d0","name":"brt_linework.png","type":"image/png","lastModified":1637683391023.441,"size":19410},{"id":"f90f36ac745a9f550f697605b3461531249fa34ab5386ee01e3e5910b8687b141d1bb30113769c2513c35056eeb70486a4aa93750cce8a6cac42f0cdb8cb4106","href":"https://static.observableusercontent.com/files/f90f36ac745a9f550f697605b3461531249fa34ab5386ee01e3e5910b8687b141d1bb30113769c2513c35056eeb70486a4aa93750cce8a6cac42f0cdb8cb4106","name":"image.png","type":"image/png","lastModified":1637677627767.599,"size":757483},{"id":"827d19c171a800cf167c389be267b418b762496c8e9376b7daab3192e614b170f4cd7bb5f2c0b7bab4b7351bec3c16f9fd0bf5e17e1856bbe12861afd85656a3","href":"https://static.observableusercontent.com/files/827d19c171a800cf167c389be267b418b762496c8e9376b7daab3192e614b170f4cd7bb5f2c0b7bab4b7351bec3c16f9fd0bf5e17e1856bbe12861afd85656a3","name":"image@2.png","type":"image/png","lastModified":1637678304061.705,"size":917425},{"id":"fbaf4738154939886523e6f23a73ee7fb68d222319f41e6db3be5135f2cd2abab5704c5b39ae761c80a19df189d3941da058eafc51d6d56554426a427397efd4","href":"https://static.observableusercontent.com/files/fbaf4738154939886523e6f23a73ee7fb68d222319f41e6db3be5135f2cd2abab5704c5b39ae761c80a19df189d3941da058eafc51d6d56554426a427397efd4","name":"image@5.png","type":"image/png","lastModified":1637679138998.494,"size":29973},{"id":"b155bf48b7db00d9ffd8306e804fcc9548df983da4240930a8e7fd79630272847840f876205918d43c07623d30ba7432adfc644a8f9b7fc6b9fb332263f35473","href":"https://static.observableusercontent.com/files/b155bf48b7db00d9ffd8306e804fcc9548df983da4240930a8e7fd79630272847840f876205918d43c07623d30ba7432adfc644a8f9b7fc6b9fb332263f35473","name":"image@3.png","type":"image/png","lastModified":1637678554027.89,"size":642291},{"id":"cca610b84fb12368dd4cb1349fe7a60d49c21f8ef285c7ca2b155280e82ba3e8add6a22f1abe06d43f206dfc7c219591733e7d15d7e75cb5d0bccf5132900af6","href":"https://static.observableusercontent.com/files/cca610b84fb12368dd4cb1349fe7a60d49c21f8ef285c7ca2b155280e82ba3e8add6a22f1abe06d43f206dfc7c219591733e7d15d7e75cb5d0bccf5132900af6","name":"image@6.png","type":"image/png","lastModified":1637679198118.288,"size":568966},{"id":"40be06fd74bb88178940921858d0be797b30649baf22ff23cc2ab74cda9ccf6ac455a0dc2a582758faf7b12ff80b406f94bc7c177e62c67587f9032495ab3e52","href":"https://static.observableusercontent.com/files/40be06fd74bb88178940921858d0be797b30649baf22ff23cc2ab74cda9ccf6ac455a0dc2a582758faf7b12ff80b406f94bc7c177e62c67587f9032495ab3e52","name":"image@7.png","type":"image/png","lastModified":1637679628797.171,"size":45309},{"id":"b4b0fb9f50ae00bc0884d9ee387906ed0c09519d088f19ff3c1e57278ed573c4491d34ecd023cdc970d33e91a28b6a59d98a666638fbd74a70910771b076fc91","href":"https://static.observableusercontent.com/files/b4b0fb9f50ae00bc0884d9ee387906ed0c09519d088f19ff3c1e57278ed573c4491d34ecd023cdc970d33e91a28b6a59d98a666638fbd74a70910771b076fc91","name":"corine_lod.png","type":"image/png","lastModified":1637683789435.538,"size":10525},{"id":"e6c5376994cf174f286d2c78f721be7a29d37f61e4f80b10d9b8b1140fbd21823401aaa9f47002f6503e6681cf9214a51dbaa9b971341b7c4e1dcd453f02eb05","href":"https://static.observableusercontent.com/files/e6c5376994cf174f286d2c78f721be7a29d37f61e4f80b10d9b8b1140fbd21823401aaa9f47002f6503e6681cf9214a51dbaa9b971341b7c4e1dcd453f02eb05","name":"bgt_lod.png","type":"image/png","lastModified":1637683789624.386,"size":132066},{"id":"2cec7430db418299d85f97788914910e1b31c5628852da302183dc3f44d73cdfb07f840ac04caf56e677fa90e35eb9263f0b353e9dbbf3ddb2cddeb32aa6671d","href":"https://static.observableusercontent.com/files/2cec7430db418299d85f97788914910e1b31c5628852da302183dc3f44d73cdfb07f840ac04caf56e677fa90e35eb9263f0b353e9dbbf3ddb2cddeb32aa6671d","name":"brt_lod.png","type":"image/png","lastModified":1637683789550.943,"size":86435},{"id":"5d6b74e0a4d3b45a571f0f8727f1d880a212f4ec7c095b043558ae10e0b9d3bf5fa61850e32566be1460acca422f4679bb653cc2ef1e4188498766e4ea9f85f9","href":"https://static.observableusercontent.com/files/5d6b74e0a4d3b45a571f0f8727f1d880a212f4ec7c095b043558ae10e0b9d3bf5fa61850e32566be1460acca422f4679bb653cc2ef1e4188498766e4ea9f85f9","name":"image@8.png","type":"image/png","lastModified":1637686390609.855,"size":8924},{"id":"dbe9dbc6fb0311b70f479959c94fb15ff820de0c2bb3c9dee4e1e9aef6464ab8f644234e8933a613afcb7db24a5fd9d745d4f4c7156f515c9e863f44b1617184","href":"https://static.observableusercontent.com/files/dbe9dbc6fb0311b70f479959c94fb15ff820de0c2bb3c9dee4e1e9aef6464ab8f644234e8933a613afcb7db24a5fd9d745d4f4c7156f515c9e863f44b1617184","name":"image@9.png","type":"image/png","lastModified":1637687230913.634,"size":370040},{"id":"433ae7ab7f68e1867084fbb1f42ed798fe3988f18d98e7c9b6a2e759fb603df8be59dc0c88639f9aa25b6205f9548504dacd47025a71641148fc2edacef22d7a","href":"https://static.observableusercontent.com/files/433ae7ab7f68e1867084fbb1f42ed798fe3988f18d98e7c9b6a2e759fb603df8be59dc0c88639f9aa25b6205f9548504dacd47025a71641148fc2edacef22d7a","name":"image@10.png","type":"image/png","lastModified":1637687587357.105,"size":342662},{"id":"3591cc64482314b82035050a05af31e3729621206bc7017b463ae744fd68a7fd95881223fc6e5d424c524291492af2e6513a240ec7c317a48fb723abba447e5b","href":"https://static.observableusercontent.com/files/3591cc64482314b82035050a05af31e3729621206bc7017b463ae744fd68a7fd95881223fc6e5d424c524291492af2e6513a240ec7c317a48fb723abba447e5b","name":"image@11.png","type":"image/png","lastModified":1637689607396.992,"size":87127},{"id":"16cb11fb884f251e2f48000a95c8ccc426331d2ea4b05ec660b83f2588b8373a6a15e2caa9e8d7930169f4b073e5168c19ee2d064a9300c7ce3864018c0c89f5","href":"https://static.observableusercontent.com/files/16cb11fb884f251e2f48000a95c8ccc426331d2ea4b05ec660b83f2588b8373a6a15e2caa9e8d7930169f4b073e5168c19ee2d064a9300c7ce3864018c0c89f5","name":"image@12.png","type":"image/png","lastModified":1637689620774.09,"size":51672},{"id":"fc44bc693c2f3a74f34cecf0beeab7200b0e14a4a8a3536cf1271ee5116db4a08ae16d426190e58f2de259f3e525ee95a5cd985efbcccd3dd906f1cc2bf4e1e0","href":"https://static.observableusercontent.com/files/fc44bc693c2f3a74f34cecf0beeab7200b0e14a4a8a3536cf1271ee5116db4a08ae16d426190e58f2de259f3e525ee95a5cd985efbcccd3dd906f1cc2bf4e1e0","name":"image@13.png","type":"image/png","lastModified":1637690903379.484,"size":632849},{"id":"dae9b70e2d09d2da8f68d8806abac4e81a646afaeea827bdaae94653f9891af814490c228b73f751859b5c56ca9d7a79beee48a02e723452f9ef080e7ccf008f","href":"https://static.observableusercontent.com/files/dae9b70e2d09d2da8f68d8806abac4e81a646afaeea827bdaae94653f9891af814490c228b73f751859b5c56ca9d7a79beee48a02e723452f9ef080e7ccf008f","name":"image@14.png","type":"image/png","lastModified":1637761008999.425,"size":130687},{"id":"601ba4c51372a371ab4277b3c656e312ae5a12c3c0a94120a01acd68112bcf24858099ca9f64d405da92eb4c50626b99df4018c83df0ebe3a49d0d7adb1f7539","href":"https://static.observableusercontent.com/files/601ba4c51372a371ab4277b3c656e312ae5a12c3c0a94120a01acd68112bcf24858099ca9f64d405da92eb4c50626b99df4018c83df0ebe3a49d0d7adb1f7539","name":"bgt_linework.png","type":"image/png","lastModified":1637683391197.681,"size":70789},{"id":"d73955a2ad79152c876f2ca939609d5ab5fb2eff556a783cd325c206fe427eec2dbf2bd2ba4c860a251260b3db4e251700c98744940e2bb27fb1e74836c1168e","href":"https://static.observableusercontent.com/files/d73955a2ad79152c876f2ca939609d5ab5fb2eff556a783cd325c206fe427eec2dbf2bd2ba4c860a251260b3db4e251700c98744940e2bb27fb1e74836c1168e","name":"corine_linework.png","type":"image/png","lastModified":1637683391097.398,"size":1013},{"id":"11c2b484cf1b62dc048361e70ca1c76df2aa7aaca5747cdd645defb46192ae714a7184d097e7fc9349a861ea3cd4919b56ab053566ea6f9c2b6dae87ed841066","href":"https://static.observableusercontent.com/files/11c2b484cf1b62dc048361e70ca1c76df2aa7aaca5747cdd645defb46192ae714a7184d097e7fc9349a861ea3cd4919b56ab053566ea6f9c2b6dae87ed841066","name":"image@4.png","type":"image/png","lastModified":1637678597818.328,"size":625630}] 2d:{"id":"0b209c5efcd14c8e","slug":"how-to-make-a-vectortile-basemap","title":"How to make a vectortile-basemap","thumbnail":"c112a913999b0c69772bfa3a74cb9fcd064e19cdaa67418a3d91b07a0100c28a","access_level":"public","create_time":"$D2021-11-23T14:22:43.894Z","update_time":"$D2021-12-21T16:11:05.455Z","view_time":null,"first_public_time":"$D2021-11-30T10:05:16.614Z","has_unpublished_edits":null,"fork":{"id":"662080314eb2cb2c","slug":"blog-post","title":"Blog Post","owner":{"id":"f35c755083683fe5","login":"observablehq","name":"Observable"}},"stars":5,"starred":false,"owner":{"type":"team","id":"e51ebae4171835c5","login":"geodan","name":"Geodan research","avatar_url":"https://avatars-next.observableusercontent.com/6813a9eccdca19e5c7220422908591478b06a880d6e666b49ba801798927a67f","tier":"pro"},"version":771,"publish_version":771,"role":null,"authored":false,"authors":[{"type":"individual","id":"297b5a11f99fd24d","login":"tom-geodan","name":"Tom van Tilburg","avatar_url":"https://avatars-next.observableusercontent.com/ed4a5bf29b4eccf3ddf6f6bdfb1dddd94846ca0861d162d0bc598f1e5f33a06f","tier":null}],"members":[{"type":"individual","id":"297b5a11f99fd24d","login":"tom-geodan","name":"Tom van Tilburg","avatar_url":"https://avatars-next.observableusercontent.com/ed4a5bf29b4eccf3ddf6f6bdfb1dddd94846ca0861d162d0bc598f1e5f33a06f","tier":null,"role":"editor"},{"type":"team","id":"e51ebae4171835c5","login":"geodan","name":"Geodan research","avatar_url":"https://avatars-next.observableusercontent.com/6813a9eccdca19e5c7220422908591478b06a880d6e666b49ba801798927a67f","tier":"pro","role":"viewer"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":4,"collection":null,"type":"notebook","body":{"title":"How to make a vectortile-basemap","cells":[{"id":0,"value":"# How to make a vectortile-basemap\n## with postgis and pg_tileserv","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":4,"value":"\n

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":41,"value":"---\n\nThis blog post is an excerpt of a talk I gave earlier at the Postgisday 2021\n\n---","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":8,"value":"I would like to tell you how we made a map.\n\nNot just a map but a digital basemap out of vectortiles with the help of Postgis and pg_tileserv. \nThis post will take you along on the journey we had from the raw data until publishing on the web.\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":35,"value":"We start our journey with a demand. As a company we have been doing webmaps for a long time, but we had a couple of demands that forced us off the beaten track.\n\nFirst of all, we wanted to have a nicer looking base-layer, preferably vector based instead of raster\nSecondly we wanted to be able to switch between multiple 'themes' or 'stylings'.\nOn top of that we know that part of our base data is going to be updated once a week\nAnd since we are a serious company ;-) , we want to adhere to the standards.\n\nAnd, not unimportant: ALL of our data is already in postgis.\n\nSo… what can we do?\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":764,"value":"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":80,"value":"Obviously, we couldn't hang on to our good old tiled raster sets. Why deliver these clunky rasters when all your data is in vector? Here we have only one serious option which is vectortiles.\n\nGeoserver, which we already use, can do vectortiles. But…. it is also a bit bulky. Geoserver runs on java and takes a non-trivial amount of resources.\nAlso, we already did some early experiments with the vector-tile function ST_AsMvt in postgis, which gave us very promising results. \n\nSo this leaves us with a couple of options, for instance tegola which has been around for quite some time, but also martin and postile are good services.\nThere was one contemporary work that stood out though because of the simplicity of it, which is **pg_tileserv**.\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":85,"value":"## Intermezzo on the data\n\nBut first…\nsomething on the Dutch data that we are dealing with.\n\n

Image courtesy from somewhere on the internet

\n\nIn the Netherlands, there is the tendency to have an asset management of the whole landscape. Anything in the public domain has a name and a place.\nSo the roads, the canals, the grassy fields, every tree, every windmill and of course the flowerfields are mapped.\nThe database with all these assets is called…. \n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":87,"value":"$2f","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":108,"value":"the 'BasisRegsitratie Topografie'.\n\nThis dataset has less detail than the previous one, and is more suitable for mapping on a 1:10.000 scale. It is the standard dataset for paper topographic maps in the country.\n\n

\n\nAs you can see, it makes pretty decent maps, this is an example from a styling in QGIS.\n

Example QGIS styling of top10nl. *Image courtesy of J.W. van Aalst*

\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":122,"value":"Then I should mention one more dataset which is on a European scale called \"Corine-landcover\".\n\nIt is great for overview-maps where you have a whole country in one view and I will show you later how we used it in our basemap.\n

\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":129,"value":"## Get on with the map\n\nObviously, we are going to use these three datasets as separate levels of detail for our different zoomlevels so we don't have to do the massive simplification ourselves. \nTo get an idea of the effect of how these datasets relate to each other, you can see here first the Large Scale Topography… \nfollowed by the Normal 1:10000 topographical data….. \nfollowed by the Corine landcover set. \n

Three levels of detail from three different datasources

${viewof lodimg}

\nAlthough not perfectly matching, you can see where this is going when we start using it for different zoomlevels.\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":179,"value":"viewof lodimg = {\n const imgs = [\n {name:'BGT',value:FileAttachment(\"bgt_lod.png\")},\n {name:'BRT', value:FileAttachment(\"brt_lod.png\")},\n {name:'Corine', value:FileAttachment(\"corine_lod.png\")}];\n return Inputs.radio(imgs, {value: imgs[0],label: \"Level Of Detail\", format: x => x.name})\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":138,"value":"All in all, we have over 35 million polygons in our postgres database to be used as a basemap.\nSome work is still needed though, because as you can see there is a large difference between the datasets.\n

Differences between levels of detail (BGT, BRT, Corine).

","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":153,"value":"viewof favorite = {\n const imgs = [\n {name:'BGT',value:FileAttachment(\"bgt_linework.png\")},\n {name:'BRT', value:FileAttachment(\"brt_linework.png\")},\n {name:'Corine', value:FileAttachment(\"corine_linework.png\")}];\n return Inputs.radio(imgs, {value: imgs[0],label: \"Level Of Detail\", format: x => x.name});\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":174,"value":"Where you see the distinction between road, sidewalk and buildings in the BGT, this distinction is completely gone in the next level. The third level, Corine, is a story on itself since at that point it has no other value than 'city'.\n\nIt would be nice if we could create some intermediate data where we don't have to load every single asset to still show a nice map.\n\nIn the following part I am going to show you how we dealt with that.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":150,"value":"## Reconstructing the map\n\n### Reclassify\n\nFirst step is the reclassification of the data. \nThe most detailed dataset has *over* 80 classes, which is a bit much for showing on a base map. Chances are that you will only distract the viewer.\n\n

Over 80 different classes of landcover in the BGT

\n\nSo we brought this down to only 13 classes showing the most prominent features of the landscape.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":187,"value":"FileAttachment(\"image@8.png\").image()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":213,"value":"### merge\n\nOn top of that, we merged the features that would otherwise be visible as one big feature anyway. In postgis this is easy with a UNION.\n\n ST_Subdivide(\n ST_CollectionExtract(\n (ST_Dump(\n ST_Union(geom)\n )).geom\n ,3)\n )::Geometry(Polygon,28992)\n\nOn caveat here is that you may be creating *very* large and complex polygons, which are a liability. Therefore you can see the ST_Subdivide in the code and picture, which I'll come back to later.\n\n

\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":228,"value":"### simplify\n\nnow there is a separate story to be told on the simplifying of geometries in postgis. We could have used simply Simplify, SimplifyPreserveTopology - which will try to maintain topological integrity- or even Simplify with the Visvalingam-Whyatt algorithm which tends to give a bit smoother simplification. \n\n

Mysid, CC BY-SA 3.0 , via Wikimedia Commons

\n\nBut none of these will protect you from this:\n

","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":280,"value":"viewof simplifyimg = {\n const imgs = [\n {name:'Original',value:FileAttachment(\"image@12.png\")},\n {name:'Simplified', value:FileAttachment(\"image@11.png\")}];\n return Inputs.radio(imgs, {value: imgs[0],label: \"Simplify\", format: x => x.name})\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":285,"value":"An inherent damage being done to your network topology by all forms of simplification in Postgis. \nThat is why we made use of some dark-magic in the database, involving an algorithm that only exists in Javascript called topojson.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":297,"value":"$30","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":303,"value":"### subdivide\n\nNow to come back to the subdivide function.\n\nAs I said, when you have very large and complex polygons you might be giving an extra job to the vector-tiler because it needs to fetch the large polygon and do an intersectiontest for all its vertices…\nwhereas you maybe only need a small corner of that polygon. Especially waterbodies, of which we have a lot in the Netherlands, tend to become unreasonably large.\n\nTherefore we made sure to run an ST_Subdivide in advance in postgis on every polygon\n

\nAs you can see, polygons get smaller according to the number of vertices they have in them.\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":311,"value":"$31","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":321,"value":"## getting on with the deployment\n\nThis is out of my league but I have some colleagues who are good at this and they told me that deploying this on kubernetes is the logical thing to do. Here you can see the (relatively simple) helm-chart that does all the deployment (postgres running somewhere else).\n\n

\n\nTwo things to keep in mind:\n\n1. make sure you put MAX FEATURES PER TILE on -1, to avoid incidental gaps\n2. it needs relatively little memory, even less than this might be possible\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":199,"value":"viewof map1 = {\n const container = html`

`;\n let resolve;\n container.value = new Promise(_ => resolve = _);\n \n yield container; // Give the container dimensions.\n \n const map = new maplibre.Map({\n container: container,\n center: [5.16236624,52.29568567],\n zoom: 17,\n style: \"https://fileserv.beta.geodan.nl/mapbox/styles/basiskaart_style-dev-no-cache.json\",\n scrollZoom: false\n });\n resolve(map); // Expose the Map instance as the view’s value.\n \n invalidation.then(() => map.remove());\n}\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":488,"value":"### The effect of caching\nAlthough the postgis mvt creation is extremely fast, it doesn't hurt to add a bit of cache to your service because you never know how many people at once may be requesting a map. But just to show how fast the creation of mvt-tiles is you can see here in a side-by-side comparison. Left is cached, right is without cache. Note that on the higher zoomlevels (closer to surface) there is virtually no difference whereas on the lower zoomlevels you can see the latency of the service kicking in.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":390,"value":"html`

\n\n

${viewof map1}

${viewof map2}

\n \n

Cached

Realtime

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":346,"value":"viewof location = Inputs.select(locations, {label: \"FlyTo: \", format: x => x.name, value: locations[0]})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":509,"value":"### no need to stay flat\nThe maplibre viewer I am using here is running on webgl and has the capabilities to show objects in 3D as well. Now the advantage of vectortiles is that you can pass in a parameter that tells something about the height of the polygon and it will extrude the polygon for you so you get a semi (2.5D) 3D view of buildingheights.\n\nRead more about this example at: https://research.geodan.nl/all-dutch-3d-buildings-in-1-9gb/","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":514,"value":"$32","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":616,"value":"### the final goal, a baselayer for a 3D map\n\nThe ultimate goal of this basemap was to create a base-layer for a 3D map where the terrain stays flat but other parts of the landscape are added as 3D features, similar to a 3D scale model of a city where all features are displayed in a simplified version.\n\nThis worked out pretty well, as you can see in the demo below:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":527,"value":"viewof map3dtiles = {\n const container = html`

`;\n let resolve;\n container.value = new Promise(_ => resolve = _);\n \n yield container; // Give the container dimensions.\n \n let map = new maplibre.Map({\n container: container,\n style: \"https://fileserv.beta.geodan.nl/mapbox/styles/basiskaart_style-dev.json\",\n center: [4.87342800, 52.33881317], \n zoom: 14,\n bearing: 0,\n pitch: 0,\n hash: true,\n antialias: true,\n bearingSnap: false,\n interactive: true\n });\n\n \n\n resolve(map); // Expose the Map instance as the view’s value.\n invalidation.then(() => map.remove());\n \n} ","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":711,"value":"viewof flytochoice = {\n const imgs = [\n {name:'Original',value:FileAttachment(\"image@12.png\")},\n {name:'Simplified', value:FileAttachment(\"image@11.png\")}];\n return Inputs.radio(maquettelocations, {value: center,label: \"FlyTo\", format: x => x.name})\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":748,"value":"$33","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":753,"value":"$34","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":192,"value":"## Functionality\n\nBelow this line there is some code for the moving parts of this blog.\n\n---","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":647,"value":"layer = new Mapbox3DTiles.Mapbox3DTilesLayer(threedeeconfig)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":649,"value":"map3dtiles.on('style.load',()=>{\n \n map3dtiles.addLayer(layer);\n \n layer.scene.setHemisphereIntensity(0.75);\n layer.scene.setShadowOpacity(0.15);\n layer.scene.lights[1].position.set(85.95479335896457, -500.3727753754697, 861.5328543715947);\n\n const tileset = layer.implementation.tilesetManager.getTileset(\"GebouwenNL\");\n //tileset.setStyle(new Mapbox3DTiles.BuildingShadeStyle('shader', { shadeOffset: 0.0, colorA: 0x892121, colorB: 0xf7a10b}));\n tileset.setStyle({\n id: \"light-shade\",\n type: \"shade\",\n settings: {\n color: 0xffffff,\n opacity: 1,\n colorAttribute: 'id',\n }\n });\n \n});","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":656,"value":"//No idea why we need to re-run this for the building style\nsetTimeout(()=>{\n //tileset.setStyle(new Mapbox3DTiles.BuildingShadeStyle('shader', { shadeOffset: 0.0, colorA: 0x892121, colorB: 0xf7a10b}));\n map3dtiles.getLayer(\"maquette\").implementation.tilesetManager.getTileset(\"GebouwenNL\").setStyle({\n id: \"light-shade\",\n type: \"shade\",\n settings: {\n color: 0xffffff,\n opacity: 1,\n colorAttribute: 'id',\n }\n });\n},5000);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":337,"value":"locations = [\n {name: \"Loc. 1\", center: [5.15722344, 52.29512444]},\n {name: \"Loc. 2\", center: [4.90029047,52.37257180]},\n {name: \"Loc. 3\", center: [5.11439793, 52.20078723]},\n]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":713,"value":"map3dtiles.flyTo(flytochoice.params)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":709,"value":"maquettelocations = [\n {name: \"Naarden\", params: {\n bearing: 19,\n center: { lng: 5.161204, lat: 52.293684 },\n pitch: 41,\n zoom: 15.94,speed:0.3 }\n },\n {name: \"Windmolens\", params: { \n bearing: -10.9,\n center: { lng: 4.762223, lat: 52.856257 },\n pitch: 57,\n zoom: 16.08,speed:0.3\n } },\n {name: \"Leiden\", params: {\n bearing: 111.2,\n center: { lng: 4.4827238, lat: 52.1540149 },\n pitch: 46,\n zoom: 19.19,speed:0.3\n } }\n]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":525,"value":"$35","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":369,"value":"center = location.center","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":351,"value":"map1.flyTo({center,zoom: 17})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":463,"value":"map2.flyTo({center,zoom: 17})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":197,"value":"maplibre = await require(\"maplibre-gl@1.14.0\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":523,"value":"Mapbox3DTiles=require('@geodanresearch/mapbox-3dtiles@0.7.7/dist/Mapbox3DTiles_umd.js')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":195,"value":"html``\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W36","stdlib":"1","titling":"auto"}} 2e:"$2d:body"