2f:T46e,## Insights In order to get an overview of the classes used we can use Triply's [insights panel](https://lod.uba.uva.nl/UB-UVA/Beeldbank/insights/classFrequency?graph=https%3A%2F%2Flod.uba.uva.nl%2FUB-UVA%2FBeeldbank%2Fgraphs%2Fdefault) for the beeldbank. We can see that all maps contain the following classes: date, language, extent, label, type, description, identifier, title, medium, source and isFormatOf. Some of the maps additionally have these classes: publisher, spatial, subject, hasBeginTimeStamp, contributor, temporal, isReferencedBy, relation, alternative, creator, hasEarliestBeginTimeStamp, hasLatestEndTimeStamp, hasEndTimeStamp, requires, tableOfContents, issued, isPartOf and abstract. _Note that isShownBy and depiction are both missing from this list. Does this (falsely) presume all map have images...? Are these not seen as classes?_ For the SELECT query this means that the second list of classes needs to be set as OPTIONAL under WHERE: ``` WHERE { ?id a schema:Map ; rdfs:label ?label ; ... OPTIONAL { ?id dc:subject ?subject . } OPTIONAL { ?id dc:contributor ?contributor . } ... } ```30:T764,thirdSelectQuery = { const query = ` PREFIX dc: PREFIX dct: PREFIX edm: PREFIX rdf: PREFIX rdfs: PREFIX schema: SELECT ?id ?label ?title ?extent ?isFormatOf ?source ?creator (GROUP_CONCAT(DISTINCT ?alternative; separator="$") AS ?alternative) (GROUP_CONCAT(DISTINCT ?relation; separator="$") AS ?relation) (GROUP_CONCAT(DISTINCT ?isReferencedBy; separator="$") AS ?isReferencedBy) (GROUP_CONCAT(DISTINCT ?subject; separator="$") AS ?subject) (GROUP_CONCAT(DISTINCT ?contributor; separator="$") AS ?contributor) (GROUP_CONCAT(DISTINCT ?spatial; separator="$") AS ?spatial) (GROUP_CONCAT(DISTINCT ?language; separator="$") AS ?language) (GROUP_CONCAT(DISTINCT ?image; separator="$") AS ?image) (GROUP_CONCAT(DISTINCT ?description; separator="$") AS ?description) (GROUP_CONCAT(DISTINCT ?publisher; separator="$") AS ?publisher) (GROUP_CONCAT(DISTINCT ?date; separator="$") AS ?date) WHERE { ?id a schema:Map ; rdfs:label ?label ; dc:date ?date ; dc:language ?language ; dct:extent ?extent ; dc:title ?title ; dc:description ?description ; dct:isFormatOf ?isFormatOf ; dc:source ?source . OPTIONAL { ?id dc:subject ?subject . } OPTIONAL { ?id dc:contributor ?contributor . } OPTIONAL { ?id dc:creator ?creator . } OPTIONAL { ?id dc:publisher ?publisher . } OPTIONAL { ?id edm:isShownBy ?image . } OPTIONAL { ?id dct:spatial ?spatial . } OPTIONAL { ?id dct:isReferencedBy ?isReferencedBy . } OPTIONAL { ?id dc:relation ?relation . } OPTIONAL { ?id dct:alternative ?alternative . } } LIMIT 1000 `; const result = fetchJson(apiUrl + "?query=" + encodeURIComponent(query)); return result; }31:[] 2d:{"title":"Georeferencing UvA Maps with Allmaps","cells":[{"id":0,"value":"# Georeferencing UvA Maps with Allmaps\n\nThe library of the University of Amsterdam uses [Triply](https://lod.uba.uva.nl/UB-UVA/-/overview) to expose their collections as Linked Open Data. Leon van Wissen ([Amsterdam Time Machine](https://www.amsterdamtimemachine.nl/)) kindly made me aware of the available maps _and_ IIIF Images in the [digital image repository](https://lod.uba.uva.nl/UB-UVA/Beeldbank) graph. This means that [Allmaps](https://allmaps.org/) can be used to georeference these maps.\n\nIn this Notebook, my goal is to eventually create IIIF Manifests of the available maps using the SPARQL endpoint. (I haven't used SPARQL before so Leon has been helping me along the way). This investigation relates to a forthcoming FAQ on IIIF and LOD that I'm making together with Bob Coret of the Netwerk Digitaal Erfgoed.\n\nYou can now already search for a map below and open it in the Allmaps Editor.","pinned":false,"hidden":false,"mode":"md"},{"id":657,"value":"viewof search = Inputs.search(cleanLabels)","pinned":false,"hidden":false,"mode":"ojs"},{"id":661,"value":"viewof selectedHandle = Inputs.table(search, {\n value: search[0],\n multiple: false,\n columns: [\"label\", \"uri\"],\n header: {\n label: \"Label\",\n uri: \"Handle\"\n }\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":334,"value":"_Preview of selected map:_\n\n![${selectedHandle.label}](${infoJson.split(\"info.json\")[0] + \"full/\" + Math.round(width / 2) + \",/0/default.jpg\"})\n\n_[${selectedHandle.title.replaceAll(/[\\[\\]]/g, \"\")}](${selectedHandle.uri})_\n\n[Open in Allmaps Editor](https://editor.allmaps.org/#/collection?url=${infoJson})","pinned":false,"hidden":false,"mode":"md"},{"id":707,"value":"## Implementation","pinned":false,"hidden":false,"mode":"md"},{"id":724,"value":"SPARQL Endpoint:","pinned":false,"hidden":false,"mode":"md"},{"id":9,"value":"apiUrl = (\"https://api.lod.uba.uva.nl/datasets/UB-UVA/Beeldbank/services/virtuoso/sparql\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":251,"value":"A simple query to lists all the maps in the beeldbank graph containing images, together with the label and full title:","pinned":false,"hidden":false,"mode":"md"},{"id":730,"value":"firstSelectQuery = `PREFIX schema: \nPREFIX rdfs: \nPREFIX edm: \nPREFIX dc: \nSELECT * WHERE {\n?uri a schema:Map ;\nrdfs:label ?label ;\ndc:title ?title ;\nedm:isShownBy ?image .\n}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":739,"value":"This returns a list of ${firstSelectQueryResult.length} objects:","pinned":false,"hidden":false,"mode":"md"},{"id":244,"value":"firstSelectQueryResult = fetchJson(\n apiUrl + \"?query=\" + encodeURIComponent(firstSelectQuery)\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":743,"value":"Cleaning up the labels a bit:","pinned":false,"hidden":false,"mode":"md"},{"id":585,"value":"cleanLabels = firstSelectQueryResult.map((item) => ({\n label: item.label.replace(/ *[/.:;=]$/, \"\").replaceAll(/[\\[\\]]/g, \"\"),\n uri: item.uri,\n title: item.title,\n iiif: item.image\n}))","pinned":false,"hidden":false,"mode":"ojs"},{"id":745,"value":"Constructing the IIIF info.json url:","pinned":false,"hidden":false,"mode":"md"},{"id":709,"value":"infoJson = selectedHandle.iiif.split(\"full/\")[0] + \"info.json\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":747,"value":"## Fetching metadata","pinned":false,"hidden":false,"mode":"md"},{"id":259,"value":"The following query can be used to fetch more information about this map:","pinned":false,"hidden":false,"mode":"md"},{"id":753,"value":"secondSelectQuery = `SELECT * WHERE {\n<${selectedHandle.uri}> ?pred ?obj .\n}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":757,"value":"This returns an array with all the predicates and objects related to this particular map (or _subject_ in RDF terminology):","pinned":false,"hidden":false,"mode":"md"},{"id":266,"value":"secondSelectQueryResult = fetchJson(\n apiUrl + \"?query=\" + encodeURIComponent(secondSelectQuery)\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":276,"value":"(The array also includes the IIIF Image, which is related to the subject with both the predicate `isShownBy` (of the [Europeana Data Model](https://pro.europeana.eu/page/edm-documentation)) and the predicate `depiction` of the [FOAF vocabulary](http://xmlns.com/foaf/0.1/))","pinned":false,"hidden":false,"mode":"md"},{"id":281,"value":"secondSelectQueryImages = secondSelectQueryResult.filter(\n (item) => item.pred.includes(\"isShownBy\") || item.pred.includes(\"depiction\")\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":357,"value":"In order to include the results of the query above in a IIIF Manifest, we need to convert the predicates and objects into simple label and value pairs. IIIF does not support structured metadata as part of the `metadata` property, as can be read in the introduction to the [Presentation API](https://iiif.io/api/presentation/3.0/#11-objectives-and-scope):\n\n> _\\[Presenting digital objects to a human user] is the sole purpose of the API and therefore descriptive information is given in a way that is intended for humans to read, but not semantically available to machines._","pinned":false,"hidden":false,"mode":"md"},{"id":771,"value":"The SELECT query above returns results as a list. It's also possible to request JSON-LD with a CONSTRUCT query:","pinned":false,"hidden":false,"mode":"md"},{"id":774,"value":"firstConstructQuery = `\nCONSTRUCT WHERE {\n<${selectedHandle.uri}> ?pred ?obj ;\n}`","pinned":false,"hidden":false,"mode":"ojs"},{"id":787,"value":" This returns the results as triples:","pinned":false,"hidden":false,"mode":"md"},{"id":776,"value":"firstConstructQueryResult = fetchJsonWithOptions(apiUrl, firstConstructQuery)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1011,"value":"The following SELECT query provides an overview of all the possible predicates:","pinned":false,"hidden":false,"mode":"md"},{"id":952,"value":"insightQuery = `PREFIX schema: \nPREFIX rdf: \nPREFIX rdfs: \nSELECT DISTINCT ?pred WHERE {\n ?map a schema:Map ;\n ?pred ?obj .\n \n FILTER(?pred != rdf:type)\n \n} LIMIT 100`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1013,"value":"From this we can create a context file that can be used to compact the JSON-LD response of the CONSTRUCT query:","pinned":false,"hidden":false,"mode":"md"},{"id":955,"value":"insightQueryResult = fetchJson(\n apiUrl + \"?query=\" + encodeURIComponent(insightQuery)\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":959,"value":"contextFromInsightQuery = insightQueryResult.map((item) => ({\n [item.pred.match(/\\w*$/)[0]]: item.pred\n}))","pinned":false,"hidden":false,"mode":"ojs"},{"id":942,"value":"The [jsonld.js](https://github.com/digitalbazaar/jsonld.js/) can be used to process the response:","pinned":false,"hidden":false,"mode":"md"},{"id":933,"value":"compactedJsonLd = await jsonld.compact(\n firstConstructQueryResult,\n contextFromInsightQuery\n)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1045,"value":"Which can then be converted to key, value pairs required by the [IIIF Presentation API](https://iiif.io/api/presentation/3.0/#metadata):","pinned":false,"hidden":false,"mode":"md"},{"id":1015,"value":"iiifMetadata = {\n let array = [];\n for (const property in compactedJsonLd) {\n if (\n !property.includes(\"@\") &&\n !property.includes(\"depiction\") &&\n !property.includes(\"isShownBy\") &&\n !property.includes(\"label\") &&\n !property.includes(\"hasEndTimeStamp\") &&\n !property.includes(\"hasLatestEndTimeStamp\") &&\n !property.includes(\"hasBeginTimeStamp\") &&\n !property.includes(\"hasEarliestBeginTimeStamp\")\n ) {\n const value = compactedJsonLd[property];\n const valueArray = Array.isArray(value) ? value : [value];\n const propertyArray = [\n property.charAt(0).toUpperCase() + property.slice(1)\n ];\n array.push({\n label: { nl: propertyArray },\n value: { nl: valueArray }\n });\n }\n }\n return array;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":389,"value":"$2f","pinned":false,"hidden":false,"mode":"md"},{"id":409,"value":"The subsequent problem is that a single handle can have repeating classes, such as multiple contributors. The results can be combined in a single row in a SELECT query by using GROUP_CONCAT (in this case I'm using a dollar sign as a separator to be able to split them later with Javascript):\n\n```sparql\nSELECT\n\n ?id\n ?label\n ...\n\n (GROUP_CONCAT(DISTINCT ?subject; separator=\"$\") AS ?subject)\n (GROUP_CONCAT(DISTINCT ?contributor; separator=\"$\") AS ?contributor)\n ...\n```\n\n_Note: is there an easy way to know which classes repeat and which not? This is lacking from the insights._","pinned":false,"hidden":false,"mode":"md"},{"id":420,"value":"A more complex SELECT query for all the maps:","pinned":false,"hidden":false,"mode":"md"},{"id":422,"value":"$30","pinned":false,"hidden":false,"mode":"ojs"},{"id":432,"value":"Surprisingly, it seems that ${mapsWithoutImages.length} maps have no IIIF images:\n\n(I used `edm:isShownBy` for the query but also tried `foaf:depiction` to make sure)","pinned":false,"hidden":false,"mode":"md"},{"id":76,"value":"mapsWithoutImages = thirdSelectQuery.filter((map) => !map.image)","pinned":false,"hidden":false,"mode":"ojs"},{"id":448,"value":"(We can now also create an array of the ${infoJsons.length} IIIF Images that are available:)","pinned":false,"hidden":false,"mode":"md"},{"id":158,"value":"infoJsons = thirdSelectQuery\n .filter((map) => map.image)\n .map((map) => map.image.match(/(.*?)\\/full\\/full.*/)[1] + \"/info.json\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":471,"value":"Some of the objects contain URIs that have to be resolved in order to use as 'human readable' metadata. The list of contributors for example contains names, but also URIs from [data.bibliotheken.nl](https://data.bibliotheken.nl):","pinned":false,"hidden":false,"mode":"md"},{"id":542,"value":"contributors = thirdSelectQuery\n .filter((item) => item.contributor)\n .map((item) => item.contributor.split(\"$\"))\n .flat()","pinned":false,"hidden":false,"mode":"ojs"},{"id":552,"value":"Is it necessary to resolve all these URIs in order to get the labels? Looking at the Triply interface, it seems that the beeldbank graph already includes those labels.\n\nObjects with URIs:\n- Contributors, `foaf:name`\n- Creator, `foaf:name`\n- Subject, `skos:prefLabel`\n- Spatial, `skos:prefLabel` (alleen voor data.bibliotheken.nl)\n- Language, no label\n\nTodo:\n- [Images to Manifest](https://observablehq.com/@allmaps/manifest-from-iiif-images)\n- Requesting the labels (example SELECT query from Leon: https://api.lod.uba.uva.nl/s/P4giq6oKW)\n- Another example: https://lod.uba.uva.nl/UB-UVA/-/queries/Maps-creators/3\n- Cleaning up the JSON-LD response with [jsonld.js](https://github.com/digitalbazaar/jsonld.js/)\n- Other libraries: https://rdf.js.org\n- Check https://grlc.io","pinned":false,"hidden":false,"mode":"md"},{"id":509,"value":"## Evaluation\n\nThe paradox of is SPARQL is that the more you know about the data, the better you are at querying it. This means you need some insights before you can 'craft' the right query. Luckily the interface offered by Triply offers some of these insights.\n\nData observations:\n\n- Not all maps in the beeldbank include images\n- Some maps have multiple images but they're not in the graph, eg:\n - https://hdl.handle.net/11245/3.27450\n - https://hdl.handle.net/11245/3.27214\n - https://hdl.handle.net/11245/3.20465\n - https://hdl.handle.net/11245/3.3706\n- This handle does not resolve: https://hdl.handle.net/11245/3.27450\n- There's another copy of the atlas on the website which is not in the graph: https://hdl.handle.net/11245/3.1537","pinned":false,"hidden":false,"mode":"md"},{"id":505,"value":"## Alternative approaches\n\nIt's possible to construct a manifest with SPARQL:\n- Triply\n - Query: https://triplydb.com/Triply/-/queries/iris-iiif-manifest/56\n - Manifest: https://api.triplydb.com/queries/Triply/iris-iiif-manifest/run\n- RKD:\n - Query: https://rkd.triply.cc/rkd/-/queries/images-iiif-manifest/68\n - Manifest: https://api.rkd.triply.cc/queries/rkd/images-iiif-manifest/run?\n\nAnd `ImageObject` offers a better way to refer to a IIIF endpoint:\n- IISG: https://druid.datalegend.net/andb/ANDB-ADB-all/browser?resource=https%3A%2F%2Fiisg.amsterdam%2Fid%2Fandb%2F5407","pinned":false,"hidden":false,"mode":"md"},{"id":30,"value":"## Notes\n\nLeon's links:\n- https://lod.uba.uva.nl/CREATE/-/stories/Cartographers-ECARTICO\n- https://lod.uba.uva.nl/CREATE/-/stories/STCN--ECARTICO-Janssonius-cs\n- https://journal.dhbenelux.org/journal/issues/003/article-36-Nijboer.pdf\n- http://www.learningsparql.com/\n- https://www.w3.org/TR/rdf-sparql-query/\n- https://uvacreate.gitlab.io/amsterdam-time-machine/datasprint-lod-uba-2022/index.html","pinned":false,"hidden":false,"mode":"md"},{"id":443,"value":"## Functions","pinned":false,"hidden":false,"mode":"md"},{"id":7,"value":"fetchJson = (url) => fetch(url).then((response) => response.json())","pinned":false,"hidden":false,"mode":"ojs"},{"id":177,"value":"fetchJsonWithOptions = (url, query) =>\n fetch(url, {\n method: \"POST\",\n headers: {\n Accept: \"application/ld+json\", // application/sparql-results+json\n \"Content-Type\": \"application/x-www-form-urlencoded\"\n },\n body: `query=${encodeURIComponent(query)}`\n }).then((response) => response.json())","pinned":false,"hidden":false,"mode":"ojs"},{"id":910,"value":"## Imports","pinned":false,"hidden":false,"mode":"md"},{"id":912,"value":"jsonld = require(\"jsonld@8.1.1/dist/jsonld.min.js\")","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W31","stdlib":"1","titling":"auto"}