It uses a new feature of Sparnatural : the ability to act as a single UI facade to multiple SPARQL endpoints. The user visually writes a single query, the query is sent to multiple data sources, and results are aggregated to be presented to the user in a single result set, that includes each result provenance.

This is depicted in the diagram below:

The user writes his query visually in Sparnatural, here « All archaelogical sites where burials have been found, with the name of their discoverer, if known » :

The visual query is translated into SPARQL, and that SPARQL query is sent to each data source in the federation (actually, the user can select the ones he wants to query). This is depicted by the green arrows in the diagram, numbered « 1 ».

The SPARQL query looks like the following; note how it uses complex CIDOC-CRM property paths, such as the highlighted one, while the visual user query was simple:

Each SPARQL service returns a result. This is depicted by the orange arrows in the diagram, numbered « 2 ».

When every endpoint have answered, their results are aggregated into a single result set. This is number « 3 » in the diagram. During this aggregation, an extra column is added in the result set, containing the name of the source from which the result was retrieved.

The user sees the aggregated result; here, the name of the site, the name of its discovered when known, and the source in which the result was found:

This is possible thanks to the catalog configuration of Sparnatural.

Sparnatural can be passed a catalog of SPARQL endpoints in a federation, and in this case, it will send the same SPARQL query to each, and will aggregate the results. This happens for the final query of course, but also during selection of values in the query UI.

There are two main limits of this approach:

• Limit 1 : all sources in the federation must share the same data model, as the same query is sent to every source
• Limit 2 : each source must be independant : there should be no links from one source to another source so that the query can be solved by each endpoint independantly (so actually, no truly distributed linked data)

Those are the reasons I have entitled the post « simple federation facade ». Those 2 hypothesis are met in OpenArchaeo, and they were also met in the case of the (never released) prototype of the Europeana Linked Data taskforce. If you know other cases of data federation in which this is also true, tell us ! (we could actually try the same on a few DBPedia endpoints using the dbo ontology as a pivot model)

Now guess what ? in Sparnatural we have a « query UI to SPARQL » transformation step, thanks to the SHACL configuration of Sparnatural. Basically we can map a UI property on an underlying property path. Then it would not be too difficult to do this mapping on a source-by-source basis, to have different queries sent to each source, from a single query in the UI. The result set structure would be the same, and result set aggregation can still happen. We would then overcome the first limit described above. That’s the next step !

Cet article Sparnatural as a simple data federation facade est apparu en premier sur Sparna Blog.

Paired with the CIDOC-CRM in a federated way, OpenArchaeo aims at :

• making available the archaeological datasets produced by the MASAplus consortium’s partners on the semantic web, in the form of a triplestore with data aligned with the ontology and its extensions dedicated to archaeology ;
• providing an intuitive query interface for archaeological data.

The latter query interface integrates the Sparnatural knowledge graph exploration component. The UI of this component was heavily inspired by the British Museum’s ResearchSpace semantic search feature, as the system proposes the user to build his own queries based on the CIDOC-CRM model underlying the data.

About ResearchSpace platform

Initiated in 2009 by a cross-disciplinary team at the British Museum, ResearchSpace is « A full CIDOC-CRM authoring and search system, based on an exhaustive collection of forms that reflects all applicable relationships from the CIDOC CRM ontology. »

Among a wide range of semantic tools to create, manipulate, analyse and visualise data, the platform provides a semantic structured search component based on categories and relations.

While open source, ResearchSpace’s code didn’t fit our architecture : we just chose to follow the simple visual elements of ResearchSpace’s query interface to develop our own Sparnatural query builder for OpenArchaeo, and set up a system of icons to identify the main components of the archaeological data.

ResearchSpace has recently (december 2024) released a brand new 4.0.0 version. This latest can be installed easily and now comes with a default setup of forms based on the CIDOC-CRM. It enables image annotations, knowledge maps creations, semantic narratives writing, timeline productions, and more semantic tools.

Sparnatural’s first use-case was OpenArchaeo’s CIDOC-CRM model !

The structure of the knowledge graph of OpenArchaeo relies on the CIDOC-CRM and some of its extension (CRMarchaeo, CRMsci and CRMba). It is a generic model that covers the basic concepts found in most archaeological corpuses (site, operation, structure, feature, wall, burial, stratigraphic unit and artifact).

Here a focus on Class S19 :

Several external thesauri were added too for querying the datasets : PACTOLS thesaurus for archaelogy, but also Geonames and Periodo for spatial and temporal searches.

This way, when users wish to connect two elements (artifact and site for example), the interface automatically suggests the available relationships between these entities, enabling users to formulate their request in a simple way without having to know either the entities and properties of CIDOC CRM, or the structure of the system : the SPARQL queries that correspond to the sentences visually built by users will be automatically computed. In addition, the usage of thesauri allows the users to cross-reference easily multiple datasets through the different widgets proposed in Sparnatural.

Get the latest release of Sparnatural !

Since it was created for OpenArchaeo in 2019, Sparnatural UI has been fully redesigned. It now offers a large panel of features, from different widgets for value selection (dropdown lists, ordered by occurrence count or alphabetically, autocomplete search fields, date pickers, tree widgets…) to brand new result display plugins : the default visualisation is a table of results, but if the results are geolocalized they can be shown in a map. Also grid, stats, pie or bar charts, and a timeline plugin have been made available and documented.

To go further on OpenArchaeo’s platform …

See a presentation of the project on the CIDOC Museum Documentation Channel
(« Semantic modelling of archaeological data online workshop series »)

The platform : http://openarchaeo.huma-num.fr/

The project : https://masa.hypotheses.org/openarchaeo

Read full research paper about the project : https://ceur-ws.org/Vol-2375/paper1.pdf

Image : Vestiges of a large villa in Courbehaye « les Deux Muids / le Moulin de Mongé », photo Alain Lelong (2003), Atlas des Établissements Ruraux de Beauce Antique, licence CC BY-NC-SA

Cet article The Genesis of Sparnatural in the context of the OpenArchaeo platform est apparu en premier sur Sparna Blog.

Voilà maintenant près d’un an et demi que FranceArchives, le Portail national des Archives de France, a annoncé le déploiement de l’outil « Supernatural » (comprendre Sparnatural) via ses réseaux, dans l’optique de proposer à ses usagers « un accès nouveau aux métadonnées archivistiques, complémentaire de la recherche classique par le moteur du portail ».

Porté par le Service interministériel des Archives de France, le portail FranceArchives offre une recherche fédérée dans près de 26 millions de métadonnées archivistiques produites par près de 170 institutions et entièrement sémantisées en RDF par le biais de l’ontologie RiC-O version 0.2 publiée en février 2021.

C’est une des premières utilisations de RiC-O à grande échelle (même s’il faudra à l’avenir qu’il se mette à jour sur la version 1.0 de RiC-O publiée depuis !), et c’est également l’un des premiers entrepôts de données archivistiques de cette taille sur le Linked Open Data.

… Un graphe de données qui a tout pour être « Supernaturalisé »

Des données de qualité à une recherche augmentée

… Enfin, il va surtout s’agir de ses « données de qualité », autrement dit les :

• inventaires avec leurs composants,
• notices descriptives de producteurs d’archives,
• fiches signalétiques des services d’archives,

 … tous objets liés à une autorité « personnes et institutions », « lieux » et « thèmes » de qualité (soit moins de 5% des métadonnées du portail avant conversion… et plus de 70% de l’ensemble du réservoir en RDF !), autorités de qualité elles-mêmes harmonisées et alignées vers des référentiels nationaux et internationaux.

Modèle particulièrement adapté à la description des archives en RDF, c’est l’ontologie RiC-O (v0.2) qui a été utilisée pour la sémantisation des données XML EAD – XML EAC-CPF vers RDF, complétée de schema.org pour les fiches signalétiques des services de l’annuaire au format XML EAG.

Les informations relatives aux archives et à leurs producteurs étant décrites dans des fichiers différents, la recherche avancée via SPARQL rend désormais possible une interrogation fédérée plus fine d’un vaste corpus de notices en « traversant » le graphe structuré selon le modèle RiC-O. En effet, l’intérêt de l’interrogation via SPARQL est de casser les silos entre types de métadonnées : il permet de faire une recherche transversale entre données provenant de fichiers EAD et de fichier EAC-CPF.

Les notices affichées en résultats de recherche montrent les alignements existants vers les notices de producteurs externes, Wikidata, data.bnf, GeoNames ou encore le Thesaurus pour l’indexation matières des archives locales. C’est ainsi dans l’onglet Personnes/indexations liées que sont exploités les résultats de la conversion en RDF, par le biais de suggestions de recherches complémentaires sur le portail classique.

Une façon de faire bénéficier le grand public du RDF de manière complètement transparente pour lui !

Quelques exemples de requêtes…

On accède à l’outil via le menu « Recherche SPARQL » en haut à droite du site du portail :

Plusieurs exemples de requêtes sont à disposition pour explorer les données :

De la requête la plus simple :

Personne est membre de Institution

À des requêtes de plus en plus élaborées et complexes, comme ici :

Lieux qui sont le sujet des archives reliées au fonds « Fabrique de berlingot Eysséric »

Où l’on voit que l’on peut retracer le cheminement de la requête à travers le graphe de l’ontologie RiC-O en cliquant sur « Afficher/masquer l’éditeur SPARQL ».

Des archives à la page…

À noter que le projet, qui avait fait l’objet d’une présentation à l’occasion de SWIB (Semantic Web in Libraries) et de SemWebPro 2023 a été entièrement déployé (et configuré !) à partir de la documentation disponible sur le site web de Sparnatural.

N’hésitez pas à aller la consulter !

Hello Sparnatural

How-to configure in SHACL

Reference documentation of Sparnatural widgets

Pour aller plus loin sur la sémantisation des archives…

Le déploiement de Sparnatural sur FranceArchives fait suite à une autre réalisation de l’année précédente, le démonstrateur Sparnatural des Archives nationales. Celui-ci avait permis de faire évoluer Sparnatural et de le déployer sur un graphe sémantique en RiC-O de 20 millions de triplets (hors inférence), alimenté avec le contenu de 1577 instruments de recherche décrivant les archives de 40 des 122 études notariales de Paris conservées aux Archives nationales, de 1120 notices décrivant ces études et les notaires qui y ont exercé, et d’autres référentiels des Archives nationales notamment sur les lieux de Paris. La réalisation de ce démonstrateur a été entièrement documentée en français et en anglais. Ce démonstrateur et ses interfaces évolueront d’ailleurs bientôt.

Depuis, Sparna s’est impliqué dans le domaine de la sémantisation des archives puisque nous développons également, pour les comptes des Archives Nationales, l’outil Ric-O converter.

Celui-ci permet la conversion de notices EAD et EAC vers du RDF exprimé en RiC-O. Nous finalisons actuellement une nouvelle version du convertisseur pour le rendre compatible RiC-O 1.0 (et même 1.1 dont la sortie est imminente).

Un nouvel article à paraître ici sur RiC-O ? … Stay tuned !

Cet article Retour sur … Le déploiement de Sparnatural pour FranceArchives est apparu en premier sur Sparna Blog.

“

The Sparnatural knowledge graph explorer leverages SHACL specifications to drive a user interface (UI) that allows end users to easily discover the content of an RDF graph. What is the best way to make this UI-oriented SHACL specification ? if a SHACL specification for the knowledge graph structure already exists, can it be used directly ? does it require customization ? or is the Sparnatural SHACL spec completely decoupled from an existing knowledge graph spec ? and what if no SHACL spec exists at all ?

We faced all these different situations while deploying Sparnatural, and used various approaches to produce a satisfying end-user oriented specification. In particular, the Nakala repository is one of the latest graph for which Sparnatural was deployed. Nakala is a data repository that aims to preserve and disseminate data produced by French research projects in the Humanities and Social Sciences, in compliance with the FAIR principles. Nakala is a service offered by Huma-Num, a research infrastructure dedicated to the digital humanities. The Nakala knowledge graph contains `dcterms` metadata provided by researchers to describe the resources they upload. Additional non-dcterms metadata can also be provided. The metadata varies in quality and quantity depending on the researcher. When exposed in a SPARQL endpoint, resources, collections of resources and agents are described using the Europeana Data Model (EDM).

As the EDM dissemination channel for Nakala was new, no SHACL specification existed for it. We could have designed one for Sparnatural from scratch, but the choice was make to generate it automatically, with no human intervention. This was for three reasons : ease of configuration, flexibility in maintenance over time, and pedagogical reason, as it was important to explain the structure of the graph to target users.

Sparnatural UI

Let’s first have a look at what the Sparnatural UI looks like on an example from Nakala:

Once you know that « ProvidedCHO » stands for « Provided Cultural Heritage Object », and that « asWKT » encodes the location of a Place, you will be able to understand that the query searches for all ProvidedCHO entries gathered into a certain collection (« Cartes Université Bordeaux Montaigne » – a collection of maps), and selects their location and an optional description (and yes, the results of this query are displayed on a map, but that’s out of scope).

SHACL is derived automatically

In this project we wanted the shortest path from the graph to the query UI. Hence we used a SHACL generation algorithm, available in SHACL Play. By issuing SPARQL queries on an RDF graph, the algorithm determines the NodeShapes (targeting the classes used as values of `rdf:type`), and PropertyShapes (from all predicates used on instances on each class) of the model, with their node kinds, datatypes, class range, and cardinalities. It generates `sh:or` constraints when multiple datatypes or ranges are found. Note that in the case of Nakala a large variety of ranges are used, since the data comes from very open user inputs : the same `dcterms` property can be either an IRI or a Literal, with varying datatypes.

In addition, the algorithm computes some statistics on the dataset : the number of targets of each NodeShapes, the number of occurrences and the number of distinct values for each property shapes. The statistics are expressed using the `void` vocabulary, and `dcterms:conformsTo` is used to link void partitions to the corresponding shapes.

The SHACL Play documentation tool was then used to generate a report of the generated SHACL combined with the statistics. A few errors were spotted in the exported data, and fixed. We also saw that around 70 properties were present only a few times out of 700.000+ ProvidedCHO records. These properties were applied by probably a single or very few researchers when describing their data. It was decided to filter them out to keep the final UI simple, with an extra filtering step : based on statistics, property shapes used less than 0.1% of the number of targets of their node shapes are removed.

Here is a screenshot of the report : the right column shows the number of distinct values, and the column before is the number of total occurrences; we can immediately see that `dct:isReplacedBy` occurs only once, and `dct:isRequiredBy` occurs 81 times. They will be filtered out.

Sparnatural reads SHACL

Sparnatural can then read the SPARQL specification, together with the dataset statistics. When designing a query, value selection widgets for literal properties are determined by looking at the `sh:datatype` constraint (for number, dates, boolean, or map widgets). For IRI properties, statistics are used to distinguish between list and autocomplete widgets. Predicates with less than 500 distinct values will use a dropdown list, and those with more will use an autocomplete search field. The range is determined by reading `sh:class` or `sh:node`. The label to show in dropdown lists or to search on autocomplete field is determined by looking at a `dash:propertyRole = dash:LabelRole` annotation.

How about labels ? Sparnatural can read them from classes and properties of the original OWL file, if provided with it. Otherwise local names of target classes or predicates are used.

Other configuration techniques

Other Sparnatural deployments, such as the DBPedia demo are designed in SHACL from scratch, using spreadsheets. This requires more manual work, but has the advantage of tailoring the UI to exactly what needs to be shown, including user-oriented labels/tooltips/icons, hiding some properties, taking shortcuts or declaring inverses using property paths, etc. In the case of DBPedia, no SHACL spec exists, and deriving it automatically for the entire graph would probably not make a lot of sense, hence the necessity for a manual design.

For other projects we are working on a third configuration technique : a SHACL spec that describes the exact content of the graph is first built. It is used to publish the documentation of the model and to validate the data. A separate shapes file containing a Sparnatural-specific configuration layer is then added on top of it. That layer can hide shapes by applying an `sh:deactivated` annotation on them, can specify the UI widgets to use, add additional `dash:LabelRole` flags, add shortcut or inverse properties, etc.

The 3 configuration paths are shown in the following diagram:

Your query UI in minutes

We combined 4 tools (all open-source) : an algorithm to generate a « profile » in SHACL of an RDF dataset, a statistical report generator, a SHACL filter based on statistics, and the Sparnatural query UI. The ability to generate the SHACL profile and review it in the report provided a way to understand the structure of the data in a matter of minutes, while hours would have been necessary with SPARQL queries, without a guarantee of completeness. The provision of the query UI was made by dropping the SHACL file and the statistics to Sparnatural, without manual intervention. This shows the pivotal role of SHACL for data quality and model-driven approaches for knowledge graphs projects.

”

We look forward to reading Veronika’s book, and you ?

Cet article Nakala : from an RDF dataset to a query UI in minutes – SHACL automated generation and Sparnatural est apparu en premier sur Sparna Blog.

The possible entities, connections and properties that are shown to the user need to be specified to Sparnatural. This configuration is written in a SHACL specification. This specification encodes both the structure of the knowledge graph (as we want it to be presented to the user), plus some additionnal UI-oriented information, like icons, order of entries, or value selection widgets to use.

How can the SHACL configuration of Sparnatural be produced ? we faced 3 different situations : either we do it manually, or (semi-)automatically, or with an off-the-shelf specification complemented with manual annotations. We will give you a brief description of each possibility below.

Write your SHACL manually (in Excel)

SHACL has one main disadvantage : there is no widely available, free-to-use SHACL editor to create/edit these specs (while Protégé, for example, allows anyone to edit an OWL ontology). To overcome this lack of tool, we are using an Excel-to-SHACL conversion tool, xls2rdf. We (and our users) write the specification in an Excel template, that is then turned in SHACL by an API call to xls2rdf. This is what we described in a previous post.

We have a good documentation on how to design your Sparnatural documentation in Excel, if you want to try it (but see the Hello Sparnatural tutorial first to setup your working test page). And we are here to help if you need to ! (ask questions on the Github repository).

If you are looking for a general-purpose SHACL Excel template, there is one in SHACL Play.

Get your SHACL automatically (statistics included)

SHACL is machine-readable and also writable, so we can use our SHACL generation algorithm to produce a SHACL « profile » of an RDF dataset. The algorithm sends SPARQL queries to identify all classes and properties, with their range, cardinalities, datatype, etc. This SHACL profile can be fed to Sparnatural directly. With an additionnal bonus : it is also very useful to gather statistics of the dataset at this stage. Number of instances of each class, number of occurrences of each property and number of distinct values. Why is it useful ? first because Sparnatural is able to show them in the UI, giving a hint to the user on how many entities of each type exist in the dataset:

second because by knowing how many distinct values exist for a given property, Sparnatural can show either a dropdown list (if there are less than 500) or an autocomplete search field (if there are more).

Also, as a side effect, seeing the profile and statistics of your dataset can help you spot errors (« why does the statistics tells me that this identifier is not always present ? it should be mandatory ! »).

You can also provide the OWL ontology to Sparnatural along with the SHACL specification. Simply pass the 2 files in the configuration of the component. The ontology can be leveraged for 2 things : the hierarchy of classes and properties from the ontology can be used in the Sparnatural UI, and all labels and comments of classes and properties can be read to be displayed in the UI, giving nice defaults if they are not present in the SHACL spec.
This automated SHACL generation was the one used for the Nakala Sparnatural query interface, and here is how the properties look like, you can see it is organized according to the DublinCore properties hierarchy:

Grab an off-the-shelf SHACL specification

Some clients do write and publish SHACL specifications. Yes. This is the case for example for the European Parliament in their open-data portal. They publish the documentation of their data-model with our SHACL Play documentation generator. The specification existed long before we tried Sparnatural on their data. We simply took their specification, loaded it in Sparnatural and… voilà ! it worked seamlessly, but to make it nicer we added in a separate file additionnal icons and tooltips information.

SHACL Synergies

There is a lot of synergies to find with a SHACL configuration in a model-driven perspective. You can publish the documentation, validate data, power your Sparnatural UI, and generate your API JSON schema from the same specification file. And probably much more – what would YOU do with a SHACL specification of your knowledge graph ?

Cet article Sparnatural SHACL configuration : manual, automated, off-the-shelf est apparu en premier sur Sparna Blog.

Another star to light on EU’s linked open data maturity flag !

Not talking about 2024 exceptional Northern Lights to come, but this one’s also good news for science !

➡️ Late 2023, the Publications Office of the European Union announced on social media the public release of the new CORDIS SPARQL endpoint.

CORDIS, aka « the Community Research and Development Information Service of the European Commission », is « the […] primary source of results from the projects funded by the EU’s framework programmes for research and innovation, from FP1 to Horizon Europe ». Described as a « rich and structured public repository with all project information held by the European Commission such as project factsheets, participants, reports, deliverables and links to open-access publications », the CORDIS catalog has also been made available in 6 European languages by Publications Office’s editorial team.

Cherry on top of a whole process, the CORDIS SPARQL endpoint release comes to crown a long-term linked open data project. The aim identifying, acquiring, preserving and providing access to knowledge in a common will to share with the widest public possible a trust-worthy, qualified and structured information (see Publications Office 2021 Annual Management Report).

In the context of the pandemic (and recent opening of data.europa.eu, the official portal for European data, as defined in 2017–2025 European Open Data Space strategy), the EuroSciVoc taxonomy of fields of science was released April 2020, followed December 2021 by the publishing of European research information ontology (EURIO) on the EU Vocabularies website .

As presented at ENDORSE conference March 2021, the redesign of CORDIS data-model in accordance with Semantic Web standards contributed to bring the platform « from acting as a data repository to finally playing an active role as data provider », where EuroSciVoc taxonomy & EURIO ontology both played key roles in the creation of future CORDIS knowledge graph and SPARQL endpoint :

EuroSciVoc […] is a multilingual, SKOS-XL based taxonomy that represents all the main fields of science that were discovered from the CORDIS content, e.g., project abstracts. It was built starting from the hierarchy of the OECD’s Fields of R&D classification (FoRD) as root and extended through a semi-automatic process based on NLP techniques. It contains almost 1 000 categories in 6 languages (English, French, German, Italian, Polish and Spanish) and each category is enriched with relevant keywords extracted from the textual description of CORDIS projects. It is constantly evolving and is available on EU Vocabularies website […].

In order to transform CORDIS data into Linked Open Data, thus aligning with Semantic Web standards, best practices and tools in industry and public organizations, the need for an ontology emerged. CORDIS created the EURIO (European Research Information Ontology) based on data about research projects funded by the EU’s framework programmes for research and innovation. EURIO is aligned with EU ontologies such as DINGO and FRAPO and de facto standard ontologies such as schema.org and the Organization Ontology from W3C. It models projects, their results and actors such as people and organizations, and includes administrative information like funding schemes and grants.

EURIO, which is available on EU Vocabularies website, was the starting point to develop a Knowledge Graph of CORDIS data that will be publicly available via a dedicated SPARQL endpoint. »

(Enrico Bignotti & Baya Remaoun, « EuroSciVoc taxonomy and EURIO ontology: CORDIS as (semantic) data provider  » , ENDORSE March 16, 2021. PDF VIDEO)

… A Knowledge graph that was soon released in 2022-2023 (see INDUSTRY TRACK 1 on Tuesday, 25 October of ISWC 2022 Conference for more detail), until final opening of a CORDIS SPARQL endpoint late november 2023.

Now fancy a few SPARQL queries in there ?

Follow the SPARQL

CORDIS SPARQL endpoint is actually made available on CORDIS Datalab (and already referenced in EU Knowledge Graph among other European SPARQL endpoints ! see the query / see the results)

Here you can access a quick documentation guide to CORDIS Linked Open Data : https://cordis.europa.eu/about/sparql.

Let’s have a look at EURIO ontology first : we need to understand it to query CORDIS knowledge graph.

As we are told in the guide, the latest version can be downloaded on EU Vocabularies website. When we unzip the archive we access the whole documentation about EURIO Classes & properties that we need to write our SPARQL queries – and a diagram of main classes and properties of CORDIS data model : 

At first sight we can observe on the schema 3 main groups of entities :

• On the top right, the projects & publications associated, key ressources of CORDIS ;
• On the top left, the fundings & grants materials, on « monetary » side of the project ;
• On the bottom, the organisations & persons implied, with references & coordinates.

Let’s open CORDIS SPARQL endpoint – some easy queries can be run to begin exploring CORDIS knowledge graph.

Nb : the data on SPARQL endpoint is a snapshot, but freshest dumps can be found on European data portal !

Here a simple one to find a list of FundingSchemes with their titles and IDs corresponding to « Horizon 2020 » programme :

▶️ See the results

The FILTER REGEX enables us to display the IDs corresponding to H2020 Funding Schemes.

We can make another query to get the projects with the Funding Scheme Programme they are related to (note that, in EURIO a eurio:hasFundingSchemeProgramme is a sub-property of eurio:fundingScheme) :

▶️ See the results

(Here we used a property path with a « / » to shorten the query to get the acronyms of projects & Funding Scheme Programmes codes).

… and combining with the first query we can find the projects depending on H2020 Funding Scheme Programme in particular :

▶️ See the results

It is also possible to get the list of all existing Funding Scheme Programmes CORDIS projects have been funded by – we observe 27 of them here (from the SPARQL endpoint) – while adding a count function to know how many projects per FundingSchemeProgramme :

▶️ See the results

Querying the organisations properties will return other kind of useful informations about geographical location of the projects stakeholders. Let’s figure out we want to find the projects whose coordinating organisations have sites located in France :

Projects whose coordinating organisations have sites located in France  PREFIX skos: <http://www.w3.org/2004/02/skos/core#> PREFIX eurio: <http://data.europa.eu/s66#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> SELECT ?project ?acronym ?role ?organisation ?country WHERE { # select the projects with their acronyms … ?project a eurio:Project. ?project eurio:hasAcronym/eurio:shortForm ?acronym. # … and organisations with ‘coordinator’ role and name … ?project eurio:hasInvolvedParty ?organisationrole. ?organisationrole eurio:roleLabel ?role. ?organisationrole eurio:roleLabel « coordinator ». ?organisationrole eurio:isRoleOf/eurio:legalName ?organisation. # … with address country for the sites defined at ‘FR’ ?organisationrole eurio:isRoleOf/eurio:hasSite/eurio:hasAddress/eurio:addressCountry ?country. VALUES ?country { ‘FR’ } } LIMIT 100

▶️ See the results

Depending on available data, you can either query via PostalAddress info (eurio:addressCountry ‘FR’) or AdministrativeArea (eurio:hasGeographicalLocation) … Here we’re lucky as both fields are mandatory ones.

Last but not least, we can also play with CORDIS vocabularies : here you’ll have the choice to investigate via plain keywords of Projects or Publications items, querying titles, abstracts or other types of literals…

An example of projects with abstracts containing string ❄ ‘winter’ ❄ – the URL giving the exact link to the project online :

▶️ See the results

But funniest way will be using EuroSciVoc taxonomy (and navigating through thesaurus hierarchy) : to do so we need to navigate through property « eurio:hasEuroSciVocClassification » to get the Concepts skosxl:prefLabel property … to finally obtain the thesaurus labels (don’t forget to choose a prefered language with a FILTER (lang parameter) :

Projects with their associated EuroSciVoc keywords (English prefLabels ) PREFIX skosxl: <http://www.w3.org/2008/05/skos-xl#> PREFIX skos: <http://www.w3.org/2004/02/skos/core#> PREFIX eurio: <http://data.europa.eu/s66#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> SELECT ?project ?acronym ?ESV WHERE { # select the projects with their acronyms … ?project eurio:hasAcronym/eurio:shortForm ?acronym. # … with EuroSciVoc Classification prefLabels … ?project eurio:hasEuroSciVocClassification/skosxl:prefLabel/skosxl:literalForm ?ESV. # … only returning ‘English’ prefLabels FILTER (lang(?ESV) = ‘en’) } LIMIT 100

▶️ See the results

A bit more complex one using first level of hierarchy of the taxonomy : here we are searching for all skos:broader concepts « with no other broader concept » (the FILTER NOT EXISTS formula), aka the top concepts or root concepts of the vocabulary used to describe the projects. Then counting the projects by each category :

▶️ See the results

… and maybe again more explicit results if refined to level 2 of hierarchy  :

▶️ See the results

And a little last one with a count, to enumerate most used EuroSciVoc Concepts for indexing projects :

▶️ See the results

This one an ideal one to generate a word cloud maybe ?

What if we send the CSV data to some nice online word cloud generator then ?

(OMG they also have a shooting star shape  in there 🤩)

As a conclusion…

According to Science (CORDIS saying !), New Year’s resolutions appear difficult to be held… because most of time too ambitious, restrictive or unprecisely formulated : indeed, « the effectiveness of resolutions depends on how they are framed. »

Horizon 2024, let’s suggest a(n RDF ?) well-framed one : may CORDIS SPARQL endpoint initiative be an example for other structures who want to share Linked Open Data !

Wishing you Best Interoperability and a Very Merry ✨ Sparqling New Year ! ✨

Cet article CORDIS : a SPARQL endpoint is born ! est apparu en premier sur Sparna Blog.

This guide will describe every step you need to know in order to create a Looker Studio Dashboard from SPARQL queries. All along, an example will be shown to illustrate all the steps with screenshots, code text and quotes.

Step 1 : Getting the SPARQL Connector 

Looker Studio does not provide any native connector for SPARQL. But a community connector exists, called SPARQL Connector, made by Datafabrics LLC, that can be used to create the data source. You can find it by searching for community connectors, or use this link. The code is available in this Github repository.

You have to grant access to your Google account for SPARQL Connector before using it. You will be able to find it in the connectors panel, in the Partner Connectors section, for your next queries.

Step 2 : Connect your knowledge graph

From your report, click on “Add Data” on the bottom right of the screen to open the connector panel. Select the SPARQL Connector in the connector panel (you can also search for it by entering “sparql” in the research field).

Then, follow the steps to create your own data source:

• Enter the URL of the SPARQL endpoint (the endpoint must be publicly accessible, without authentication), for example, with DBPedia:

https://dbpedia.org/sparql

• Then enter the SPARQL query, for example the following selects countries, their capital city label and their total population:
  

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX dbr: <http://dbpedia.org/resource/>
PREFIX dbo: <http://dbpedia.org/ontology/>
SELECT ?capital_city_label ?country_label  ?population
WHERE {
?capital_city  dbo:type dbr:Capital_city.
?capital_city rdfs:label ?capital_city_label.
?capital_city dbo:country ?country.
?country rdfs:label ?country_label.
OPTIONAL {?capital_city dbo:populationMetro ?population.}
FILTER (lang(?capital_city_label) = 'en')
FILTER (lang(?country_label) = 'en')
}

• For each field on your query, you have to create one field on your data source and select its type. To do so, you have to build a schema like this one:
  

[{"name": "capital_city_label", "dataType": "STRING"},
    {"name": "country_label", "dataType": "STRING"},
    {"name": "population", "dataType": "NUMBER"}]

Be sure your “name” fields match the fields you have on your query in the same order. You have to select the “dataType” you want for each of your fields, but you can change it later within Google Data Studio. Click here to learn more about data types.

Once every field is completed, you have to click twice on “Add”. If everything goes well, the connector panel will disappear and your new data source will appear on the right of the window and is ready to use. It is defaultly named as “SPARQL Connector”.

If you made a mistake while creating your data source, the SPARQL Connector panel can :

• Show an error message, that will indicate you the error type (endpoint, for example)

• Do nothing and you will have to check on your schema to be sure everything is correct.

• Create a data source as it should do, but Google Data Studio can’t use your data source, and show you this message on your chart :

If you click on “See details” Google Data Studio will show you the error type from the connector :

Step 3 : Transform your data 

First, you can change the name of your data source by clicking on the icon on the left of the data source on Google Data Studio (the icon will change into a pencil) to open the data source edition panel.

Then, click on the top left of the new panel where the name of your data source is to modify it.

Change name of the example data source to “Capital city Data (DBpedia)”

You can also change your data source by modifying your parameters in SPARQL Connector. To do so, click on “EDIT CONNECTION”. The SPARQL Connector panel will open with your current parameters and you can modify them.

In the data source edition panel, you can also change the type of your fields so it fits your needs (numbers can be changed as currency, text can be changed as geographic data, etc.).

Be careful of your fields format, you may not be able to use your data anymore. For example, if you have a “,” as a decimal separator, you can change your data type but you won’t be able to use this field as Google Data Studio uses “.” as a decimal separator.

The connector will also apply default values in query results which don’t have a value for a requested field. The default values are 0 for numbers, “” for strings and false for booleans.

The population field on DBpedia has some null values, but the connector transformed all these values into default values (0 for numbers).

You may need to use calculated fields in order to obtain new fields or to transform data. To create one,  click on “ADD A FIELD” on the right side of the same panel. Check the following page from the documentation to learn more about calculated fields.

By using a calculated field, the population data can be switched back to the original values.

In the new panel, choose the name of your new field, enter the formula. To ensure your formula is correct, a green check appears at the bottom of the panel. If not, it will turn into a red cross.

Enter the new field name: « population_recalculated ». Then enter the formula of the field : « NULLIF(population,0) ». In this case, if any population value is equal to 0 in the population field, it will turn into a null value in the calculated field. 

Step 4 : Improve performance with data extraction

Once you manage to create all your calculated fields, you may have some useless fields in your data source. Those fields may decrease the speed of your dashboard. You can use the “Extract Data” to keep the fields you need in another data source that you will use to make your report.

To use it, click on “Add Data” on the bottom right of the screen and select “Extract Data”.

Then, select your data source and the fields you want to keep in your report. You can make many extractions from one data source if you need. 

Choose the data source and keep only 3 fields : “capital_city_label”, “country_label” and “population_recalculated”.

You can also configure the auto-update tool to make sure your extracted data are up to date with the latest version of your data source from SPARQL Connector. In the bottom right of the panel, switch the auto-update button then choose the occurrence of the update (between daily, weekly and monthly).

A data source defaultly named “Extract Data” appears with the fields you selected from the previous data source.

This method only works for data sources, you won’t be able to use it on blended data. Make sure to do the extraction before blending to improve your performance. To learn more about blending, see this page from the Looker Studio documentation.

Step 5 : Create your dashboard 

Here is a quick guide on how to create a chart in Google Data Studio. Check the chart reference documentation for more information about charts available by default.

To build a dashboard, you will need to select a widget first (pie chart, table, histograms, etc.). Click on “Add a chart” on the top of the screen and select the one you need.

Click on “Add a chart” and select a pie chart.

Select your chart on the report, it will open a panel on the right side of the screen where you can see the chart type and modify it. You can select the data to display in the “SETUP” panel. You can also customize the chart with the “STYLE” panel.

Place the chart on your dashboard anywhere you want to see it. Google Data Studio will automatically choose the data source and some fields which fit the charts, but you can choose to modify them in the “SETUP” panel on the right.

Choose “capital_city_label” as dimension and “population recalculated” as metric.

Here is the result of this configuration :

In the “STYLE” panel, you can choose to modify some options in the chart to customize it.

Change the number of slices from 10 to 6 to see the 5 top values + others value.

The chart will change automatically with your new parameters as you change them.

Congratulations, you have successfully made your first chart!Try to get your own data sources with SPARQL Connector, make your own dashboards with Looker Studio, and send us the links !

Cet article Dashboards from SPARQL knowledge graphs using Looker Studio (Google Data Studio) est apparu en premier sur Sparna Blog.

Cet article Supports de formation SPARQL CC-BY-SA est apparu en premier sur Sparna Blog.

UPDATE avril 2021 : Sparnatural a un nouveau site web a http://sparnatural.eu !

Dans la copie d’écran ci-dessus, on demande « Toutes les oeuvres exposées dans un musée Français qui expose une oeuvre du Caravage, et dont l’auteur est Italien ».

Démo de Sparnatural sur DBPedia

Le screencast ci-dessus est extrait de la démo de Sparnatural paramétrée sur DBPedia avec laquelle vous pouvez jouer en ligne.

Le développement de ce composant a été réalisé dans le cadre du projet OpenArchaeo où il est utilisé pour naviguer dans des données archéologiques. Il est autonome du projet et peut être réutilisé dans le cadre de sa license LGPL. Le code source est ouvert et il est interdit de « refermer » le code source, toute modification doit être publiée sous la même licence, et idéalement reversée dans le dépôt Github du projet.

Sparnatural s’inspire en grande partie de la navigation proposée par l’interface ResearchSpace du British Museum.

Et pourquoi c’est cool ?

• Parce que ça n’existait pas !  (en dehors de ResearchSpace, mais dont le source est en React), en tout cas pas comme un composant autonome et paramétrable
• Parce que le composant est paramétrable à souhait pour construire des requêtes sur différentes structures de graphe, en changeant le paramétrage des classes et des propriétés; vous pouvez jeter un oeil au fichier de paramétrage de la démo.
• Parce qu’il est orienté end-user et que, en particulier, la structure du graphe que l’on présente à l’utilisateur n’est pas obligatoirement – en fait n’est jamais – celle du graphe de données sous-jacent:
  • Chaque classe dans le composant peut être mappé sur un critère plus complexe (On présente à l’utilisateur « Type d’activité », qui est mappé sur « tous les skos:Concept ayant un skos:inScheme ex:ActivityType »)
  • Chaque lien dans le composant peut être mappé sur une séquence de liens RDF dans le graphe (un property path) (Le lien « Musée expose oeuvre » dans le composant de construction de query est l’inverse du lien RDF « Oeuvre dbpedia:museum Musée »). Typiquement le paramétrage de tous les liens inverses permet à un utilisateur d’explorer le graphe en le prenant par n’importe quel bout;
  • On peut limiter les types d’objets et les types de liens que l’on présente dans le composant pour ne permettre d’interroger qu’une sous-partie des données;
• Parce qu’il offre plusieurs mode de sélection des valeurs :
  • un champ d’autocompletion, à associer à une requête SPARQL (ou pas SPARQL) qui ira proposer des valeurs sur la base des caractères tapés dans le champs;
  • un champ de dropdown, pour les petites listes;
  • un champ de recherche texte;
  • un champ d’input de date (début / fin);
• Parce que la requête peut s’exécuter au fur et à mesure de la construction de l’équation de recherche; pas besoin d’appuyer sur un bouton; cela fait beaucoup pour obtenir une expérience utilisateur de découverte des données;
• Parce que c’est multilingue : on peut associer des libellés en plusieurs langue à chaque classe et chaque propriétés;
• Parce que c’est facile d’injecter des icônes fontawesome pour illustrer chaque classe dans les menus;
• Parce qu’on peut post-traiter la requête après que le composant l’a construite : ajouter la sélection de plusieurs colonnes, ajouter des préfixes, etc.
• Parce que ça peut joliment égayer la platitude morne des formulaires de requêtes SPARQL que l’on expose à des utilisateurs, en s’intégrant avec YASGUI, pour permettre une découverte intuitive des données;

Le résultat, au-delà d’un simple éditeur SPARQL, offre une vraie expérience d’exploration des données, avec des mécanismes d’essai-erreur, retour arrière, prise du graphe par un autre bout, etc.

Limites de l’exercice

L’objectif est d’offrir un moyen simple et compréhensible de naviguer dans des données. En conséquence, Sparnatural n’est capable que de construire des motifs de graphe SPARQL simple, et ne sais pas gérer les UNION, OPTIONAL, sous-select, BIND, etc.

Par ailleurs le composant s’arrête à sélectionner les URIs des objets cherchés, il n’est pas possible pour un utilisateur de choisir les colonnes présentées dans le tableau de résultats. Il faut post-traiter la requête pour injecter la sélection des valeurs de colonnes.

Si, comme pour la démo DBPedia, vous intégrez Sparnatural avec YASGui et YASR et que la page HTML envoie la requête SPARQL, faites attention que le service SPARQL doit supporter les requêtes CORS (Cross-Origin Resource Sharing), ce qui n’est pas le cas de tous les services SPARQL… mais ça devrait !

Envie d’essayer ?

Rendez-vous sur le dépôt Github de Sparnatural si vous voulez un peu plus de doc ou que vous voulez remonter un ticket, un bug, ou contribuer au code. D’autres démos devraient suivre, stay tuned !

Cet article SPARNAtural : écrire des requêtes SPARQL, tout naturellement est apparu en premier sur Sparna Blog.

Voici un tutorial guidé d’interrogation SPARQL des données de Doremus, autour de Gabriel Fauré; ce tutorial explore la modélisation de « création » entre un compositeur et une oeuvre musicale (ou plutôt son expression au sens FRBR). Il montre comment :

• Trouver un compositeur dans le navigateur Overture de Doremus, le navigateur à facette et le service SPARQL;
• Naviguer dans ses activités de création, vers ses oeuvres et leurs expressions, et récupérer les dates de création des oeuvres; le tout en illustrant les opérateurs SPARQL de property path, aggrégation, COUNT, assignation, etc;
• Générer une timeline des compositeurs classiques de façon semi-automatique en utilisant le service timelinejs :

CIDOC-CRM + SPARQL Tutorial sur les données Doremus

Introduction aux requêtes SPARQL sur les données du projet Doremus (http://data.doremus.org) qui modélise et diffuse les données de création d’oeuvres musicale…

Pour réutiliser ce document dans un cadre non-commercial vous pouvez me contacter.

Dans la même veine, vous pouvez consulter le tutorial SPARQL sur les données data.bnf.fr.

Image du post : Carte postale éditée en 1900 lors de la représentation de l’opéra Prométhée de Gabriel Fauré au Théâtre des Arènes, à Béziers. Sur Wikimedia Commons.

Cet article SPARQL sur DOREMUS : une balade autour de Gabriel Fauré est apparu en premier sur Sparna Blog.