Quality Indicators Dictionary and Markup Language - QualityML

The scenario of rapidly growing geodata catalogues requires tools focused on facilitate users the choice of products. Having quality fields populated in metadata allow the users to rank and then select the best fit-for-purpose products. In this direction, QualityML is a dictionary that contains hierarchically structured concepts to precisely define and relate quality levels: from quality classes to quality measurements. This levels are used to encode quality semantics for geospatial data by mapping them to the corresponding metadata schemas. The benefits of having encoded quality semantics, in the case of data producers, are related with improvements in their product discovery and better transmission of their characteristics. In the case of data users, they would better compare quality and uncertainty measures to take the best selection of data as well as to perform dataset intercomparison. Also it allows other components (such as visualization, discovery, or comparison tools) to be quality-aware and interoperable.

On one hand, the QualityML is a profile of the ISO geospatial metadata standards (e.g. ISO 19157) providing a set of rules for precisely documenting quality measure parameters that is structured in 5 levels. On the other hand, QualityML includes semantics and vocabularies for the quality concepts. Whenever possible, it uses statistic expressions from the UncertML dictionary (http://www.uncertml.org) encoding. However it also extends UncertML to provide a list of alternative metrics that are commonly used to quantify quality beyond the uncertainty concept. Unfortunatelly the website of UncertML was shotdown in 2016. It still can be queried from the Web Archive project. Given the situation, in 2018 QualityML decided to duplicated all UncertML records that were used at that time.

Finally, keep in mind that QualityML is not just suitable for encoding geospatial dataset level quality but also considers pixel and object level uncertainties. This is done by linking the metadata quality descriptions with layers representing not just the data but the uncertainty values associated with each geospatial element.

This page is structured in the following sections:

• Generic quality element
• Definition of quality classes
• Definition of quality indicators
• Quality measure matrices
• Definition of the domain where quality metrics are applied (including link to the schema)
• List of metrics defined (including link to schemas)
• Quality metrics sets
• Reserved words
• XML encoding

All relative URIs in this page refer to http://www.qualityml.org/1.0/

Generic quality element

A Quality element is a combination of a quality class, a quality indicator, a quality domain, a quality metric (which include a metrics name, metrics description, metrics parameters, its values and units of measure). The combination of a quality domain and a quality metrics are commonly known as quality measures. We suggest 2 ways to map the QualityML concepts to ISO 19139 and ISO 19115-3. The first way makes a simple use of the ISO values keeping the structure of Record as simple as possible. The second one extends Record to allow a more structured encoding:

Quality classes

The ISO 19157:2013 Geographic information - Data quality defines 7 data quality elements (or classes) describing a certain aspect of the quality of geographic data in which quality elements can be classified.

Quality indicators

Quality measures matrices

Quality class: Completeness

Quality class: Logical consistency

Quality class: Positional accuracy

Quality class: Temporal quality

Quality class: Thematic accuracy

Quality domain

Quality Metrics

XML schemas and examples are available here (for convinience a exact copy of uncertml is included).

URI	Metric	Parameters	Origin
metrics/items	Items	value (choice of: indicator (boolean), count (int), rate (real, attribute: max (real)))	ISO 19157
metrics/Half-lengthConfidenceInterval	Half-length of the confidence interval	Extends http://www.uncertml.org/statistics/confidence-interval (upper and lower values) adding a half-lenght value	ISO 19157
metrics/MeanAbsolute	Mean Absolute Error (MAE)		ISO 19157
metrics/MeanAbsolute2D	Mean Absolute Error (MAE)		ISO 19157
metrics/MeanAbsolute3D	Mean Absolute Error (MAE)		ISO 19157
metrics/MeanBias	Mean Bias		ISO 19157
metrics/NormalizedMeanBias	Mean Bias Error (MBE)		ISO 19157
metrics/RootMeanSquareError	Root mean square error		ISO 19157
metrics/NormalizedRootMeanSquareError	Normalized root mean square error		ISO 19157
metrics/CoefficientOfVariationRootMeanSquareError	coefficient of variation Root Mean Square Error		ISO 19157
metrics/LMAS	Absolute linear error at 90% significance level. Alternative 1		ISO 19157
metrics/ALE	Absolute linear error at 90% significance level. Alternative 2		ISO 19157
metrics/ACE	Absolute circular error at 90% significance level. Alternative 1		ISO 19157
metrics/CMAS	Absolute cirular error at 90% significance level. Alternative 2		ISO 19157
metrics/ConfidenceEllipse	Confidence ellipse	a, b, angle in Multiple values, confidence	ISO 19157
metrics/ConfusionMatrix	Confusion matrix		ISO 19157
http://www.uncertml.org/statistics/covariance-matrix	Covariance Matrix		ISO 19157
metrics/CorrespondenceMatrix	Correspondence Matrix		GVQ
metrics/RelativeCorrespondenceMatrix	Relative Correspondence Matrix		GVQ
metrics/RelativeError	1D Relative error	confidence	ISO 19157
metrics/RelativeError2D	2D Relative error	confidence	ISO 19157
metrics/NormalizedConfusionMatrix	Normalized confusion matrix	Inspired in http://www.uncertml.org/statistics/confusion-matrix but using double type for count	ISO 19157
metrics/KappaCoefficient	Kappa coefficient		ISO 19157
metrics/OmmisionError	Omission Error	actual categories	GVQ
metrics/CommissionError	Commission Error	predicted categories	GVQ
metrics/Promiscuity	Promiscuity		GVQ
metrics/MajorityCategories	Majority Categories		GVQ
metrics/Purity	Purity		GVQ
metrics/CoefficientOfDetermination	Coefficient of determination		GVQ
metrics/DiscreteConfusionMatrix	Discrete Confusion Matrix		GVQ
metrics/TruePositive	True Positive		GVQ
metrics/TrueNegative	True Negative		GVQ
metrics/FalsePositive	False Positive		GVQ
metrics/FalseNegative	False Negative		GVQ
metrics/Sensitivity	Sensitivity		GVQ
metrics/Specificity	Specificity		GVQ
metrics/OverallAccuracy	Overall Accuracy		GVQ
metrics/FalsePositiveRate	False Positive Rate		GVQ
metrics/PositivePredictiveValue	Positive Predictive Value		GVQ
metrics/FalseDiscoveryRate	False Discovery Rate		GVQ
metrics/MatthewsCorrelationCoefficient	Matthews Correlation Coefficient		GVQ
metrics/AreaUnderROCCurve	Area Under ROC Curve		GVQ
metrics/Reliability	Reliability		GVQ
metrics/AverageReliability	Average Reliability		GVQ
metrics/Accuracy	Accuracy		GVQ
metrics/AverageAccuracy	Average Accuracy		GVQ

Quality metrics sets

Other reserved words

XML encoding

Example using UncertML in an ISO metadata record

Example using QualityML in an ISO metadata record for dataset level quality

Example using QualityML in an ISO metadata record pixel level quality

The research leading to these results has been carried out in the GeoViQua project that has received funding from the European Union Seventh Framework Programme (FP7/2010-2013) under grant agreement no. 265178