User Tools

Site Tools


data_management:data_quality:data_quality_monitoring

This is an old revision of the document!


Data Quality Monitoring

Introduction

This factsheet describes knowledge about Data Quality Monitoring in a nutshell. Data Quality Monitoring are highlighted from different angles in a structured way.

Definition

Data Quality Monitoring is the process of detecting and reporting data issues.

Synonyms

Data Quality Measuring

Purpose

The purpose of Data Quality Monitoring is:

  • to gain insight into data issues, both on individual level and statistically (dashboard),
  • to be able to follow and track the number of data issues in time,
  • to be able to correct data issues and to analyse the cause of data issues (see factsheet Data Issues).

Procedure to manage Data Quality Monitoring

Phase Activity
Plan * Establish the scope of Data Quality Monitoring.
* Decide which tool will be used to apply Data Quality Monitoring.
* Implement the tool.
* Establish the Data Quality Rules that will be applied.
* Assign Roles and responsibilities.
Do * Run Data Quality Monitoring periodically to detect data issues.
* Use the results of Data Quality Monitoring for solving the data issues.
Check * Evaluate the effectiveness and costs of Data Quality Monitoring
Act * Adapt the Data Quality Rules.
* Make other changes: scope, tool, roles and responsibilities.

Characteristics and requirements

Characteristic Requirement
Efficiency Data Quality Monitoring should be done efficiently and cost-effective.
Effectivensss Data Quality Monitoring should be effective.

Relationships

  • Data Quality Monitoring can be focussed on Critical Data Elements.
  • Data Quality Monitoring detects and reports Data Issues.
  • Data Quality Monitoring can be input for or is part of Data Cleansing.
  • Data Quality Monitoring can be applied to various Data Quality Dimensions.
  • Roles and Responsibilities are assigned regarding Data Quality Monitoring.
  • Data Quality Monitoring will monitor the Quality of the Data Supplier source data.

Story

A municipality wants to improve and guarantee the quality of its property register. First, they want to get an insight into the situation before they start improving. Therefore, they first purchase a tool with which, the quality of the data can be efficiently measured and monitored. They set the right data quality rules in the tool and produce their first report. It shows that there are two percent locations outside the municipal boundaries and that there are property categories that are not consistent with the description of the property.

Based on this report, the municipality decided to first solve the problem of inaccurate locations. This is where they suffer the most. They start a process of Data Cleansing to solve this. Partly, they do this by comparing the register with another file. For another part, the objects are visited by officials and the locations are redetermined. The monitor measures the progress of the cleaning action. Furthermore, an extra check have been built into the application with which new objects are entered, so that this problem can no longer occur.  

References

Borna. (2021, March 9). How to measure data quality — 13 metrics you should be tracking (But don't). Show Me The Data. https://showmethedata.blog/how-to-measure-data-quality-13-metrics

DAMA (2017). DAMA-DMBOK. Data Management Body of Knowledge. 2nd Edition. Technics PublicationsLlc. August 2017.

DAMA Dictionary of Data Management. 2nd Edition 2011. Technics Publications, LLC, New Jersey. References Data monitoring. (n.d.). Soda Documentation. https://docs.soda.io/soda-sql/documentation/data-monitoring.html

Dataqualitysolution.com. (n.d.). dataqualitysolution.com - dataqualitysolution Resources and Information. https://www.dataqualitysolution.com/data-quality-monitoring-and-reporting Inzicht in datakwaliteit: De data quality monitor. (2020, October 18). Emerce. https://www.emerce.nl/achtergrond/inzicht-in-datakwaliteit-de-data-quality-monitor

ISO 8000:150-Data quality monitoring | DPA. (2017, January 26). Data and Process Advantage. https://www.dpadvantage.co.uk/2015/01/10/iso-8000150-data-quality-monitoring/

LESSON - Data quality monitoring: The basis for ongoing information quality management | Transforming data with intelligence. (2007, May 8). Transforming Data with Intelligence. https://tdwi.org/articles/2007/05/08/data-quality-monitoring-the-basis-for-ongoing-information-quality-management.aspx

Monitoring data quality. (n.d.). DIME Wiki. Retrieved June 9, 2021, from https://dimewiki.worldbank.org/Monitoring_Data_Quality

Syed, I. Q. (2021, April 20). Implementing data quality monitoring frameworks & process | Redgate. Simple Talk. https://www.red-gate.com/simple-talk/sql/bi/implementing-a-data-quality-monitoring-framework/

What is data quality monitoring? - Definition from Techopedia. (2014, September 15). Techopedia.com. https://www.techopedia.com/definition/30608/data-quality-monitoring

data_management/data_quality/data_quality_monitoring.1646680661.txt.gz · Last modified: 2022/03/07 20:17 by rene