Data quality issue
Definition
A data quality issue is an issue with regard to data quality.
Notes
Generally, a data quality issue concerns an incorrect data value, but it may also involve any other data quality dimensions, e.g., completeness and timeliness.
Attributes of a data quality issue are:
- Description of a data quality issues
- Classification of a data quality issues
- Impact, severity/urgency
- Assignment of the data quality issues to a person who will resolve the data quality issues
- What is changed, why en how in the resolution of a data quality issues
- Resolution time/Duration
Synonym(s)
- Data anomalies
- Data error
- Data quality incident
- Defect
- Error
- Nonconformity
Lifecycle
| Phase | Activity | |
|---|---|---|
| Plan | * To detect or identify data quality issues * To determine if similar data quality issues exist * To register data quality issues * To count the number of data quality issues * To categorize or classify data quality issues * To diagnose data quality issues(‘root cause analysis’) * To formulate options for remediation of data quality issues. * To assess the business impact of data quality quality issues(‘severity’) * To prioritize data quality issuesbased on business impact (by data owners and data stewards) * To eventually escalate data quality issues. See figure. | |
| Do | * To eliminate or resolve a data quality issues(‘correction’, ‘remediation’, ‘scrubbing’ or ‘cleansing’) * To eliminate the cause of a data quality issuesto prevent recurring (‘corrective action’) * To eliminate the cause of a potential data quality issues(‘preventive action’) * To deal with the consequences of the data quality issues, e.g., to report data isquality issuessues to data users * To document actions taken. | |
| Check | * To monitor the status of data quality issuesand their resolutions. * To review the effectiveness of corrective and preventive action. | |
| Act | * To adapt corrective and preventive action. |
Characteristics
| Characteristic | Requirements |
|---|---|
| Severity | Data quality issuesare resolved in order of severity. |
Relations
| Data quality issue | is child of | issue |
| Data quality issue | is an element of a | data quality management system |
| Data quality issue | is one of the principles of a | data quality program |
| Data quality issue | can be caused by | data entry processes |
| Data quality issue | can be caused by | data migration |
| Data quality issue | is a subject in a | data quality service level agreement |
| Data quality issue | is identified by | data profiling |
| Data quality issue | is prevented by applying | data quality rules |
| Data quality issue | is identified and resolved by | data stewards |
| Data quality issue | is resolved by | data cleansing |
| Data quality issue | is eliminated by | corrections |
| Data quality issue | is prevented by | corrective actions |
| Data quality issue | is prevented by | preventive actions |
| Data quality issue | can be easiers assessed by | data lineage |
| Data quality issue | can be tracked in an | incident tracking system |
| Data quality issue | is detected and reported by | data quality monitoring |
Data quality issues escalation path
The figure shows a possible escalation path of data quality issues.
Figure 2: Data quality issues escalation path.
Story
A customer of the TradeAll trade company complained that he had been delivered an incorrect item. The sales department reported this incident in the Incident Tracking System, which was assigned to the Data Steward. The Data Steward then went to see how this could have happened. He found that the item number did not match the item description and suspected that this was due to a data entry error. Based on the item number, the products were taken from the warehouse.
He corrected the item number (Correction), but to prevent this kind of error, he took the measure that the entry of items was checked by a second person to prevent more incidents (Corrective Action). He also made sure to check for duplicate item numbers in the database (Corrective Action).. The customer was apologised for the error and the correct article was delivered as soon as possible.
References
DAMA (2017). DAMA-DMBOK. Data Management Body of Knowledge. 2nd Edition. Technics Publications Llc. August 2017. Chapter 13 Data Quality, section 2.7.3 Develop Operational Procedures for Managing Data Issues.
DAMA Dictionary of Data Management. 2nd Edition 2011. Technics Publications, LLC, New Jersey.
ISO 9000:2015. Quality Management Systems – Requirements.
ISO 9001:2015. Quality Management Systems – Fundamentals and vocabulary.