User Tools

Site Tools

data_quality_management_system:data_cleansing

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
data_quality_management_system:data_cleansing [2023/03/01 16:46]
peter 		data_quality_management_system:data_cleansing [2024/05/26 19:33] (current)
peter
Line 1: Line 1:
 ===== Data cleansing ===== ===== Data cleansing =====
- 
-=== Introduction === 
-This factsheet describes knowledge about data cleansing in a nutshell. Data cleansing is highlighted from different angles in a structured way. 
  
 === Definition === === Definition ===
-Data cleansing is the process of detecting and correcting [[data_quality_management_system/data_issue|data issues]] to improve the quality of data to an acceptable level.+Data cleansing is the [[general_term/process|process]] of detecting and correcting [[data_quality_management_system:data_quality_issue|data issues]] to improve the quality of data to an acceptable level.
  
 ===Notes=== ===Notes===
  
-What an acceptable data quality level is for an organization should be defined for each [[data_quality_general/data_quality_dimension|data quality dimension]].+An organization should define an acceptable data quality level for each [[data_quality_general/data_quality_dimension|data quality dimension]].
  
 Dimensions of data quality that can be improved by data cleansing are: Dimensions of data quality that can be improved by data cleansing are:
Line 16: Line 13:
   * [[data_quality_dimension/completeness|Completeness of records]]   * [[data_quality_dimension/completeness|Completeness of records]]
   * [[data_quality_dimension/completeness|Completeness of data values]]   * [[data_quality_dimension/completeness|Completeness of data values]]
-  * [[data_quality_dimension/compliance|Compliance]] of data with laws, regulations, and standards]]+  * [[data_quality_management_system:compliance|Compliance]] of data with laws, regulations, and standards]]
   * [[data_quality_dimension/consistency|Consistency]] of data values   * [[data_quality_dimension/consistency|Consistency]] of data values
-  * Currency of data values+  * [[data_quality_dimension/currency|Currency]] of data values
   * [[data_quality_dimension/integrity|Integrity]] of data values   * [[data_quality_dimension/integrity|Integrity]] of data values
   * [[data_quality_dimension/linkability|Linkability]] of data files   * [[data_quality_dimension/linkability|Linkability]] of data files
Line 34: Line 31:
  
 === Purpose === === Purpose ===
-To detect and correct [[data_quality_management_system/data_issue|data issues]] and inconsistencies.+To detect and correct [[data_quality_management_system:data_quality_issue|data issues]] and inconsistencies.
  
 === Life cycle === === Life cycle ===
Line 45: Line 42:
  
 === Methods === === Methods ===
-The next methods of correcting [[data_quality_management_system/data_issue|data issues]] can be distinguished:+The following methods of correcting [[data_quality_management_system:data_quality_issue|data issues]] can be distinguished:
 ^ Method                                    ^ Example or explanation                                                                                                                                                                                                                                                                                                                                                                                                            ^ ^ Method                                    ^ Example or explanation                                                                                                                                                                                                                                                                                                                                                                                                            ^
 | Abbreviation expansion                    | Abbreviation expansion transforms abbreviations into their full form. There are different kinds of abbreviation. One type shortens each of a set of words to a smaller form, where the abbreviation consists of a prefix of the original data value. E.g. “USA” stands for “United States of America.”                                                                                                                            | | Abbreviation expansion                    | Abbreviation expansion transforms abbreviations into their full form. There are different kinds of abbreviation. One type shortens each of a set of words to a smaller form, where the abbreviation consists of a prefix of the original data value. E.g. “USA” stands for “United States of America.”                                                                                                                            |
Line 68: Line 65:
 | Type conversion                           | Type conversion (also called casting) is an operation that converts a piece of data of one data type to another data type. Type conversion can be used to make sure that numbers are stored as numerical data types and that a date should be stored as a date object.                                                                                                                                                            | | Type conversion                           | Type conversion (also called casting) is an operation that converts a piece of data of one data type to another data type. Type conversion can be used to make sure that numbers are stored as numerical data types and that a date should be stored as a date object.                                                                                                                                                            |
 | Edit rules                                | Edit Rules, a new class of data quality rules, are rules that tells how to fix errors, i.e. which attributes are wrong and what values they should take.                                                                                                                                                                                                                                                                          | | Edit rules                                | Edit Rules, a new class of data quality rules, are rules that tells how to fix errors, i.e. which attributes are wrong and what values they should take.                                                                                                                                                                                                                                                                          |
-| Data lifecycle management                 | Data Lifecycle Management can be defined as the different stages that the data traverses throughout its life from the time of inception to destruction. Data lifecycle stages encompass creation, utilization, sharing, storage, and deletion.                                                                                                                                                                                    |+| Data lifecycle management                 | Data Lifecycle Management can be defined as the different stages that the data traverses throughout its life from the time of inception to destruction. Data lifecycle stages encompass creation, utilisation, sharing, storage, and deletion.                                                                                                                                                                                    |
  
 Note 4: Data issue prevention is far superior to data issue detection and cleansing, as it is cheaper and more efficient to prevent issues than to try and find them and correct them later. Note 4: Data issue prevention is far superior to data issue detection and cleansing, as it is cheaper and more efficient to prevent issues than to try and find them and correct them later.
Line 96: Line 93:
 | Cost-effectiveness of data cleansing | Data cleansing must lead to a positive business case, i.e. the benefits must be bigger than the costs.  | | Cost-effectiveness of data cleansing | Data cleansing must lead to a positive business case, i.e. the benefits must be bigger than the costs.  |
  
-=== Relationss === +=== Relations === 
-  * Data cleansing resolves [[data_quality_management_system/data_issue|data issues]]. +|Data cleansing| is child of |[[general_term/process|process]]| 
-  * [[data_quality_management_system/data_quality_monitoring|Data quality monitoring]] is follow-up by data cleansing. +|Data cleansing| is an element of a |[[data_quality_general:data_quality_management_system|data quality management system]]| 
-  * Purpose of data cleansing is to improve [[data_quality_general/data_quality|data quality]] and make it ‘fit for use’ +|Data cleansing| resolves |[[data_quality_management_system:data_quality_issue|data issues]]| 
-  * A data cleansing process can be performed using one of the different data cleansing methods. +|Data cleansing|is the successor of|[[data_quality_management_system/data_quality_monitoring|data quality monitoring]]| 
-  * [[data_quality_management_system/critical_data_element|Critical data elements]] are input for the procedure to data cleansing.+|Data cleansing|uses|data cleansing methods| 
 +|Data cleansing|wil be applied firstly to|[[data_quality_management_system/critical_data_element|critical data elements]]| 
 +|Data cleansing|improves|[[data_quality_general/data_quality|data quality]]| 
 +|Data cleansing|needs|[[data_quality_management_system/data_quality_rule|data quality rules]]| 
 {{:data_management:data_quality:data_cleansing.jpg?500|}} {{:data_management:data_quality:data_cleansing.jpg?500|}}
  
Line 164: Line 165:
  
 What is data cleansing? Guide to data cleansing tools, services and strategy. (2020, August 13). Talend Real-Time Open Source Data Integration Software. https://www.talend.com/resources/what-is-data-cleansing/ What is data cleansing? Guide to data cleansing tools, services and strategy. (2020, August 13). Talend Real-Time Open Source Data Integration Software. https://www.talend.com/resources/what-is-data-cleansing/
 +
 +{{tag>All DQMS}}
 +
data_quality_management_system/data_cleansing.1677689201.txt.gz · Last modified: 2024/03/08 13:33 (external edit)

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution 4.0 International