A model-based evaluation of data quality activities in KDD

作者：Mezzanzanica Mario; Boselli Roberto; Cesarini Mirko^*; Mercorio Fabio

来源：Information Processing & Management, 2015, 51(2): 144-166.

DOI：10.1016/j.ipm.2014.07.007

• Data quality
• Data cleansing
• Model checking
• Real-life application

We live in the Information Age, where most of the personal, business, and administrative data are collected and managed electronically. However, poor data quality may affect the effectiveness of knowledge discovery processes, thus making the development of the data improvement steps a significant concern.
In this paper we propose the Multidimensional Robust Data Quality Analysis, a domain-independent technique aimed to improve data quality by evaluating the effectiveness of a black-box cleansing function. Here, the proposed approach has been realized through model checking techniques and then applied on a weakly structured dataset describing the working careers of millions of people. Our experimental outcomes show the effectiveness of our model-based approach for data quality as they provide a fine-grained analysis of both the source dataset and the cleansing procedures, enabling domain experts to identify the most relevant quality issues as well as the action points for improving the cleansing activities.
Finally, an anonymized version of the dataset and the analysis results have been made publicly available to the community.

• 出版日期2015-3