关键词: 数据质量, 条件谓词依赖, 不一致数据, 属性错误率概率模型, 修复可能性

Abstract: Inconsistent data violating integrity constraints is a common data quality issue in the databases of aviation maintenance information systems. Compared to the “equality constraint” forms of Functional Dependency (FD) and Conditional Functional Dependency (CFD), the Conditional Functional Dependency with built-in Predicates (CFDps) that include special forms of predicates (such as “greater than”, “less than”, and “equal to”) present greater difficulty in repairing errors by finding the initial true value. This is because there are theoretically infinitely many candidate values that can satisfy the dependency. Aiming at the problem that it was more difficult to repair inconsistent data violating CFDps, a Heuristic Repair method based on Maximum Possibility, namely Heuristic-MPR (Heuristic- Maximum Possibility Repair), was proposed. This method was mainly divided into four stages: first, the conflicting tuples and conflicting attributes in the dataset were identified according to the CFDps. Second, a probability model for attribute error rates was established, and the attribute with the highest error probability was selected as the candidate conflicting attribute for priority repair. Third, when conducting repairs, drawing on the concepts of machine learning, the correlations between the candidate conflicting attribute and other attributes in the dataset were considered, the repair probabilities of all repair schemes were calculated, and the scheme with the maximum probability was selected as the repair value. Finally, the extent to which the repaired results satisfy the dependency rules was verified, and a determination was made as to whether it was necessary to reselect candidate conflicting attributes. The experimental results on real data show that the Heuristic-MPR method has good repair capabilities for inconsistent data of various attribute types. The average repair accuracy that is completely consistent with the initial true value is 82.22%, and the average repair deviation rate that is not completely consistent with the initial true value is 1.27%.

Key words: data quality, conditional functional dependency with built-in predicates (CFDps), inconsistent data, attribute error probability model, repair possibility