Label noise filtering method based on dynamic probability sampling

doi:10.11772/j.issn.1001-9081.2021061026

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (12): 3485-3491.DOI: 10.11772/j.issn.1001-9081.2021061026

Special Issue: 第十八届中国机器学习会议(CCML 2021)

• The 18th China Conference on Machine Learning • Previous Articles Next Articles

Label noise filtering method based on dynamic probability sampling

Zenghui ZHANG¹, Gaoxia JIANG¹^,², Wenjian WANG¹^,²()

^1.School of Computer and Information Technology，Shanxi University，Taiyuan Shanxi 030006，China
^2.Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education （Shanxi University），Taiyuan Shanxi 030006，China

Received:2021-03-16 Revised:2021-06-29 Accepted:2021-07-21 Online:2021-12-28 Published:2021-12-10
Contact: Wenjian WANG
About author:ZHANG Zenghui， born in 1996， M. S. candidate. Her research interests include machine learning.
JIANG Gaoxia， born in 1987， Ph. D.， associate professor. His research interests include data quality analysis， machine learning.
Supported by:
the National Natural Science Foundation of China(62076154);the Program of Shanxi Province International Scientific and Technological Cooperation(201903D421050);the Project of Central Government to Guide Local Scientific and Technological Development(YDZX20201400001224);the Scientific and Technological Innovation Program of Higher Education Institutions in Shanxi(2020L0007)

基于动态概率抽样的标签噪声过滤方法

张增辉¹, 姜高霞¹^,², 王文剑¹^,²()

^1.山西大学计算机与信息技术学院，太原 030006
^2.计算智能与中文信息处理教育部重点实验室（山西大学），太原 030006

通讯作者: 王文剑
作者简介:张增辉（1996—），女，山西太原人，硕士研究生，主要研究方向：机器学习
姜高霞（1987—），男，山西新绛人，副教授，博士，CCF会员，主要研究方向：数据质量分析、机器学习；
基金资助:
国家自然科学基金资助项目(62076154);山西国际科技合作计划项目(201903D421050);中央引导地方科技发展资金项目(YDZX20201400001224);山西省高等学校科技创新项目(2020L0007)

Abstract

Abstract:

In machine learning， data quality has a far-reaching impact on the accuracy of system prediction. Due to the difficulty of obtaining information and the subjective and limited cognition of human， experts cannot accurately mark all samples. And some probability sampling methods proposed in resent years fail to avoid the problem of unreasonable and subjective sample division by human. To solve this problem， a label noise filtering method based on Dynamic Probability Sampling （DPS） was proposed， which fully considered the differences between samples of each dataset. By counting the frequency of built-in confidence distribution in each interval and analyzing the trend of information entropy of built-in confidence distribution in each interval， the reasonable threshold was determined. Fourteen datasets were selected from UCI classic datasets， and the proposed algorithm was compared with Random Forest （RF）， High Agreement Random Forest Filter （HARF）， Majority Vote Filter （MVF） and Local Probability Sampling （LPS） methods. Experimental results show that the proposed method shows high ability on both label noise recognition and classification generalization.

Key words: label noise, Dynamic Probability Sampling (DPS), noise filtering, label confidence, confidence

摘要：

在机器学习问题中，数据质量对系统预测的准确性产生了深远的影响。由于信息获取的难度大，人类的认知主观且有限，导致了专家无法准确标记所有样本。而近年来出现的一些概率抽样方法无法避免样本人为划分不合理且主观性较强的问题。针对这一问题，提出一种基于动态概率抽样（DPS）的标签噪声过滤方法，充分考虑各个数据集样本间的差异性，通过统计各个区间内置信度分布频率，分析各个区间内置信度分布信息熵的走势，确定合理阈值。在UCI经典数据集中选取了14个数据集，将所提方法与随机森林（RF）、HARF、MVF、局部概率抽样（LPS）等方法进行了对比实验。实验结果表明，所提出的方法在标签噪声识别和分类泛化上均展示出了较高的能力。

关键词: 标签噪声, 动态概率抽样, 噪声过滤, 标签置信度, 置信度

CLC Number:

TP181

Zenghui ZHANG, Gaoxia JIANG, Wenjian WANG. Label noise filtering method based on dynamic probability sampling[J]. Journal of Computer Applications, 2021, 41(12): 3485-3491.

张增辉, 姜高霞, 王文剑. 基于动态概率抽样的标签噪声过滤方法[J]. 《计算机应用》唯一官方网站, 2021, 41(12): 3485-3491.

Figures/Tables 9

Fig. 1 Confidence distribution in ideal condition

Fig. 2 Confidence distribution of different datasets

Tab. 1 Experimental dataset description

数据集	样本数	特征数	数据集	样本数	特征数
sonar	208	60	diabetic	1 151	20
heart	270	13	winequality_red	1 319	11
haberman	306	3	abalone	1 323	8
ionosphere	351	34	titanic	2 201	3
climate	540	18	splice	3 175	60
fourclass	862	2	winequality_white	3 655	11
germanNumer	1 000	24	banana	5 300	2

Fig. 3 First-order difference graphs of 14 datasets

Fig. 4 Comparison of AccRe and Pre of 5 algorithms

Fig. 5 Comparison of F1 of 5 algorithms

Fig. 6 Comparison of F1mean value of 5 algorithms

Fig. 7 Comparison of Classif of 5 algorithms

Fig. 8 Comparison of Classif mean value of 5 algorithms

References 15

1	GARCÍA-GIL D， LUENGO J， GARCÍA S， et al. Enabling smart data： noise filtering in big data classification［J］. Information Sciences， 2019， 479： 135-152. 10.1016/j.ins.2018.12.002
2	SUN J W， ZHAO F Y， WANG C J， et al. Identifying and correcting mislabeled training instances［C］// Proceedings of the 2007 Future Generation Communication and Networking. Piscataway： IEEE， 2007：244-250. 10.1109/fgcn.2007.146
3	FENG W， QUAN Y H， DAUPHIN G. Label noise cleaning with an adaptive ensemble method based on noise detection metric［J］. Sensors， 2020， 20（23）： No.6718. 10.3390/s20236718
4	姜高霞，樊瑞宣，王文剑. 近邻感知的标签噪声过滤算法［J］. 模式识别与人工智能， 2020， 33（6）： 518-529. 10.16451/j.cnki.issn1003-6059.202006005
	JIANG G X， FAN R X， WANG W J. Label noise filtering via perception of nearest nerghbors［J］. Pattern Recogniton and Artificial Intelligence， 2020， 33（6）： 518-529. 10.16451/j.cnki.issn1003-6059.202006005
5	SEGATA N， BLANZIERI E， DELANY S J， et al. Noise reduction for instance-based learning with a local maximal margin approach［J］. Journal of Intelligent Information Systems， 2010， 35（2）： 301-331. 10.1007/s10844-009-0101-z
6	GAMBERGER D， LAVRAČ N， GROŠELJ C. Experiments with noise filtering in a medical domain［C］// Proceedings of the 16th International Conference of Machine Learning. San Francisco： Morgan Kaufmann Publishers Inc.， 1999： 143-151. 10.1007/3-540-48720-4_39
7	CHEN H S， SHEN C S， HE G， et al. Critical noise of majority-vote model on complex networks［J］. Physical Review. E， Statistical， Nonlinear， and Soft Matter Physics， 2015， 91（2）： No.022816. 10.1103/physreve.91.022816
8	XIE S Y， GUO L. Analysis of normalized least mean squares-based consensus adaptive filters under a general information condition［J］. SIAM Journal on Control and Optimization， 2018， 56（5）： 3404-3431. 10.1137/16m1106791
9	SLUBAN B， GAMBERGER D， LAVRA N. Advances in class noise detection［C］// Proceedings of the 19th European Conference on Artificial Intelligence， Frontiers in Artificial Intelligence and Applications 215. Amsterdam： IOS Press， 2010： 1105-1106.
10	KHOSHGOFTAAR T M， REBOURS P. Improving software quality prediction by noise filtering techniques［J］. Journal of Computer Science and Technology， 2007， 22（3）： 387-396. 10.1007/s11390-007-9054-2
11	SÁEZ J A， GALAR M， LUENGO J， et al. INFFC： an iterative class noise filter based on the fusion of classifiers with noise sensitivity control［J］. Information Fusion， 2016， 27： 19-32. 10.1016/j.inffus.2015.04.002
12	ZHU X Q， WU X D， CHEN Q J. Eliminating class noise in large datasets［C］// Proceedings of the 20th International Conference on Machine Learning. Palo Alto， CA： AAAI Press， 2003： 920-927.
13	YUAN W W， GUAN D H， MA T H， et al. Classification with class noises through probabilistic sampling［J］. Information Fusion， 2018， 41： 57-67. 10.1016/j.inffus.2017.08.007
14	陈庆强，王文剑，姜高霞. 基于数据分布的标签噪声过滤［J］. 清华大学学报（自然科学版）， 2019， 59（4）： 262-269. 10.16511/j.cnki.qhdxxb.2018.26.059
	CHEN Q Q， WANG W J， JIANG G X. Label noise filtering based on the data distribution［J］. Journal of Tsinghua University （Science and Technology）， 2019， 59（4）： 262-269. 10.16511/j.cnki.qhdxxb.2018.26.059
15	张增辉，姜高霞，王文剑. 基于局部概率抽样的标签噪声过滤方法［J］. 计算机应用， 2021， 41（1）： 67-73.
	ZHANG Z H， JIANG G X， WANG W J. Label noise filtering method based on local probability sampling［J］. Journal of Computer Applications， 2019， 41（1）：67-73.

[1]	Yifei SONG, Yi LIU. Fast adversarial training method based on data augmentation and label noise [J]. Journal of Computer Applications, 2024, 44(12): 3798-3807.
[2]	Hao WANG, Zicheng WANG, Chao ZHANG, Yunsheng MA. Generative adversarial network based data uncertainty quantification method [J]. Journal of Computer Applications, 2023, 43(4): 1094-1101.
[3]	Boyi FU, Yuncong PENG, Xin LAN, Xiaolin QIN. Survey of label noise learning algorithms based on deep learning [J]. Journal of Computer Applications, 2023, 43(3): 674-684.
[4]	Chengyu LIN, Lei WANG, Cong XUE. Weakly-supervised text classification with label semantic enhancement [J]. Journal of Computer Applications, 2023, 43(2): 335-342.
[5]	Yizhen BI, Huan MA, Changqing ZHANG. Dynamic evaluation method for benefit of modality augmentation [J]. Journal of Computer Applications, 2023, 43(10): 3099-3106.
[6]	LIU Xiaoyu, CHEN Huaixin, LIU Biyuan, LIN Ying, MA Teng. License plate detection algorithm in unrestricted scenes based on adaptive confidence threshold [J]. Journal of Computer Applications, 2023, 43(1): 67-73.
[7]	Jinye LI, Ruizhang HUANG, Yongbin QIN, Yanping CHEN, Xiaoyu TIAN. Recognition of sentencing circumstances in adjudication documents based on abductive learning [J]. Journal of Computer Applications, 2022, 42(6): 1802-1807.
[8]	Hao FENG, Chaobing HUANG, Yuanqiao WEN. Remote sensing image small target detection based on improved YOLOv3 [J]. Journal of Computer Applications, 2022, 42(12): 3723-3732.
[9]	ZHANG Zenghui, JIANG Gaoxia, WANG Wenjian. Label noise filtering method based on local probability sampling [J]. Journal of Computer Applications, 2021, 41(1): 67-73.
[10]	CHENG Kai, WANG Yan, LIU Jianfei. Semi-supervised learning method for automatic nuclei segmentation using generative adversarial network [J]. Journal of Computer Applications, 2020, 40(10): 2917-2922.
[11]	YIN Yu, ZHAN Yongzhao, JIANG Zhen. Semi-supervised ensemble learning for video semantic detection based on pseudo-label confidence selection [J]. Journal of Computer Applications, 2019, 39(8): 2204-2209.
[12]	YANG Chunde, LIU Jing, QU Zhong. Fast scale adaptive object tracking algorithm with separating window [J]. Journal of Computer Applications, 2019, 39(4): 1145-1149.
[13]	YANG Wenxia, ZHANG Liang. Image inpainting model based on structure-texture decomposition and local total variation minimization [J]. Journal of Computer Applications, 2018, 38(8): 2386-2392.
[14]	CHEN Liu, FENG Shan. Two-level confidence threshold setting method for positive and negative association rules [J]. Journal of Computer Applications, 2018, 38(5): 1315-1319.
[15]	WANG Yaqiang, TANG Ming, ZENG Qin, TANG Dan, SHU Hongping. Cascaded and low-consuming online method for large-scale Web page category acquisition [J]. Journal of Computer Applications, 2017, 37(4): 924-927.

Label noise filtering method based on dynamic probability sampling

基于动态概率抽样的标签噪声过滤方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 15

Related Articles 15

Recommended Articles

Metrics