Open set fuzzy domain adaptation algorithm via progressive separation

doi:10.11772/j.issn.1001-9081.2021010061

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (11): 3127-3131.DOI: 10.11772/j.issn.1001-9081.2021010061

• Artificial intelligence • Previous Articles Next Articles

Open set fuzzy domain adaptation algorithm via progressive separation

Xiaolong LIU¹(), Shitong WANG²

^1.School of Artificial Intelligence and Computer Science，Jiangnan University，Wuxi Jiangsu 214122，China
^2.Jiangsu Key Laboratory of Media Design and Software Technology （Jiangnan University），Wuxi Jiangsu 214122，China

Received:2021-01-13 Revised:2021-03-25 Accepted:2021-03-30 Online:2021-04-15 Published:2021-11-10
Contact: Xiaolong LIU
About author:LIU Xiaolong， born in 1995， M. S. candidate. His research interests include artificial intelligence， pattern recognition， machine learning.
WANG Shitong，born in 1964，Ph. D.，professor. His research interests include artificial intelligence，pattern recognition，fuzzy system.
Supported by:
the National Natural Science Foundation of China(61572236)

渐进式分离的开放集模糊域自适应算法

刘晓龙¹(), 王士同²

^1.江南大学人工智能与计算机学院，江苏无锡 214122
^2.江苏省媒体设计与软件技术重点实验室（江南大学），江苏无锡 214122

通讯作者: 刘晓龙
作者简介:刘晓龙（1995—），男，山东潍坊人，硕士研究生，主要研究方向：人工智能、模式识别、机器学习
王士同（1964—），男，江苏扬州人，教授，博士生导师，博士，CCF会员，主要研究方向：人工智能、模式识别、模糊系统。
基金资助:
国家自然科学基金资助项目(61572236)

Abstract

Abstract:

The aim of domain adaptation is to use the knowledge in a labeled （source） domain to improve the model classification performance of an unlabeled （target） domain， and this method has achieved good results. However， in the open realistic scenes， the target domain usually contains unknown classes that are not observed in the source domain， which is called open set domain adaptation problem. For such challenging scene setting， the traditional domain adaptation algorithm is powerless. Therefore， an open set fuzzy domain adaptation algorithm via progressive separation was proposed. Firstly， based on the open set fuzzy domain adaptation algorithm with membership degree introduced， the method of separating the known and unknown class samples in the target domain step by step was explored. Then， only known classes separated from the target domain were aligned with the source domain， so as to reduce the distribution difference between the two domains and perform the fuzzy adaptation. The negative transfer effect caused by the mismatch between unknown and known classes was reduced well by the proposed algorithm. Six domain transformation experimental results on Office dataset show that， the accuracy of the proposed algorithm has the significant improvement in image classification compared with the traditional domain adaptation algorithm， and verify that the proposed algorithm can gradually enhance the accuracy and robustness of the domain adaptation classification model.

Key words: machine learning, open set, fuzzy domain adaptation, progressive separation, transfer learning

摘要：

域自适应的目的是利用有标记（源）域中的信息来提高未标记（目标）域模型的分类性能，且这种方法已经取得了不错的成果。然而在具有开放性的现实场景下，目标域通常包含源域中未观察到的未知类样本，这被称为开放集域自适应问题。传统的域自适应算法对这样具有挑战性的场景设定无能为力，因此提出了渐进式分离的开放集模糊域自适应算法。首先，基于引进隶属度的开放集模糊域自适应算法，探索了逐步分离目标域中已知类和未知类样本的方法；然后，仅将从目标域中分离出的已知类与源域对齐，从而减小两个域之间的分布差异，进行模糊域自适应。所提算法很好地解决了由于未知类和已知类之间的不匹配而导致的负迁移所带来的影响。在Office数据集上的6组域自适应转化实验结果表明，与传统的域自适应算法比较，所提算法在图像分类中的精度有显著的提升，验证了该算法可以逐步增强域自适应分类模型的准确性和鲁棒性。

关键词: 机器学习, 开放集, 模糊域自适应, 渐进式分离, 迁移学习

CLC Number:

TP181

Xiaolong LIU, Shitong WANG. Open set fuzzy domain adaptation algorithm via progressive separation[J]. Journal of Computer Applications, 2021, 41(11): 3127-3131.

刘晓龙, 王士同. 渐进式分离的开放集模糊域自适应算法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3127-3131.

Figures/Tables 5

Fig. 1 Open set domain adaptation

Fig. 2 Process description of open set fuzzy domain adaptation algorithm via progressive separation

Tab. 1 Accuracy comparison of 6 Office dataset domain transformation experiments under close-set condition

算法	A→D	A→W	D→A	D→W	W→A	W→D	平均值
$C o v N e t s r c$	48.84	49.41	65.20	67.20	41.20	76.40	58.42
DAN	88.12	90.05	93.40	96.10	84.90	99.36	91.98
DANN	75.50	60.57	62.70	88.90	63.00	98.32	75.17
SVM	83.44	78.30	80.06	93.56	75.68	98.09	84.85
CORAL	79.62	77.29	81.63	95.93	69.31	99.36	83.86
GFK	80.25	77.29	77.14	90.17	90.17	99.36	82.44
BDA	87.26	82.37	88.72	96.64	96.64	99.36	90.13
SFDA	94.90	89.85	91.86	96.95	90.65	99.36	93.93

Tab. 1 Accuracy comparison of 6 Office dataset domain transformation experiments under close-set condition

算法	A→D	A→W	D→A	D→W	W→A	W→D	平均值
$C o v N e t s r c$	48.84	49.41	65.20	67.20	41.20	76.40	58.42
DAN	88.12	90.05	93.40	96.10	84.90	99.36	91.98
DANN	75.50	60.57	62.70	88.90	63.00	98.32	75.17
SVM	83.44	78.30	80.06	93.56	75.68	98.09	84.85
CORAL	79.62	77.29	81.63	95.93	69.31	99.36	83.86
GFK	80.25	77.29	77.14	90.17	90.17	99.36	82.44
BDA	87.26	82.37	88.72	96.64	96.64	99.36	90.13
SFDA	94.90	89.85	91.86	96.95	90.65	99.36	93.93

Tab. 2 Means and variances of accuracies of 6 Office dataset domain transformation experiments with different open degrees

开放性 $Θ$	A→D		A→W			D→A		D→W		W→A		W→D		平均值
开放性 $Θ$	均值	方差	均值	方差		均值	方差	均值	方差	均值	方差	均值	方差	平均值
0.2	95.33	±0.00	88.44	±1.12		91.03	±0.89	91.38	±0.81	91.09	±0.46	95.67	±0.72	92.16
0.4	95.80	±0.08	89.21	±0.95		92.99	±0.54	89.89	±0.31	91.86	±0.63	96.29	±0.08	91.61
0.5	96.22	±0.11	89.30	±1.44		91.97	±1.42	91.37	±0.48	90.18	±1.70	95.81	±0.11	92.48
0.6	96.40	±0.07	89.32	±1.68		90.70	±1.64	91.43	±0.52	90.68	±1.49	95.66	±0.59	91.66
0.8	96.79	±0.07	92.67	±0.36		92.75	±1.90	90.95	±0.53	92.45	±1.58	95.45	±0.08	92.76
1.0	94.28	±0.00	93.79		±0.00	90.65	±0.00	92.20	±0.00	90.19	±0.00	96.10	±0.00	92.87

Tab. 2 Means and variances of accuracies of 6 Office dataset domain transformation experiments with different open degrees

开放性 $Θ$	A→D		A→W			D→A		D→W		W→A		W→D		平均值
开放性 $Θ$	均值	方差	均值	方差		均值	方差	均值	方差	均值	方差	均值	方差	平均值
0.2	95.33	±0.00	88.44	±1.12		91.03	±0.89	91.38	±0.81	91.09	±0.46	95.67	±0.72	92.16
0.4	95.80	±0.08	89.21	±0.95		92.99	±0.54	89.89	±0.31	91.86	±0.63	96.29	±0.08	91.61
0.5	96.22	±0.11	89.30	±1.44		91.97	±1.42	91.37	±0.48	90.18	±1.70	95.81	±0.11	92.48
0.6	96.40	±0.07	89.32	±1.68		90.70	±1.64	91.43	±0.52	90.68	±1.49	95.66	±0.59	91.66
0.8	96.79	±0.07	92.67	±0.36		92.75	±1.90	90.95	±0.53	92.45	±1.58	95.45	±0.08	92.76
1.0	94.28	±0.00	93.79		±0.00	90.65	±0.00	92.20	±0.00	90.19	±0.00	96.10	±0.00	92.87

Fig. 3 Algorithm accuracy in each iteration with process of open degree Θ=1.0

References 25

1	ADAMS N. Dataset shift in machine learning ［J］. Journal of the Royal Statistical Society， 2010， 173（1）：274-274. 10.1111/j.1467-985x.2009.00624_10.x
2	LU J， BEHBOOD V， HAO P， et al. Transfer learning using computational intelligence： a survey ［J］. Knowledge-Based Systems， 2015， 80： 14-23. 10.1016/j.knosys.2015.01.010
3	MOTIION S， PICCIRILLI M， ADJEROH D A， et al. Unified deep supervised domain adaptation and generalization ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 5715-5725. 10.1109/iccv.2017.609
4	WANG J D， CHEN Y Q， HAO S J， et al. Balanced distribution adaptation for transfer learning ［C］// Proceedings of the 2017 IEEE International Conference on Data Mining. Piscataway： IEEE， 2017：1129-1134. 10.1109/icdm.2017.150
5	LONG M S， WANG J M， DING G G， et al. Transfer feature learning with joint distribution adaptation ［C］// Proceedings of the 2013 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2013：2200-2207. 10.1109/iccv.2013.274
6	BUSTO P P， GALL J. Open set domain adaptation ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017：754-763. 10.1109/iccv.2017.88
7	唐宋，陈利娟，陈志贤，等.基于目标域局部近邻几何信息的域自适应图像分类方法［J］.计算机应用，2017，37（4）：1164-1168. 10.11772/j.issn.1001-9081.2017.04.1164
	TANG S， CHEN L J， CHEN Z X， et al. Domain adaptation image classification based on target local-neighbor geometrical information ［J］. Journal of Computer Applications， 2017， 37（4）： 1164-1168. 10.11772/j.issn.1001-9081.2017.04.1164
8	GANIN Y， USTINOVA E， AJAKAN H， et al. Domain-adversarial training of neural networks ［J］. Journal of Machine Learning Research， 2016， 17： 1-35. 10.1007/978-3-319-58347-1_10
9	SUN B C， FENG J S， SAENKO K. Return of frustratingly easy domain adaptation ［C］// Proceedings of the 2016 30th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2016： 2058-2065.
10	LONG M S， CAO Y， WANG J M， et al. Learning transferable features with deep adaptation networks ［C］// Proceedings of the 2015 32nd International Conference on Machine Learning. New York： ACM， 2015： 97-105.
11	BELKIN M， NIYOGI P， SINDHWANI V. Manifold regularization： a geometric framework for learning from labeled and unlabeled examples ［J］. Journal of Machine Learning Research， 2006， 7： 2399-2434.
12	LONG M S， WANG J M， DING G G， et al. Adaptation regularization： a general framework for transfer learning ［J］. IEEE Transactions on Knowledge and Data Engineering， 2014， 26（5）： 1076-1089. 10.1109/tkde.2013.111
13	BAKTASHMOTLAGH M， FARAKI M， DRUMMOND T， et al. Learning factorized representations for open-set domain adaptation ［EB/OL］. ［2020-12-09］. .
14	PAN S J， YANG Q. A survey on transfer learning ［J］. IEEE Transactions on Knowledge and Data Engineering， 2010， 22（10）： 1345-1359. 10.1109/tkde.2009.191
15	刘晓龙，王士同.面向开放集图像分类的模糊域自适应方法［J］.计算机科学与探索，2021，15（3）：515-523. 10.1049/ipr2.12266
	LIU X L， WANG S T. Fuzzy domain adaptation algorithm for open set image classification ［J］. Journal of Frontiers of Computer Science and Technology， 2021， 15（3）： 515-523. 10.1049/ipr2.12266
16	SAITO K， YAMAMOTO S， USHIKU Y， et al. Open set domain adaptation by backpropagation ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11209. Cham： Springer， 2018： 156-171.
17	LIU H， CAO Z J， LONG M S， et al. Separate to adapt： open set domain adaptation via progressive separation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 2922-2931. 10.1109/cvpr.2019.00304
18	SCHEIRER W J， DE REZENDE ROCHA A， SAPKOTA A， et al. Toward open set recognition ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2013， 35（7）： 1757-1772. 10.1109/tpami.2012.256
19	王骏，王士同.基于混合距离学习的双指数模糊C均值算法［J］.软件学报，2010，21（8）：1878-1888. 10.3724/SP.J.1001.2010.03607
	WANG J， WANG S T. Double indices FCM algorithm based on hybrid distance metric learning ［J］. Journal of Software， 2010， 21（8）： 1878-1888. 10.3724/SP.J.1001.2010.03607
20	ZUO H， ZHANG G Q， PEDRYCZ W， et al. Fuzzy regression transfer learning in Takagi-Sugeno fuzzy models ［J］. IEEE Transactions on Fuzzy Systems， 2017， 25（6）： 1795-1807. 10.1109/tfuzz.2016.2633376
21	SAENKO K， KULIS B， FRITZ M， et al. Adapting visual category models to new domains ［C］// Proceedings of the 2010 European Conference on Computer Vision， LNCS6314. Berlin： Springer， 2010： 213-226.
22	GONG B Q， SHI Y， SHA F， et al. Geodesic flow kernel for unsupervised domain adaptation ［C］// Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2012： 2066-2073. 10.1109/cvpr.2012.6247911
23	FERNANDO B， HABRARD A， SEBBAN M， et al. Unsupervised visual domain adaptation using subspace alignment ［C］// Proceedings of the 2013 IEEE International Conference on Computer Vision. PiscatawayIEEE， 2013： 2960-2967. 10.1109/iccv.2013.368
24	史荧中，王士同，蒋亦樟，等.迁移学习支持向量回归机［J］.计算机应用，2013，33（11）：3084-3089. 10.11772/j.issn.1001-9081.2013.11.3084
	SHI Y Z， WANG S T， JIANG Y Z， et al. Transfer learning support vector regression ［J］. Journal of Computer Applications， 2013， 33（11）： 3084-3089. 10.11772/j.issn.1001-9081.2013.11.3084
25	GOPALAN R， LI R N， CHELLAPPA R. Domain adaptation for object recognition： an unsupervised approach ［C］// Proceedings of the 2011 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2011： 999-1006. 10.1109/iccv.2011.6126344

[1]	GUO Mian, ZHANG Jinyou. Computation offloading policy for machine learning in mobile edge computing environments [J]. Journal of Computer Applications, 2021, 41(9): 2639-2645.
[2]	MAO Mingze, CAO Ruihao, YAN Chungang. Semi-supervised classification algorithm based on weight diversity [J]. Journal of Computer Applications, 2021, 41(9): 2473-2480.
[3]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[4]	CHEN Zhengtao, HUANG Can, YANG Bo, ZHAO Li, LIAO Yong. Yak face recognition algorithm of parallel convolutional neural network based on transfer learning [J]. Journal of Computer Applications, 2021, 41(5): 1332-1336.
[5]	JIANG Qianyu, WANG Fengying, JIA Lipeng. Malware detection method based on perceptual hash algorithm and feature fusion [J]. Journal of Computer Applications, 2021, 41(3): 780-785.
[6]	WANG Jinkai, JIA Xu. Vein recognition algorithm based on Siamese nonnegative matrix factorization with transferability [J]. Journal of Computer Applications, 2021, 41(3): 898-903.
[7]	QIN Jing, ZUO Changqing, WANG Zumin, JI Changqing, WANG Baofeng. Design of abnormal electrocardiograph monitoring model based on stacking classifier [J]. Journal of Computer Applications, 2021, 41(3): 887-890.
[8]	MENG Xiangrui, YANG Wenzhong, WANG Ting. Survey of sentiment analysis based on image and text fusion [J]. Journal of Computer Applications, 2021, 41(2): 307-317.
[9]	Jianfang CAO, Minmin YAN, Yiming JIA, Xiaodong TIAN. Application of Inception-v3 model integrated with transfer learning in dynasty identification of ancient murals [J]. Journal of Computer Applications, 2021, 41(11): 3219-3227.
[10]	Haojie LOU, Yuanlin ZHENG, Kaiyang LIAO, Hao LEI, Jia LI. Defect target detection for printed matter based on Siamese-YOLOv4 [J]. Journal of Computer Applications, 2021, 41(11): 3206-3212.
[11]	Jin ZHAO, Wen’ai SONG, Jun TAI, Jijiang YANG, Qing WANG, Xiaodan LI, Yi LEI, Yue QIU. Review of computer-aided face diagnosis for obstructive sleep apnea in children [J]. Journal of Computer Applications, 2021, 41(11): 3394-3401.
[12]	WANG Yahui, QIAN Yuhua, LIU Guoqing. Ordinal decision tree algorithm based on fuzzy advantage complementary mutual information [J]. Journal of Computer Applications, 2021, 41(10): 2785-2792.
[13]	JIANG Yangsheng, WANG Shengnan, TU Jiaqi, LI Sha, WANG Hongjun. Comprehensive prediction of thermal comfort and energy consumption for high-speed railway stations [J]. Journal of Computer Applications, 2021, 41(1): 249-257.
[14]	HUANG Xueyu, XU Haote, TAO Jianwen. Multi-source adaptation classification framework with feature selection [J]. Journal of Computer Applications, 2020, 40(9): 2499-2506.
[15]	ZHANG Chenxi, TANG Shu, TANG Ke. Strategies of parameter fault detection for rocket engines based on transfer learning [J]. Journal of Computer Applications, 2020, 40(9): 2774-2780.

Open set fuzzy domain adaptation algorithm via progressive separation

渐进式分离的开放集模糊域自适应算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 25

Related Articles 15

Recommended Articles

Metrics