Domain adaptation integrating environment label smoothing and nuclear norm discrepancy

doi:10.11772/j.issn.1001-9081.2024040417

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (4): 1130-1138.DOI: 10.11772/j.issn.1001-9081.2024040417

• Artificial intelligence • Previous Articles Next Articles

Domain adaptation integrating environment label smoothing and nuclear norm discrepancy

Meirong DING, Jinxin ZHUO, Yuwu LU(), Qinglong LIU, Jicong LANG

School of Software，South China Normal University，Foshan Guangdong 528225，China

Received:2024-04-09 Revised:2024-06-27 Accepted:2024-07-02 Online:2025-04-08 Published:2025-04-10
Contact: Yuwu LU
About author:DING Meirong， born in 1972， M. S.， associate professor. Her research interests include artificial intelligence， natural language processing， intelligent software technology.
ZHUO Jinxin， born in 1999， M. S. candidate. His research interests include domain adaptation， transfer learning.
LIU Qinglong， born in 1999， M. S. candidate. His research interests include domain adaptation， transfer learning.
LANG Jicong， born in 1998， M. S. candidate. His research interests include domain adaptation， transfer learning.
Supported by:
This work is partially supported by National Natural Science Foundation of China （62176162）； Guangdong Provincial Natural Science Foundation（ 2022A1515140099， 2023A1515012875）.

融合环境标签平滑与核范数差异的领域自适应

丁美荣, 卓金鑫, 陆玉武(), 刘庆龙, 郎济聪

华南师范大学软件学院，广东佛山 528225

通讯作者: 陆玉武
作者简介:丁美荣（1972—），女，内蒙古巴彦淖尔人，副教授，硕士，CCF高级会员，主要研究方向：人工智能、自然语言处理、智能软件技术；
卓金鑫（1999—），男，广东揭阳人，硕士研究生，CCF会员，主要研究方向：领域自适应、迁移学习；
刘庆龙（1999—），男，河南驻马店人，硕士研究生，主要研究方向：领域自适应、迁移学习；
郎济聪（1998—），男，山东济宁人，硕士研究生，主要研究方向：领域自适应、迁移学习。
基金资助:
国家自然科学基金资助项目(62176162);广东省自然科学基金资助项目(2022A1515140099)

Abstract

Abstract:

The existing domain adaptation methods overly focus on fine-grained feature learning in the source domain， hindering their ability to extend to the target domain effectively， making them prone to overfitting in specific environments， and lacking robustness to complex environments. To address the above mentioned issues， a domain adaptation model that integrates Environment Label Smoothing and Nuclear norm Discrepancy （ELSND） was proposed. In the proposed model， through the environment label smoothing module， the probability of true labels was reduced and the probability of non-true labels was increased to enhance the model adaptability to different scenarios. At the same time， the nuclear norm discrepancy module was employed to measure distribution difference between the source and target domains， thereby improving the classification certainty at decision boundaries. Large number of experiments were conducted on adaptive benchmark datasets of three domains including Office-31， Office-Home and MiniDomainNet. Compared with the state-of-the-art baseline model DomainAdaptor-Aug （DomainAdaptor with generalized entropy minimization-Augmentation） on MiniDomainNet dataset， ELSND model achieves a 1.23 percentage points increase in accuracy of image classification domain adaptation tasks. Therefore， the proposed model has a higher precision and generalization in image classification.

Key words: domain adaptation, transfer learning, image classification, nuclear norm, adversarial learning

摘要：

现有的领域自适应方法过于关注源域的细粒度特征学习，从而削弱了相关方法有效推广到目标域的能力，导致这些方法容易在特定的环境中过度拟合，且缺乏对复杂环境的鲁棒性。为了解决上述问题，提出一个融合环境标签平滑与核范数差异（ELSND）的领域自适应模型。所提模型通过环境标签平滑模块，减小真实标签的概率，增大非真实标签的概率，进而增强模型对不同场景的适应性。同时，运用核范数差异模块度量源域与目标域的分布差异，从而提高决策边界处样本的分类确定性。在Office-31、Office-Home和MiniDomainNet这3个领域的自适应基准数据集上进行大量实验。结果表明，与先进的基线模型DomainAdaptor-Aug （DomainAdaptor with generalized entropy minimization-Augmentation）在MiniDomainNet数据集上相比，ELSND模型在图像分类领域自适应任务上的精确度提升了1.23个百分点。因此，所提模型在图像分类时具有更高的精确度和泛化性。

关键词: 领域自适应, 迁移学习, 图像分类, 核范数, 对抗学习

CLC Number:

TP391.4

Meirong DING, Jinxin ZHUO, Yuwu LU, Qinglong LIU, Jicong LANG. Domain adaptation integrating environment label smoothing and nuclear norm discrepancy[J]. Journal of Computer Applications, 2025, 45(4): 1130-1138.

丁美荣, 卓金鑫, 陆玉武, 刘庆龙, 郎济聪. 融合环境标签平滑与核范数差异的领域自适应[J]. 《计算机应用》唯一官方网站, 2025, 45(4): 1130-1138.

Figures/Tables 10

References 45

1	PEI Z， CAO Z， LONG M， et al. Multi-adversarial domain adaptation［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2018： 3934-3941.
2	ARJOVSKY M， CHINTALA S， BOTTOU L. Wasserstein generative adversarial networks［C］// Proceedings of the 34th International Conference on Machine Learning. New York： JMLR.org， 2017： 214-223.
3	MESCHEDER L， GEIGER A， NOWOZIN S. Which training methods for GANs do actually converge？［C］// Proceedings of the 35th International Conference on Machine Learning. New York： JMLR.org， 2018： 3481-3490.
4	ZHANG Y， WANG X， LIANG J， et al. Free lunch for domain adversarial training： environment label smoothing［EB/OL］. ［2024-06-25］. .
5	LONG M， CAO Z， WANG J， et al. Conditional adversarial domain adaptation［C］// Proceedings of the 32nd International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2018： 1647-1657.
6	GANIN Y， USTINOVA E， AJAKAN H， et al. Domain-adversarial training of neural networks［J］. Journal of Machine Learning Research， 2016， 17： 1-35.
7	KURMI V K， NAMBOODIRI V P. Looking back at labels： a class based domain adaptation technique［C］// Proceedings of the 2019 International Joint Conference on Neural Networks. Piscataway： IEEE， 2019： 1-8.
8	TANG H， JIA K. Discriminative adversarial domain adaptation［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2020： 5940-5947.
9	LU Z， YANG Y， ZHU X， et al. Stochastic classifiers for unsupervised domain adaptation［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 9108-9117.
10	LI S， LV F， XIE B， et al. Bi-classifier determinacy maximization for unsupervised domain adaptation［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 8455-8464.
11	GAO Z， ZHANG S， HUANG K， et al. Gradient distribution alignment certificates better adversarial domain adaptation［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2021： 8917-8926.
12	CHEN L， CHEN H， WEI Z， et al. Reusing the task-specific classifier as a discriminator： discriminator-free adversarial domain adaptation［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 7171-7180.
13	ZHANG Y， TANG H， JIA K， et al. Domain-symmetric networks for adversarial domain adaptation［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 5026-5035.
14	SHEN J， QU Y， ZHANG W， et al. Wasserstein distance guided representation learning for domain adaptation［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2018： 4058-4065.
15	刘皓，洪宇，朱巧明. 无监督的领域自适应机器阅读理解方法［J］. 计算机学报， 2022， 45（10）： 2133-2150.
	LIU H， HONG Y， ZHU Q M. Unsupervised domain adaptive machine reading comprehension method［J］. Chinese Journal of Computers， 2022， 45（10）： 2133-2150.
16	LONG M， CAO Y， WANG J， et al. Learning transferable features with deep adaptation networks［C］// Proceedings of the 32nd International Conference on Machine Learning. New York： JMLR.org， 2015： 97-105.
17	TZENG E， HOFFMAN J， ZHANG N， et al. Deep domain confusion： maximizing for domain invariance［EB/OL］. ［2024-01-01］..
18	ZHANG Y， LIU T， LONG M， et al. Bridging theory and algorithm for domain adaptation［C］// Proceedings of the 36th International Conference on Machine Learning. New York： JMLR.org， 2019： 7404-7413.
19	DU Z， LI J， SU H， et al. Cross-domain gradient discrepancy minimization for unsupervised domain adaptation［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern recognition. Piscataway： IEEE， 2021： 3936-3945.
20	LONG M， ZHU H， WANG J， et al. Deep transfer learning with joint adaptation networks［C］// Proceedings of the 34th International Conference on Machine Learning. New York： JMLR.org， 2017： 2208-2217.
21	ZHAO S， YUE X， ZHANG S， et al. A review of single-source deep unsupervised visual domain adaptation［J］. IEEE Transactions on Neural Networks and Learning Systems， 2022， 33（2）： 473-493.
22	GOODFELLOW I J， POUGET-ABADIE J， MIRZA M， et al. Generative adversarial nets［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems — Volume 2. Cambridge： MIT Press， 2014：2672-2680.
23	MIRZA M， OSINDERO S. Conditional generative adversarial nets［EB/OL］. ［2024-01-01］..
24	SAITO K， WATANABE K， USHIKU Y， et al. Maximum classifier discrepancy for unsupervised domain adaptation［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 3723-3732.
25	LEE C Y， BATRA T， BAIG M H， et al. Sliced Wasserstein discrepancy for unsupervised domain adaptation［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 10277-10287.
26	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2818-2826.
27	WARDE-FARLEY D， GOODFELLOW I. Adversarial perturbations of deep neural networks［M］// HAZAQN T， PAPANDREOU G， TARLOW D. Perturbations， optimization， and statistics. Cambridge： MIT Press， 2016： 311-332.
28	MÜLLER R， KORNBLITH S， HINTON G E. When does label smoothing help？［C］// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2019： 4694-4703.
29	LUKASIK M， BHOJANAPALLI S， MENON A K， et al. Does label smoothing mitigate label noise？［C］// Proceedings of the 37th International Conference on Machine Learning. New York： JMLR.org， 2020： 6448-6458.
30	RECHT B， FAZEL M， PARRILO P A. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization［J］. SIAM Review， 2010， 52（3）： 471-501.
31	GU S， ZHANG L， ZUO W， et al. Weighted nuclear norm minimization with application to image denoising［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2014： 2862-2869.
32	CUI S， WANG S， ZHUO J， et al. Towards discriminability and diversity： batch nuclear-norm maximization under label insufficient situations［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 3940-3949.
33	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778.
34	SAENKO K， KULIS B， FRITZ M， et al. Adapting visual category models to new domains［C］// Proceedings of the 2010 European Conference on Computer Vision， LNCS 6314. Berlin： Springer， 2010： 213-226.
35	VENKATESWARA H， EUSEBIO J， CHAKRABORTY S， et al. Deep hashing network for unsupervised domain adaptation［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5385-5394.
36	ZHOU K， YANG Y， QIAO Y， et al. Domain adaptive ensemble learning［J］. IEEE Transactions on Image Processing， 2021， 30： 8008-8018.
37	RANGWANI H， AITHAL S K， MISHRA M， et al. A closer look at smoothness in domain adversarial training［C］// Proceedings of the 39th International Conference on Machine Learning. New York： JMLR.org， 2022： 18378-18399.
38	VAPNIK V N. The nature of statistical learning theory［M］. New York： Springer， 2000.
39	TZENG E， HOFFMAN J， SAENKO K， et al. Adversarial discriminative domain adaptation［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2962-2971.
40	JIN Y， WANG X， LONG M， et al. Minimum class confusion for versatile domain adaptation［C］// Proceedings of the 2020 European Conference on Computer Vision， LNCS 12366. Cham： Springer， 2020： 464-480.
41	ACUNA D， ZHANG G， LAW M T， et al. f-Domain adversarial learning： theory and algorithms［C］// Proceedings of the 38th International Conference on Machine Learning. New York： JMLR.org， 2021： 66-75.
42	TANG H， CHEN K， JIA K. Unsupervised domain adaptation via structurally regularized deep clustering［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 8722-8732.
43	XU R， CHEN Z， ZUO W， et al. Deep cocktail network： multi-source unsupervised domain adaptation with category shift［C］// Proceedings of the 2018 IEEECVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 3964-3973.
44	SAITO K， KIM D， SCLAROFF S， et al. Semi-supervised domain adaptation via minimax entropy［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 8049-8057.
45	ZHANG J， QI L， SHI Y， et al. DomainAdaptor： a novel approach to test-time adaptation［C］// Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2023： 18925-18935.
	Found：This work is partially supported by National Natural Science Foundation of China （62176162）； Guangdong Provincial Natural Science Foundation（2022A1515140099， 2023A1515012875）.

SDAT	ELS	NWD	平均精确度
√			71.8
√	√		72.6
√	√	√	73.0

SDAT	ELS	NWD	平均精确度
√			71.8
√	√		72.6
√	√	√	73.0

模型	A→W	D→W	W→D	A→D	D→A	W→A	平均值
ERM^［38］	75.8	95.5	99.0	79.3	63.6	63.8	79.5
ADDA^［39］	94.6	97.5	99.7	90.0	69.6	72.5	87.3
CDAN^［5］	93.8	98.5	100.0	89.9	73.4	70.4	87.7
MCC^［40］	94.1	98.4	99.8	95.6	75.5	74.2	89.6
DANN^［6］	91.3	97.2	100.0	84.1	72.9	73.6	86.5
DANN+ELS^［4］	92.2	98.5	100.0	85.9	74.3	75.3	87.7
SDAT^［37］	92.7	98.9	100.0	93.0	78.5	75.7	89.8
SDAT+ELS^［4］	93.6	99.0	100.0	93.4	78.7	77.5	90.4
ELSND	94.3	99.2	100.0	93.8	78.9	78.0	90.7

模型	A→W	D→W	W→D	A→D	D→A	W→A	平均值
ERM^［38］	75.8	95.5	99.0	79.3	63.6	63.8	79.5
ADDA^［39］	94.6	97.5	99.7	90.0	69.6	72.5	87.3
CDAN^［5］	93.8	98.5	100.0	89.9	73.4	70.4	87.7
MCC^［40］	94.1	98.4	99.8	95.6	75.5	74.2	89.6
DANN^［6］	91.3	97.2	100.0	84.1	72.9	73.6	86.5
DANN+ELS^［4］	92.2	98.5	100.0	85.9	74.3	75.3	87.7
SDAT^［37］	92.7	98.9	100.0	93.0	78.5	75.7	89.8
SDAT+ELS^［4］	93.6	99.0	100.0	93.4	78.7	77.5	90.4
ELSND	94.3	99.2	100.0	93.8	78.9	78.0	90.7

模型	Ar→Cl	Ar→Pr	Ar→Rw	Cl→Ar	Cl→Pr	Cl→Rw	Pr→Ar	Pr→Cl	Pr→Rw	Rw→Ar	Rw→Cl	Rw→Pr	平均值
ResNet-50^［33］	34.9	50.0	58.0	37.4	41.9	46.2	38.5	31.2	60.4	53.9	41.2	59.9	46.1
DANN^［6］	45.6	59.3	70.1	47.0	58.5	60.9	46.1	43.7	68.5	63.2	51.8	76.8	57.6
CDAN^［5］	49.0	69.3	74.5	54.4	66.0	68.4	55.6	48.3	75.9	68.4	55.4	80.5	63.8
MMD^［17］	54.9	73.7	77.8	60.0	71.4	71.8	61.2	53.6	78.1	72.5	60.2	82.3	68.1
f-DAL^［41］	56.7	77.0	81.1	63.1	72.2	75.9	64.5	54.4	81.0	72.3	58.4	83.7	70.0
SRDC^［42］	52.3	76.3	81.0	69.5	76.2	78.0	68.7	53.8	81.7	76.3	57.1	85.0	71.3
SDAT^［37］	57.8	77.4	82.2	66.5	76.6	76.2	63.3	57.0	82.2	75.3	62.6	85.2	71.8
SDAT+ELS^［4］	58.2	79.7	82.5	67.5	77.2	77.2	64.6	57.9	82.2	75.4	63.1	85.5	72.6
ELSND	58.5	79.9	82.6	67.9	77.8	77.5	65.3	58.7	82.0	76.6	64.2	85.6	73.0

Domain adaptation integrating environment label smoothing and nuclear norm discrepancy

融合环境标签平滑与核范数差异的领域自适应

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 45

Related Articles 15

Recommended Articles

Metrics

模型	Clp	Pnt	Rel	Skt	平均值
DANN^［6］	65.55	46.27	58.68	47.88	54.60
DCTN^［43］	62.06	48.79	58.85	48.25	54.49
MCD^［24］	62.91	45.77	57.57	45.88	53.03
MME^［44］	68.09	47.14	63.33	43.50	55.52
DAEL^［36］	69.95	55.13	66.11	55.72	61.73
Oracle^［36］	72.59	60.53	80.47	63.44	69.26
DomainAdaptor-Aug^［45］	—	—	—	—	73.82
ELSND	76.50	73.50	79.90	70.30	75.05

[1]	Liwei ZHANG, Quan LIANG, Yutao HU, Qiaole ZHU. Channel shuffle attention mechanism based on group convolution [J]. Journal of Computer Applications, 2025, 45(4): 1069-1076.
[2]	Junxiu AN, Linwang YANG, Yuan LIU. Unsupervised text style transfer based on semantic perception of proximity [J]. Journal of Computer Applications, 2025, 45(4): 1139-1147.
[3]	Yiqin YAN, Chuan LUO, Tianrui LI, Hongmei CHEN. Cross-domain few-shot classification model based on relation network and Vision Transformer [J]. Journal of Computer Applications, 2025, 45(4): 1095-1103.
[4]	Danni DING, Bo PENG, Xi WU. VPNet： fatty liver ultrasound image classification method inspired by ventral pathway [J]. Journal of Computer Applications, 2025, 45(2): 662-669.
[5]	Xuewen YAN, Zhangjin HUANG. Few-shot image classification method based on contrast learning [J]. Journal of Computer Applications, 2025, 45(2): 383-391.
[6]	Chunxia LIU, Hanying XU, Gaimei GAO, Weichao DANG, Zilu LI. Smart contract vulnerability detection method based on echo state network [J]. Journal of Computer Applications, 2025, 45(1): 153-161.
[7]	Zongsheng ZHENG, Jia DU, Yuhe CHENG, Zecheng ZHAO, Yuewei ZHANG, Xulong WANG. Cross-modal dual-stream alternating interactive network for infrared-visible image classification [J]. Journal of Computer Applications, 2025, 45(1): 275-283.
[8]	Dongwei WANG, Baichen LIU, Zhi HAN, Yanmei WANG, Yandong TANG. Deep network compression method based on low-rank decomposition and vector quantization [J]. Journal of Computer Applications, 2024, 44(7): 1987-1994.
[9]	Feiyu ZHAI, Handa MA. Hybrid classical-quantum classification model based on DenseNet [J]. Journal of Computer Applications, 2024, 44(6): 1905-1910.
[10]	Wangjun SHI, Jing WANG, Xiaojun NING, Youfang LIN. Sleep stage classification model by meta transfer learning in few-shot scenarios [J]. Journal of Computer Applications, 2024, 44(5): 1445-1451.
[11]	Hongtian LI, Xinhao SHI, Weiguo PAN, Cheng XU, Bingxin XU, Jiazheng YUAN. Few-shot object detection via fusing multi-scale and attention mechanism [J]. Journal of Computer Applications, 2024, 44(5): 1437-1444.
[12]	Bin XIAO, Mo YANG, Min WANG, Guangyuan QIN, Huan LI. Domain generalization method of phase-frequency fusion from independent perspective [J]. Journal of Computer Applications, 2024, 44(4): 1002-1009.
[13]	Haoran WANG, Dan YU, Yuli YANG, Yao MA, Yongle CHEN. Domain transfer intrusion detection method for unknown attacks on industrial control systems [J]. Journal of Computer Applications, 2024, 44(4): 1158-1165.
[14]	Xue LI, Guangle YAO, Honghui WANG, Jun LI, Haoran ZHOU, Shaoze YE. Remote sensing image classification based on sample incremental learning [J]. Journal of Computer Applications, 2024, 44(3): 732-736.
[15]	Xuan CAO, Tianjian LUO. Dynamic multi-domain adversarial learning method for cross-subject motor imagery EEG signals [J]. Journal of Computer Applications, 2024, 44(2): 645-653.