Multi-source and multi-label pedestrian attribute recognition based on domain adaptation

doi:10.11772/j.issn.1001-9081.2021060950

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (8): 2401-2406.DOI: 10.11772/j.issn.1001-9081.2021060950

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Multi-source and multi-label pedestrian attribute recognition based on domain adaptation

Nanjiang CHENG¹^,², Zhenxia YU¹(), Lin CHEN², Hezhe QIAO²

^1.School of Computer Science，Chengdu University of Information Technology，Chengdu Sichuan 610225，China
^2.Chongqing Institute of Green and Intelligent Technology，Chinese Academy of Sciences，Chongqing 400714，China

Received:2021-06-07 Revised:2021-08-17 Accepted:2021-09-02 Online:2022-08-09 Published:2022-08-10
Contact: Zhenxia YU
About author:CHENG Nanjiang， born in 1997， M. S. candidate. His research interests include machine learning， computer vision.
YU Zhenxia， born in 1968， Ph. D.， associate professor. Her research interests include machine learning， computer vision.
CHEN Lin， born in 1985， Ph. D. His research interests include computer vision， pattern recognition.
QIAO Hezhe， born in 1998， M. S. candidate. His research interests include computer vision， big data.
Supported by:
National Key Research and Development Program of China(2020YFC0833406)

基于领域自适应的多源多标签行人属性识别

程南江¹^,², 余贞侠¹(), 陈琳², 乔贺辙²

^1.成都信息工程大学计算机学院，成都 610225
^2.中国科学院重庆绿色智能技术研究院，重庆 400714

通讯作者: 余贞侠
作者简介:程南江（1997—），男，四川内江人，硕士研究生，主要研究方向：机器学习、计算机视觉；
余贞侠（1968—），女，江苏南京人，副教授，博士，主要研究方向：机器学习、计算机视觉；
陈琳（1985—），男，重庆人，博士，主要研究方向：计算机视觉、模式识别；
乔贺辙（1998—），男，甘肃天水人，硕士研究生，主要研究方向：计算机视觉、大数据。
基金资助:
国家重点研发计划项目(2020YFC0833406)

Abstract

Abstract:

The current public datasets of Pedestrian Attribute Recognition （PAR） have the characteristics of complicated attribute annotations and various collection scenarios， leading to the large variations of the pedestrian attributes in different datasets， so that it is hard to directly utilize the existing labeled information in the public datasets for PAR in practice. To address this issue， a multi-source and multi-label PAR method based on domain adaptation was proposed. Firstly， to transfer the styles of the different datasets into a unified one， the features of the samples were aligned by the domain adaption method. Then， a multi-attribute one-hot coding and weighting algorithm was proposed to align the labels with the common attribute in multiple datasets. Finally， the multi-label semi-supervised loss function was combined to perform joint training across datasets to improve the attribute recognition accuracy. The proposed feature alignment and label alignment algorithms were able to effectively solve the heterogeneity problem of attributes in multiple PAR datasets. Experimental results after aligning three pedestrian attribute datasets PETA， RAPv1 and RAPv2 with PA-100K dataset show that the proposed method improves the average accuracy by 1.22 percentage points， 1.62 percentage points and 1.53 percentage points respectively， compared to the method StrongBaseline， demonstrating that this method has a strong advantage in cross dataset PAR.

Key words: Pedestrian Attribute Recognition (PAR), multi-label learning, domain adaption, deep learning, Convolutional Neural Network (CNN)

摘要：

当前行人属性识别（PAR）公开数据集中属性标注繁杂且采集场景多样，各数据集中行人属性差异较大，进而导致公开数据库已有的标记信息数据难以直接应用到PAR实际问题中。针对上述问题，提出一种基于领域自适应的多源多标签PAR方法。首先通过领域自适应方法对样本进行特征对齐完成多个数据集之间的统一风格转换；接着提出多属性one-hot编码加权算法，将多数据集中共有属性的标签对齐；最后结合多标签半监督损失函数，进行跨数据集联合训练以提高属性识别准确率。通过所提出的特征对齐和标签对齐算法，可有效解决PAR多数据集中属性异构性问题。将三个行人属性数据集PETA、RAPv1和RAPv2分别与PA-100K数据集对齐后的实验结果表明，所提出的方法对比StrongBaseline在平均准确率上分别提升了1.22、1.62和1.53个百分点，说明该方法在跨数据集PAR中具有一定的优势。

关键词: 行人属性识别, 多标签学习, 领域自适应, 深度学习, 卷积神经网络

CLC Number:

TP391.4

Nanjiang CHENG, Zhenxia YU, Lin CHEN, Hezhe QIAO. Multi-source and multi-label pedestrian attribute recognition based on domain adaptation[J]. Journal of Computer Applications, 2022, 42(8): 2401-2406.

程南江, 余贞侠, 陈琳, 乔贺辙. 基于领域自适应的多源多标签行人属性识别[J]. 《计算机应用》唯一官方网站, 2022, 42(8): 2401-2406.

Figures/Tables 7

References 27

1	LAYNE R， HOSPEDALES T， GONG S G. Person re-identification by attributes ［C］// Proceedings of the 2012 British Machine Vision Conference. Durham： BMVA Press， 2012： No.24. 10.5244/c.26.24
2	WANG J Y， ZHU X T， GONG S G， et al. Transferable joint attribute-identity deep learning for unsupervised person re-identification ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2275-2284. 10.1109/cvpr.2018.00242
3	KUMAR N， BERG A C， BELHUMEUR P N， et al. Attribute and simile classifiers for face verification ［C］// Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Piscataway： IEEE， 2009： 365-372. 10.1109/iccv.2009.5459250
4	FERIS R， BOBBITT R， BROWN L， et al. Attribute-based people search： lessons learnt from a practical surveillance system ［C］// Proceedings of the 4th ACM International Conference on Multimedia Retrieval. New York： ACM， 2014： 153-160. 10.1145/2578726.2578732
5	LIN Y T， ZHENG L， ZHENG Z D， et al. Improving person re-identification by attribute and identity learning［J］. Pattern Recognition， 2019， 95： 151-161. 10.1016/j.patcog.2019.06.006
6	马路宽.行人属性识别研究［J］.现代计算机， 2021（4）： 89-93. 10.3969/j.issn.1007-1423.2021.04.018
	MA L K. Research on pedestrian attribute recognition［J］. Modern Computer， 2021（4）： 89-93. 10.3969/j.issn.1007-1423.2021.04.018
7	LI D W， CHEN X T， HUANG K Q. Multi-attribute learning for pedestrian attribute recognition in surveillance scenarios ［C］// Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition. Piscataway： IEEE， 2015： 111-115. 10.1109/acpr.2015.7486476
8	SUDOWE P， SPITZER H， LEIBE B. Person attribute recognition with a jointly-trained holistic CNN model ［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision Workshops. Piscataway： IEEE， 2015： 329-337. 10.1109/iccvw.2015.51
9	ABDULNABI A H， WANG G， LU J W， et al. Multi-task CNN model for attribute prediction［J］. IEEE Transactions on Multimedia， 2015， 17（11）： 1949-1959. 10.1109/tmm.2015.2477680
10	LI D W， CHEN X T， ZHANG Z， et al. Pose guided deep model for pedestrian attribute recognition in surveillance scenarios ［C］// Proceedings of the 2018 IEEE International Conference on Multimedia and Expo. Piscataway： IEEE， 2018： 1-6. 10.1109/icme.2018.8486604
11	SARAFIANOS N， XU X， KAKADIARIS I A. Deep imbalanced attribute classification using visual attention aggregation ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11215. Cham： Springer， 2018： 708-725.
12	BOURDEV L， MAJI S， MALIK J. Describing people： a poselet-based approach to attribute classification ［C］// Proceedings of the 2011 International Conference on Computer Vision. Piscataway： IEEE， 2011： 1543-1550. 10.1109/iccv.2011.6126413
13	LIU X H， ZHAO H Y， TIAN M Q， et al. HydraPlus-Net： attentive deep features for pedestrian analysis ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 350-359. 10.1109/iccv.2017.46
14	WU M D， HUANG D， GUO Y F， et al. Distraction-aware feature learning for human attribute recognition via coarse-to-fine attention mechanism ［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 12394-12401. 10.1609/aaai.v34i07.6925
15	YAGHOUBI E， BORZA D， NEVES J， et al. An attention-based deep learning model for multiple pedestrian attributes recognition［J］. Image and Vision Computing， 2020， 102： No.103981. 10.1016/j.imavis.2020.103981
16	吴锐，刘宇，冯凯.基于双域自注意力机制的行人属性识别［J］.计算机应用， 2021， 41（2）： 372-378.
	WU R， LIU Y， FENG K. Pedestrian attribute recognition based on two-domain self-attention mechanism［J］. Journal of Computer Applications， 2021， 41（2）： 372-378.
17	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016， 770-778. 10.1109/cvpr.2016.90
18	DENG Y B， LUO P， LOY C C， et al. Pedestrian attribute recognition at far distance ［C］// Proceedings of the 22nd ACM International Conference on Multimedia. New York： ACM， 2014： 789-792. 10.1145/2647868.2654966
19	LI D W， ZHANG Z， CHEN X T， et al. A richly annotated dataset for pedestrian attribute recognition［EB/OL］. （2016-04-27）［2021-08-15］. . 10.1109/tip.2018.2878349
20	LI D W， ZHANG Z， CHEN X T， et al. A richly annotated pedestrian dataset for person retrieval in real surveillance scenarios［J］. IEEE Transactions on Image Processing， 2019， 28（4）： 1575-1590. 10.1109/tip.2018.2878349
21	刘士豪，胡学敏，姜博厚，等.基于生成对抗双网络的虚拟到真实驾驶场景的视频翻译模型［J］.计算机应用， 2020， 40（6）： 1621-1626.
	LIU S H， HU X M， JIANG B H， et al. Video translation model from virtual to real driving scenes based on generative adversarial dual networks［J］. Journal of Computer Applications， 2020， 40（6）： 1621-1626.
22	ZHU J Y， PARK T， ISOLA P， et al. Unpaired image-to-image translation using cycle-consistent adversarial networks ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2242-2251. 10.1109/iccv.2017.244
23	HAN K， WANG Y H， SHU H， et al. Attribute aware pooling for pedestrian attribute recognition ［C］// Proceedings of the 28th Joint Conference on Artificial Intelligence. California： ijcai.org， 2019： 2456-2462. 10.24963/ijcai.2019/341
24	TANG C F， SHENG L， ZHANG Z X， et al. Improving pedestrian attribute recognition with weakly-supervised multi-scale attribute-specific localization ［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 4996-5005. 10.1109/iccv.2019.00510
25	ZENG H T， AI H Z， ZHUANG Z J， et al. Multi-task learning via co-attentive sharing for pedestrian attribute recognition ［C］// Proceedings of the 2020 IEEE International Conference on Multimedia and Expo. Piscataway： IEEE， 2020： 1-6. 10.1109/icme46284.2020.9102757
26	MOGHADDAM M， CHARMI M， HASSANPOOR H. Jointly human semantic parsing and attribute recognition with feature pyramid structure in EfficientNets［J］. IET Image Processing， 2021， 15（10）： 2281-2291.
27	JIA J， HUANG H J， YANG W J， et al. Rethinking of pedestrian attribute recognition： realistic datasets with efficient method ［EB/OL］. （2020-05-26）［2021-08-15］. .

方法	mA	Acc	Prec	Rec	F1
HPNet^［13］	74.21	72.19	82.97	82.09	82.53
PGDM^［10］	74.95	73.08	84.36	82.24	83.29
CoCNN^［23］	80.56	78.30	89.49	84.36	86.85
ALM^［24］	80.68	77.08	84.21	88.84	86.46
MT-CAS^［25］	77.20	78.09	88.46	84.86	86.62
AR-BiFPN^［26］	81.45	79.13	86.24	89.46	87.94
StrongBaseline^［27］	80.50	78.84	87.24	87.12	86.78
PETA→PA-100K	81.72	78.98	87.01	87.46	87.23
RAPv1→PA-100K	82.12	79.38	86.67	88.04	87.35
RAPv2→PA-100K	82.03	79.59	87.52	88.38	87.95

方法	mA	Acc	Prec	Rec	F1
HPNet^［13］	74.21	72.19	82.97	82.09	82.53
PGDM^［10］	74.95	73.08	84.36	82.24	83.29
CoCNN^［23］	80.56	78.30	89.49	84.36	86.85
ALM^［24］	80.68	77.08	84.21	88.84	86.46
MT-CAS^［25］	77.20	78.09	88.46	84.86	86.62
AR-BiFPN^［26］	81.45	79.13	86.24	89.46	87.94
StrongBaseline^［27］	80.50	78.84	87.24	87.12	86.78
PETA→PA-100K	81.72	78.98	87.01	87.46	87.23
RAPv1→PA-100K	82.12	79.38	86.67	88.04	87.35
RAPv2→PA-100K	82.03	79.59	87.52	88.38	87.95

方法	RAPv1→PA-100K					RAPv2→PA-100K					PETA→PA-100K
方法	mA	Acc	Prec	Rec	F1	mA	Acc	Prec	Rec	F1	mA	Acc	Prec	Rec	F1
ResNet-50	80.50	78.84	87.24	87.12	86.78	80.50	78.84	87.24	87.12	86.78	80.50	78.84	87.24	87.12	86.78
ResNet-50+LAM	81.37	79.05	87.30	87.18	87.24	81.64	78.36	87.68	87.05	87.36	80.82	79.05	87.30	87.31	86.92
ResNet-50+LAM+FAM	82.12	79.38	86.67	88.04	87.35	82.03	79.59	87.52	88.38	87.95	81.72	78.98	87.01	87.46	87.23

方法	RAPv1→PA-100K					RAPv2→PA-100K					PETA→PA-100K
方法	mA	Acc	Prec	Rec	F1	mA	Acc	Prec	Rec	F1	mA	Acc	Prec	Rec	F1
ResNet-50	80.50	78.84	87.24	87.12	86.78	80.50	78.84	87.24	87.12	86.78	80.50	78.84	87.24	87.12	86.78
ResNet-50+LAM	81.37	79.05	87.30	87.18	87.24	81.64	78.36	87.68	87.05	87.36	80.82	79.05	87.30	87.31	86.92
ResNet-50+LAM+FAM	82.12	79.38	86.67	88.04	87.35	82.03	79.59	87.52	88.38	87.95	81.72	78.98	87.01	87.46	87.23

属性	识别准确率/%		提升百分点
属性	StrongBaseline	RAPv1→PA-100K	提升百分点
Hat	66.16	76.84	10.68
Glasses	81.23	87.07	5.84
ShortSleeve	90.71	91.61	0.90
Trousers	89.85	91.92	2.07
Skirt&Dress	82.78	86.48	3.70
boots	53.47	54.86	1.39
HandBag	75.93	81.03	5.10
ShoulderBag	76.43	76.07	-0.36
Backpack	79.75	82.62	2.87
Female	90.61	92.37	1.76

Multi-source and multi-label pedestrian attribute recognition based on domain adaptation

基于领域自适应的多源多标签行人属性识别

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 27

Related Articles 15

Recommended Articles

Metrics

[1]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[2]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[3]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[4]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[5]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[6]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[7]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[8]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[9]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[10]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[11]	Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263.
[12]	Dongwei WANG, Baichen LIU, Zhi HAN, Yanmei WANG, Yandong TANG. Deep network compression method based on low-rank decomposition and vector quantization [J]. Journal of Computer Applications, 2024, 44(7): 1987-1994.
[13]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[14]	Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP： defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086.
[15]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.

属性	识别准确率/%		提升百分点
属性	StrongBaseline	PETA→PA-100K	提升百分点
Hat	66.16	73.47	7.31
Glasses	81.23	85.39	4.16
ShortSleeve	90.71	91.38	0.67
UpperLogo	78.31	82.77	4.46
UpperPlaid	77.3	80.17	2.87
Trousers	89.85	91.31	1.46
Shorts	90.83	91.42	0.59
Skirt&Dress	82.78	85.43	2.65
Backpack	79.75	80.14	0.39
Female	90.61	91.22	0.61
AgeOver60	76.89	80.12	3.23

属性	识别准确率/%		提升百分点
属性	StrongBaseline	PETA→PA-100K	提升百分点
Hat	66.16	73.47	7.31
Glasses	81.23	85.39	4.16
ShortSleeve	90.71	91.38	0.67
UpperLogo	78.31	82.77	4.46
UpperPlaid	77.3	80.17	2.87
Trousers	89.85	91.31	1.46
Shorts	90.83	91.42	0.59
Skirt&Dress	82.78	85.43	2.65
Backpack	79.75	80.14	0.39
Female	90.61	91.22	0.61
AgeOver60	76.89	80.12	3.23