Multi-source and multi-label pedestrian attribute recognition based on domain adaptation

doi:10.11772/j.issn.1001-9081.2021060950

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (8): 2401-2406.DOI: 10.11772/j.issn.1001-9081.2021060950

• Artificial intelligence • Previous Articles

Multi-source and multi-label pedestrian attribute recognition based on domain adaptation

Nanjiang CHENG¹^,², Zhenxia YU¹(), Lin CHEN², Hezhe QIAO²

^1.School of Computer Science，Chengdu University of Information Technology，Chengdu Sichuan 610225，China
^2.Chongqing Institute of Green and Intelligent Technology，Chinese Academy of Sciences，Chongqing 400714，China

Received:2021-06-07 Revised:2021-08-17 Accepted:2021-09-02 Online:2022-08-09 Published:2022-08-10
Contact: Zhenxia YU
About author:CHENG Nanjiang， born in 1997， M. S. candidate. His research interests include machine learning， computer vision.
YU Zhenxia， born in 1968， Ph. D.， associate professor. Her research interests include machine learning， computer vision.
CHEN Lin， born in 1985， Ph. D. His research interests include computer vision， pattern recognition.
QIAO Hezhe， born in 1998， M. S. candidate. His research interests include computer vision， big data.
Supported by:
National Key Research and Development Program of China(2020YFC0833406)

基于领域自适应的多源多标签行人属性识别

程南江¹^,², 余贞侠¹(), 陈琳², 乔贺辙²

^1.成都信息工程大学计算机学院，成都 610225
^2.中国科学院重庆绿色智能技术研究院，重庆 400714

通讯作者: 余贞侠
作者简介:程南江（1997—），男，四川内江人，硕士研究生，主要研究方向：机器学习、计算机视觉；
余贞侠（1968—），女，江苏南京人，副教授，博士，主要研究方向：机器学习、计算机视觉；
陈琳（1985—），男，重庆人，博士，主要研究方向：计算机视觉、模式识别；
乔贺辙（1998—），男，甘肃天水人，硕士研究生，主要研究方向：计算机视觉、大数据。
基金资助:
国家重点研发计划项目(2020YFC0833406)

Abstract

Abstract:

The current public datasets of Pedestrian Attribute Recognition （PAR） have the characteristics of complicated attribute annotations and various collection scenarios， leading to the large variations of the pedestrian attributes in different datasets， so that it is hard to directly utilize the existing labeled information in the public datasets for PAR in practice. To address this issue， a multi-source and multi-label PAR method based on domain adaptation was proposed. Firstly， to transfer the styles of the different datasets into a unified one， the features of the samples were aligned by the domain adaption method. Then， a multi-attribute one-hot coding and weighting algorithm was proposed to align the labels with the common attribute in multiple datasets. Finally， the multi-label semi-supervised loss function was combined to perform joint training across datasets to improve the attribute recognition accuracy. The proposed feature alignment and label alignment algorithms were able to effectively solve the heterogeneity problem of attributes in multiple PAR datasets. Experimental results after aligning three pedestrian attribute datasets PETA， RAPv1 and RAPv2 with PA-100K dataset show that the proposed method improves the average accuracy by 1.22 percentage points， 1.62 percentage points and 1.53 percentage points respectively， compared to the method StrongBaseline， demonstrating that this method has a strong advantage in cross dataset PAR.

Key words: Pedestrian Attribute Recognition (PAR), multi-label learning, domain adaption, deep learning, Convolutional Neural Network (CNN)

摘要：

当前行人属性识别（PAR）公开数据集中属性标注繁杂且采集场景多样，各数据集中行人属性差异较大，进而导致公开数据库已有的标记信息数据难以直接应用到PAR实际问题中。针对上述问题，提出一种基于领域自适应的多源多标签PAR方法。首先通过领域自适应方法对样本进行特征对齐完成多个数据集之间的统一风格转换；接着提出多属性one-hot编码加权算法，将多数据集中共有属性的标签对齐；最后结合多标签半监督损失函数，进行跨数据集联合训练以提高属性识别准确率。通过所提出的特征对齐和标签对齐算法，可有效解决PAR多数据集中属性异构性问题。将三个行人属性数据集PETA、RAPv1和RAPv2分别与PA-100K数据集对齐后的实验结果表明，所提出的方法对比StrongBaseline在平均准确率上分别提升了1.22、1.62和1.53个百分点，说明该方法在跨数据集PAR中具有一定的优势。

关键词: 行人属性识别, 多标签学习, 领域自适应, 深度学习, 卷积神经网络

CLC Number:

TP391.4

Nanjiang CHENG, Zhenxia YU, Lin CHEN, Hezhe QIAO. Multi-source and multi-label pedestrian attribute recognition based on domain adaptation[J]. Journal of Computer Applications, 2022, 42(8): 2401-2406.

程南江, 余贞侠, 陈琳, 乔贺辙. 基于领域自适应的多源多标签行人属性识别[J]. 《计算机应用》唯一官方网站, 2022, 42(8): 2401-2406.

Figures/Tables 7

References 27

1	LAYNE R， HOSPEDALES T， GONG S G. Person re-identification by attributes ［C］// Proceedings of the 2012 British Machine Vision Conference. Durham： BMVA Press， 2012： No.24. 10.5244/c.26.24
2	WANG J Y， ZHU X T， GONG S G， et al. Transferable joint attribute-identity deep learning for unsupervised person re-identification ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2275-2284. 10.1109/cvpr.2018.00242
3	KUMAR N， BERG A C， BELHUMEUR P N， et al. Attribute and simile classifiers for face verification ［C］// Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Piscataway： IEEE， 2009： 365-372. 10.1109/iccv.2009.5459250
4	FERIS R， BOBBITT R， BROWN L， et al. Attribute-based people search： lessons learnt from a practical surveillance system ［C］// Proceedings of the 4th ACM International Conference on Multimedia Retrieval. New York： ACM， 2014： 153-160. 10.1145/2578726.2578732
5	LIN Y T， ZHENG L， ZHENG Z D， et al. Improving person re-identification by attribute and identity learning［J］. Pattern Recognition， 2019， 95： 151-161. 10.1016/j.patcog.2019.06.006
6	马路宽.行人属性识别研究［J］.现代计算机， 2021（4）： 89-93. 10.3969/j.issn.1007-1423.2021.04.018
	MA L K. Research on pedestrian attribute recognition［J］. Modern Computer， 2021（4）： 89-93. 10.3969/j.issn.1007-1423.2021.04.018
7	LI D W， CHEN X T， HUANG K Q. Multi-attribute learning for pedestrian attribute recognition in surveillance scenarios ［C］// Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition. Piscataway： IEEE， 2015： 111-115. 10.1109/acpr.2015.7486476
8	SUDOWE P， SPITZER H， LEIBE B. Person attribute recognition with a jointly-trained holistic CNN model ［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision Workshops. Piscataway： IEEE， 2015： 329-337. 10.1109/iccvw.2015.51
9	ABDULNABI A H， WANG G， LU J W， et al. Multi-task CNN model for attribute prediction［J］. IEEE Transactions on Multimedia， 2015， 17（11）： 1949-1959. 10.1109/tmm.2015.2477680
10	LI D W， CHEN X T， ZHANG Z， et al. Pose guided deep model for pedestrian attribute recognition in surveillance scenarios ［C］// Proceedings of the 2018 IEEE International Conference on Multimedia and Expo. Piscataway： IEEE， 2018： 1-6. 10.1109/icme.2018.8486604
11	SARAFIANOS N， XU X， KAKADIARIS I A. Deep imbalanced attribute classification using visual attention aggregation ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11215. Cham： Springer， 2018： 708-725.
12	BOURDEV L， MAJI S， MALIK J. Describing people： a poselet-based approach to attribute classification ［C］// Proceedings of the 2011 International Conference on Computer Vision. Piscataway： IEEE， 2011： 1543-1550. 10.1109/iccv.2011.6126413
13	LIU X H， ZHAO H Y， TIAN M Q， et al. HydraPlus-Net： attentive deep features for pedestrian analysis ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 350-359. 10.1109/iccv.2017.46
14	WU M D， HUANG D， GUO Y F， et al. Distraction-aware feature learning for human attribute recognition via coarse-to-fine attention mechanism ［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 12394-12401. 10.1609/aaai.v34i07.6925
15	YAGHOUBI E， BORZA D， NEVES J， et al. An attention-based deep learning model for multiple pedestrian attributes recognition［J］. Image and Vision Computing， 2020， 102： No.103981. 10.1016/j.imavis.2020.103981
16	吴锐，刘宇，冯凯.基于双域自注意力机制的行人属性识别［J］.计算机应用， 2021， 41（2）： 372-378.
	WU R， LIU Y， FENG K. Pedestrian attribute recognition based on two-domain self-attention mechanism［J］. Journal of Computer Applications， 2021， 41（2）： 372-378.
17	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016， 770-778. 10.1109/cvpr.2016.90
18	DENG Y B， LUO P， LOY C C， et al. Pedestrian attribute recognition at far distance ［C］// Proceedings of the 22nd ACM International Conference on Multimedia. New York： ACM， 2014： 789-792. 10.1145/2647868.2654966
19	LI D W， ZHANG Z， CHEN X T， et al. A richly annotated dataset for pedestrian attribute recognition［EB/OL］. （2016-04-27）［2021-08-15］. . 10.1109/tip.2018.2878349
20	LI D W， ZHANG Z， CHEN X T， et al. A richly annotated pedestrian dataset for person retrieval in real surveillance scenarios［J］. IEEE Transactions on Image Processing， 2019， 28（4）： 1575-1590. 10.1109/tip.2018.2878349
21	刘士豪，胡学敏，姜博厚，等.基于生成对抗双网络的虚拟到真实驾驶场景的视频翻译模型［J］.计算机应用， 2020， 40（6）： 1621-1626.
	LIU S H， HU X M， JIANG B H， et al. Video translation model from virtual to real driving scenes based on generative adversarial dual networks［J］. Journal of Computer Applications， 2020， 40（6）： 1621-1626.
22	ZHU J Y， PARK T， ISOLA P， et al. Unpaired image-to-image translation using cycle-consistent adversarial networks ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2242-2251. 10.1109/iccv.2017.244
23	HAN K， WANG Y H， SHU H， et al. Attribute aware pooling for pedestrian attribute recognition ［C］// Proceedings of the 28th Joint Conference on Artificial Intelligence. California： ijcai.org， 2019： 2456-2462. 10.24963/ijcai.2019/341
24	TANG C F， SHENG L， ZHANG Z X， et al. Improving pedestrian attribute recognition with weakly-supervised multi-scale attribute-specific localization ［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 4996-5005. 10.1109/iccv.2019.00510
25	ZENG H T， AI H Z， ZHUANG Z J， et al. Multi-task learning via co-attentive sharing for pedestrian attribute recognition ［C］// Proceedings of the 2020 IEEE International Conference on Multimedia and Expo. Piscataway： IEEE， 2020： 1-6. 10.1109/icme46284.2020.9102757
26	MOGHADDAM M， CHARMI M， HASSANPOOR H. Jointly human semantic parsing and attribute recognition with feature pyramid structure in EfficientNets［J］. IET Image Processing， 2021， 15（10）： 2281-2291.
27	JIA J， HUANG H J， YANG W J， et al. Rethinking of pedestrian attribute recognition： realistic datasets with efficient method ［EB/OL］. （2020-05-26）［2021-08-15］. .

方法	mA	Acc	Prec	Rec	F1
HPNet^［13］	74.21	72.19	82.97	82.09	82.53
PGDM^［10］	74.95	73.08	84.36	82.24	83.29
CoCNN^［23］	80.56	78.30	89.49	84.36	86.85
ALM^［24］	80.68	77.08	84.21	88.84	86.46
MT-CAS^［25］	77.20	78.09	88.46	84.86	86.62
AR-BiFPN^［26］	81.45	79.13	86.24	89.46	87.94
StrongBaseline^［27］	80.50	78.84	87.24	87.12	86.78
PETA→PA-100K	81.72	78.98	87.01	87.46	87.23
RAPv1→PA-100K	82.12	79.38	86.67	88.04	87.35
RAPv2→PA-100K	82.03	79.59	87.52	88.38	87.95

方法	mA	Acc	Prec	Rec	F1
HPNet^［13］	74.21	72.19	82.97	82.09	82.53
PGDM^［10］	74.95	73.08	84.36	82.24	83.29
CoCNN^［23］	80.56	78.30	89.49	84.36	86.85
ALM^［24］	80.68	77.08	84.21	88.84	86.46
MT-CAS^［25］	77.20	78.09	88.46	84.86	86.62
AR-BiFPN^［26］	81.45	79.13	86.24	89.46	87.94
StrongBaseline^［27］	80.50	78.84	87.24	87.12	86.78
PETA→PA-100K	81.72	78.98	87.01	87.46	87.23
RAPv1→PA-100K	82.12	79.38	86.67	88.04	87.35
RAPv2→PA-100K	82.03	79.59	87.52	88.38	87.95

方法	RAPv1→PA-100K					RAPv2→PA-100K					PETA→PA-100K
方法	mA	Acc	Prec	Rec	F1	mA	Acc	Prec	Rec	F1	mA	Acc	Prec	Rec	F1
ResNet-50	80.50	78.84	87.24	87.12	86.78	80.50	78.84	87.24	87.12	86.78	80.50	78.84	87.24	87.12	86.78
ResNet-50+LAM	81.37	79.05	87.30	87.18	87.24	81.64	78.36	87.68	87.05	87.36	80.82	79.05	87.30	87.31	86.92
ResNet-50+LAM+FAM	82.12	79.38	86.67	88.04	87.35	82.03	79.59	87.52	88.38	87.95	81.72	78.98	87.01	87.46	87.23

方法	RAPv1→PA-100K					RAPv2→PA-100K					PETA→PA-100K
方法	mA	Acc	Prec	Rec	F1	mA	Acc	Prec	Rec	F1	mA	Acc	Prec	Rec	F1
ResNet-50	80.50	78.84	87.24	87.12	86.78	80.50	78.84	87.24	87.12	86.78	80.50	78.84	87.24	87.12	86.78
ResNet-50+LAM	81.37	79.05	87.30	87.18	87.24	81.64	78.36	87.68	87.05	87.36	80.82	79.05	87.30	87.31	86.92
ResNet-50+LAM+FAM	82.12	79.38	86.67	88.04	87.35	82.03	79.59	87.52	88.38	87.95	81.72	78.98	87.01	87.46	87.23

属性	识别准确率/%		提升百分点
属性	StrongBaseline	RAPv1→PA-100K	提升百分点
Hat	66.16	76.84	10.68
Glasses	81.23	87.07	5.84
ShortSleeve	90.71	91.61	0.90
Trousers	89.85	91.92	2.07
Skirt&Dress	82.78	86.48	3.70
boots	53.47	54.86	1.39
HandBag	75.93	81.03	5.10
ShoulderBag	76.43	76.07	-0.36
Backpack	79.75	82.62	2.87
Female	90.61	92.37	1.76

Multi-source and multi-label pedestrian attribute recognition based on domain adaptation

基于领域自适应的多源多标签行人属性识别

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 27

Related Articles 15

Recommended Articles

Metrics

[1]	Zhenhu LYU, Xinzheng XU, Fangyan ZHANG. Lightweight attention mechanism module based on squeeze and excitation [J]. Journal of Computer Applications, 2022, 42(8): 2353-2360.
[2]	Xianjie ZHANG, Zhiming ZHANG. Handwritten English text recognition based on convolutional neural network and Transformer [J]. Journal of Computer Applications, 2022, 42(8): 2394-2400.
[3]	Tingwei QIN, Pengcheng ZHAO, Pinle QIN, Jianchao ZENG, Rui CHAI, Yongqi HUANG. Point cloud registration algorithm based on residual attention mechanism [J]. Journal of Computer Applications, 2022, 42(7): 2184-2191.
[4]	Yaru HAN, Lianshan YAN, Tao YAO. Deep hashing retrieval algorithm based on meta-learning [J]. Journal of Computer Applications, 2022, 42(7): 2015-2021.
[5]	Wanjun LIU, Jiaming WANG, Haicheng QU, Libing DONG, Xinyu CAO. Music genre classification algorithm based on attention spectral-spatial feature [J]. Journal of Computer Applications, 2022, 42(7): 2072-2077.
[6]	Zhenyu WANG, Lei ZHANG, Wenbin GAO, Weiming QUAN. Human activity recognition based on progressive neural architecture search [J]. Journal of Computer Applications, 2022, 42(7): 2058-2064.
[7]	Xiangyue TAN, Xiao HU, Jiaxin YANG, Junjiang XIANG. Camouflaged object detection based on progressive feature enhancement aggregation [J]. Journal of Computer Applications, 2022, 42(7): 2192-2200.
[8]	Haiqi WANG, Zhihai WANG, Liuke LI, Haoran KONG, Qiong WANG, Jianbo XU. Spatial-temporal prediction model of urban short-term traffic flow based on grid division [J]. Journal of Computer Applications, 2022, 42(7): 2274-2280.
[9]	Ning DONG, Xiaorong CHENG, Mingquan ZHANG. Intrusion detection system with dynamic weight loss function based on internet of things platform [J]. Journal of Computer Applications, 2022, 42(7): 2118-2124.
[10]	Yang ZHANG, Jiangbo HAO. Malicious code detection method based on attention mechanism and residual network [J]. Journal of Computer Applications, 2022, 42(6): 1708-1715.
[11]	Shan SU, Yang ZHANG, Dongwen ZHANG. Coupling related code smell detection method based on deep learning [J]. Journal of Computer Applications, 2022, 42(6): 1702-1707.
[12]	Jing JIANG, Yu CHEN, Jieping SUN, Shenggen JU. Integrating posterior probability calibration training into text classification algorithm [J]. Journal of Computer Applications, 2022, 42(6): 1789-1795.
[13]	Min WEN, Rongcun WANG, Shujuan JIANG. Source code vulnerability detection based on relational graph convolution network [J]. Journal of Computer Applications, 2022, 42(6): 1814-1821.
[14]	Yumin HAN, Xiaoyan HAO. Material entity recognition based on subword embedding and relative attention [J]. Journal of Computer Applications, 2022, 42(6): 1862-1868.
[15]	Lei YANG, Hongdong ZHAO, Kuaikuai YU. End-to-end speech emotion recognition based on multi-head attention [J]. Journal of Computer Applications, 2022, 42(6): 1869-1875.

属性	识别准确率/%		提升百分点
属性	StrongBaseline	PETA→PA-100K	提升百分点
Hat	66.16	73.47	7.31
Glasses	81.23	85.39	4.16
ShortSleeve	90.71	91.38	0.67
UpperLogo	78.31	82.77	4.46
UpperPlaid	77.3	80.17	2.87
Trousers	89.85	91.31	1.46
Shorts	90.83	91.42	0.59
Skirt&Dress	82.78	85.43	2.65
Backpack	79.75	80.14	0.39
Female	90.61	91.22	0.61
AgeOver60	76.89	80.12	3.23

属性	识别准确率/%		提升百分点
属性	StrongBaseline	PETA→PA-100K	提升百分点
Hat	66.16	73.47	7.31
Glasses	81.23	85.39	4.16
ShortSleeve	90.71	91.38	0.67
UpperLogo	78.31	82.77	4.46
UpperPlaid	77.3	80.17	2.87
Trousers	89.85	91.31	1.46
Shorts	90.83	91.42	0.59
Skirt&Dress	82.78	85.43	2.65
Backpack	79.75	80.14	0.39
Female	90.61	91.22	0.61
AgeOver60	76.89	80.12	3.23