基于联合损失胶囊网络的换衣行人重识别

doi:10.11772/j.issn.1001-9081.2021061090

《计算机应用》唯一官方网站 ›› 2021, Vol. 41 ›› Issue (12): 3596-3601.DOI: 10.11772/j.issn.1001-9081.2021061090

• 第十八届中国机器学习会议(CCML 2021) • 上一篇

基于联合损失胶囊网络的换衣行人重识别

刘乾¹, 王洪元¹(), 曹亮¹, 孙博言¹, 肖宇², 张继¹

^1.常州大学计算机与人工智能学院、阿里云大数据学院，江苏常州 213164
^2.常州工程职业技术学院设计艺术学院，江苏常州 213164

收稿日期:2021-05-12 修回日期:2021-08-05 接受日期:2021-08-17 发布日期:2021-12-28 出版日期:2021-12-10
通讯作者: 王洪元
作者简介:刘乾（1993—），男，江苏南京人，硕士研究生，主要研究方向：计算机视觉
曹亮（1996—），江苏盐城人，硕士研究生，主要研究方向：计算机视觉
孙博言（1996—），黑龙江双鸭山人，硕士研究生，主要研究方向：计算机视觉
肖宇（1981—），女，黑龙江伊春人，副教授，硕士，主要研究方向：数字媒体技术、图形图像处理
张继（1981—），男，江苏常州人，讲师，硕士，主要研究方向：模式识别、图像处理、机器视觉。
基金资助:
国家自然科学基金面上项目(61976028)

Cloth-changing person re-identification based on joint loss capsule network

Qian LIU¹, Hongyuan WANG¹(), Liang CAO¹, Boyan SUN¹, Yu XIAO², Ji ZHANG¹

^1.School of Computer Science and Artificial Intelligence/ Aliyun School of Big Data，Changzhou University，Changzhou Jiangsu 213164，China
^2.Design Art College，Changzhou Vocational Institute of Engineering，Changzhou Jiangsu 213164，China

Received:2021-05-12 Revised:2021-08-05 Accepted:2021-08-17 Online:2021-12-28 Published:2021-12-10
Contact: Hongyuan WANG
About author:LIU Qian， born in 1993， M. S. candidate. His research interests include computer vision.
CAO Liang， born in 1996， M. S. candidate. His research interests include computer vision.
SUN Boyan， born in 1996， M. S. candidate. His research interests include computer vision.
XIAO Yu， born in 1981， M. S.， associate professor. Her research interests include digital media technology， graphics and image processing.
ZHANG Ji， born in 1981， M. S.， lecturer. His research interests include pattern recognition， image processing， machine vision.
Supported by:
the Surface Program of National Natural Science Foundation of China(61976028)

摘要/Abstract

摘要：

目前的行人重识别（Re-ID）研究主要集中在短时间情形，即一个人的衣着不太可能发生改变的情况。然而现实中更常见的是长时间的情况，这时一个人有很大的机会更换衣服，Re-ID模型应该考虑这种情况。为此，研究了一种基于联合损失胶囊网络的换衣行人重识别方法。所提方法基于换衣行人重识别胶囊网络ReIDCaps，使用与传统的标量神经元相比包含更多信息的矢量胶囊，用其长度表示行人身份信息，用其方向表示行人衣着信息；采用软嵌入注意力（SEA）防止模型过拟合；使用特征稀疏表示（FSR）机制提取具有判别性的特征；增加标签平滑正则化交叉熵损失与Circle Loss的联合损失以提高模型的泛化能力和鲁棒性。在三个换衣行人重识别数据集Celeb-reID、Celeb-reID-light和NKUP上进行实验，实验结果表明所提方法与目前已有的Re-ID方法相比具有一定优势。

关键词: 换衣行人重识别, 胶囊网络, 矢量胶囊, 标签平滑正则化交叉熵损失, Circle Loss

Abstract:

Current research on Person Re-Identification （Re-ID） mainly concentrates on short-term situations with person’s clothing usually unchanged. However， more common practical cases are long-term situations， in which a person has higher possibility to change his clothes， which should be considered by Re-ID models. Therefore， a method of person re-identification with cloth changing based on joint loss capsule network was proposed. The proposed method was based on ReIDCaps， a capsule network for cloth-changing person re-identification. In the method， vector-neuron capsules that contain more information than traditional scalar neurons were used. The length of the vector-neuron capsule was used to represent the identity information of the person， and the direction of the capsule was used to represent the clothing information of the person. Soft Embedding Attention （SEA） was used to avoid the model over-fitting. Feature Sparse Representation （FSR） mechanism was adopted to extract discriminative features. The joint loss of label smoothing regularization cross-entropy loss and Circle Loss was added to improve the generalization ability and robustness of the model. Experimental results on three datasets including Celeb-reID， Celeb-reID-light and NKUP prove that the proposed method has certain advantages compared with the existing person re-identification methods.

Key words: cloth-changing person re-identification, capsule network, vector-neuron capsule, label smoothing regularization cross-entropy loss, Circle Loss

中图分类号:

TP391.41

刘乾, 王洪元, 曹亮, 孙博言, 肖宇, 张继. 基于联合损失胶囊网络的换衣行人重识别[J]. 计算机应用, 2021, 41(12): 3596-3601.

Qian LIU, Hongyuan WANG, Liang CAO, Boyan SUN, Yu XIAO, Ji ZHANG. Cloth-changing person re-identification based on joint loss capsule network[J]. Journal of Computer Applications, 2021, 41(12): 3596-3601.

图/表 3

参考文献 32

1	GONG S G， CRISTANI M， LOY C C， et al. The re-identification challenge［M］// GONG S G， CRISTANI M， YAN S C， et al. Person Re-Identification. London： Springer， 2014： 1-20. 10.1007/978-1-4471-6296-4_1
2	GRAY D， TAO H. Viewpoint invariant pedestrian recognition with an ensemble of localized features［C］// Proceedings of the 2008 European Conference on Computer Vision， LNCS5302. Berlin： Springer， 2008： 262-275.
3	LI W， ZHAO R， XIAO T， et al. DeepReID： deep filter pairing neural network for person re-identification［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2014： 152-159. 10.1109/cvpr.2014.27
4	ZHENG L， SHEN L Y， TIAN L， et al. Scalable person re-identification： a benchmark［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1116-1124. 10.1109/iccv.2015.133
5	ZHENG Z D， ZHENG L， YANG Y. Unlabeled samples generated by GAN improve the person re-identification baseline in vitro［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 3774-3782. 10.1109/iccv.2017.405
6	LUO H， JIANG W， GU Y Z， et al. A strong baseline and batch normalization neck for deep person re-identification［J］. IEEE Transactions on Multimedia， 2020， 22（10）： 2597-2609. 10.1109/tmm.2019.2958756
7	LUO H， JIANG W， ZHANG X， et al. AlignedReID++： dynamically matching local information for person re-identification［J］. Pattern Recognition， 2019， 94： 53-61. 10.1016/j.patcog.2019.05.028
8	WANG G S， YUAN Y F， CHEN X， et al. Learning discriminative features with multiple granularities for person re-identification［C］// Proceedings of the 26th ACM International Conference on Multimedia. New York： ACM， 2018： 274-282. 10.1145/3240508.3240552
9	HUANG Y， XU J S， WU Q， et al. Beyond scalar neuron： adopting vector-neuron capsules for long-term person re-identification［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2020， 30（10）： 3459-3471. 10.1109/tcsvt.2019.2948093
10	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2818-2826. 10.1109/cvpr.2016.308
11	SUN Y F， CHENG C M， ZHANG Y H， et al. Circle loss： a unified perspective of pair similarity optimization［C］// Proceedings of the 2020 IEEE/CVF International Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 6397-6406. 10.1109/cvpr42600.2020.00643
12	WANG K， MA Z， CHEN S Y， et al. A benchmark for clothes variation in person re-identification［J］. International Journal of Intelligent Systems， 2020， 35（12）： 1881-1898. 10.1002/int.22276
13	HAQUE A， ALAHI A， LI F F. Recurrent attention models for depth-based person identification［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1229-1238. 10.1109/cvpr.2016.138
14	MUNARO M， BASSO A， FOSSATI A， et al. 3D reconstruction of freely moving persons for re-identification with a depth sensor［C］// Proceedings of the 2014 IEEE International Conference on Robotics and Automation. Piscataway： IEEE， 2014： 4512-4519. 10.1109/icra.2014.6907518
15	MUNARO M， FOSSATI A， BASSO A， et al. One-shot person re-identification with a consumer depth camera［M］// GONG S G， CRISTANI M， YAN S C， et al. Person Re-Identification. London： Springer， 2014： 161-181. 10.1007/978-1-4471-6296-4_8
16	BARBOSA I B， CRISTANI M， DEL BUE A， et al. Re-identification with RGB-D sensors［C］// Proceedings of the 2012 European Conference on Computer Vision， LNCS7583. Berlin： Springer， 2012： 433-442.
17	YANG Q Z， WU A C， ZHENG W S. Person re-identification by contour sketch under moderate clothing change［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（6）： 2029-2046. 10.1109/tpami.2019.2960509
18	YU S J， LI S H， CHEN D P， et al. COCAS： a large-scale clothes changing person dataset for re-identification［C］// Proceedings of the 2020 IEEE/CVF International Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 3397-3406. 10.1109/cvpr42600.2020.00346
19	QIAN X L， WANG W X， ZHANG L， et al. Long-term cloth-changing person re-identification［C］// Proceedings of the 2020 Asian Conference on Computer Vision， LNCS12624. Cham： Springer， 2021： 71-88.
20	SABOUR S， FROSST N， HINTON G E. Dynamic routing between capsules［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017：3859-3869.
21	LaLONDE R， BAGCI U. Capsules for object segmentation［EB/OL］. （2018-04-11）［2021-03-12］.. 10.1016/j.media.2020.101889
22	JAISWAL A， ABDALMAGEED W， WU Y， et al. CapsuleGAN： Generative adversarial capsule network［C］// Proceedings of the 2018 European Conference on Computer Vision. Berlin： Springer， 2018： 526-535. 10.1007/978-3-030-11015-4_38
23	HINTON G， SABOUR S， FROSST N. Matrix capsules with EM routing［EB/OL］. ［2021-03-12］.. 10.4324/9781003091127-2
24	HUANG G， LIU Z， MAATEN L VAN DER， et al. Densely connected convolutional networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2261-2269. 10.1109/cvpr.2017.243
25	SRIVASTAVA N， HINTON G， KRIZHEVSKY A， et al. Dropout： a simple way to prevent neural networks from overfitting［J］. Journal of Machine Learning Research， 2014， 15： 1929-1958.
26	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
27	HUANG Y， WU Q， XU J S， et al. Celebrities-ReID： a benchmark for clothes variation in long-term person re-identification［C］// Proceedings of the 2019 International Joint Conference on Neural Networks. Piscataway： IEEE， 2019： 1-8. 10.1109/ijcnn.2019.8851957
28	KINGMA D P， BA J L. Adam： a method for stochastic optimization［EB/OL］. （2017-01-30）［2021-03-12］.
29	YU Q， CHANG X B， SONG Y Z， et al. The devil is in the middle： exploiting mid-level representations for cross-domain instance matching［EB/OL］. （2018-04-04）［2021-03-12］..
30	ZHENG Z D， ZHENG L， YANG Y. A discriminatively learned CNN embedding for person reidentification［J］. ACM Transactions on Multimedia Computing， Communications， and Applications， 2018， 14（1）： No.13. 10.1145/3159171
31	CHANG X B， HOSPEDALES T M， XIANG T. Multi-level factorisation net for person re-identification［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2109-2118. 10.1109/cvpr.2018.00225
32	SUN Y F， ZHENG L， YANG Y， et al. Beyond part models： person retrieval with refined part pooling （and a strong convolutional baseline）［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11208. Cham： Springer， 2018： 501-518.

方法	Celeb-reID			Celeb-reID-light			NKUP
方法	mAP	Rank-1	Rank-5	mAP	Rank-1	Rank-5	mAP	Rank-1	Rank-5
Baseline（ReIDCaps）^［9］	9.7	51.1	65.2	10.2	20.6	45.6	5.3	6.5	8.3
Baseline+Label Smooth	10.4	53.1	66.8	10.4	20.3	47.6	6.3	7.4	12.0
Baseline+Circle Loss	10.5	52.1	67.0	10.9	20.8	48.1	6.8	5.6	11.1
Baseline+Circle Loss+Label Smooth	10.8	51.6	66.8	11.1	21.5	51.9	7.5	8.3	13.0

方法	Celeb-reID			Celeb-reID-light			NKUP
方法	mAP	Rank-1	Rank-5	mAP	Rank-1	Rank-5	mAP	Rank-1	Rank-5
Baseline（ReIDCaps）^［9］	9.7	51.1	65.2	10.2	20.6	45.6	5.3	6.5	8.3
Baseline+Label Smooth	10.4	53.1	66.8	10.4	20.3	47.6	6.3	7.4	12.0
Baseline+Circle Loss	10.5	52.1	67.0	10.9	20.8	48.1	6.8	5.6	11.1
Baseline+Circle Loss+Label Smooth	10.8	51.6	66.8	11.1	21.5	51.9	7.5	8.3	13.0

方法	Celeb-reID			Celeb-reID-light			NKUP
方法	mAP	rank-1	rank-5	mAP	rank-1	rank-5	mAP	rank-1	rank-5
IDE+ （DenseNet-121）	5.9	42.9	56.4	5.3	10.5	24.8	4.3	6.9	10.8
ResNet-Mid^［29］	5.8	43.3	54.6	6.0	10.3	28.0	—	—	—
Two-Stream ^［30］	7.8	36.3	54.5	—	—	—	—	—	—
MLFN ^［31］	6.0	41.4	54.7	6.3	10.6	31.0	—	—	—
PCB ^［32］	8.2	37.1	57.0	—	—	—	4.1	9.9	15.9
ReIDCaps^［9］	9.8	51.2	65.4	10.2	20.6	45.6	5.3	6.5	8.3
本文方法	10.8	51.6	66.8	11.1	21.5	51.9	7.5	8.3	13.0

方法	Celeb-reID			Celeb-reID-light			NKUP
方法	mAP	rank-1	rank-5	mAP	rank-1	rank-5	mAP	rank-1	rank-5
IDE+ （DenseNet-121）	5.9	42.9	56.4	5.3	10.5	24.8	4.3	6.9	10.8
ResNet-Mid^［29］	5.8	43.3	54.6	6.0	10.3	28.0	—	—	—
Two-Stream ^［30］	7.8	36.3	54.5	—	—	—	—	—	—
MLFN ^［31］	6.0	41.4	54.7	6.3	10.6	31.0	—	—	—
PCB ^［32］	8.2	37.1	57.0	—	—	—	4.1	9.9	15.9
ReIDCaps^［9］	9.8	51.2	65.4	10.2	20.6	45.6	5.3	6.5	8.3
本文方法	10.8	51.6	66.8	11.1	21.5	51.9	7.5	8.3	13.0

[1]	高铭蔚, 桑楠, 杨茂林. 基于胶囊网络的交互式网络电视视频点播推荐模型[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3171-3177.
[2]	谌贵辉, 刘会康, 李忠兵, 彭娇, 汪少天, 林瑾瑜. 基于仲裁机制的生成对抗网络改进算法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3185-3191.
[3]	李凯, 岳秉杰. 共享转换矩阵的胶囊网络及其融合视角特征的跨视角步态识别[J]. 计算机应用, 2021, 41(1): 157-163.
[4]	尹春勇, 何苗. 基于改进胶囊网络的文本分类[J]. 计算机应用, 2020, 40(9): 2525-2530.
[5]	王敏蕊, 高曙, 袁自勇, 袁蕾. 基于动态路由序列生成模型的多标签文本分类方法[J]. 计算机应用, 2020, 40(7): 1884-1890.
[6]	曹小威, 曲志坚, 徐玲玲, 刘晓红. 基于胶囊网络的三维模型识别[J]. 计算机应用, 2020, 40(5): 1309-1314.
[7]	陈立潮, 郑佳敏, 曹建芳, 潘理虎, 张睿. 基于胶囊网络的智能交通标志识别方法[J]. 计算机应用, 2020, 40(4): 1045-1049.
[8]	陈立潮, 张雷, 曹建芳, 张睿. 梯度直方图卷积特征的胶囊网络在交通监控下的车型分类[J]. 计算机应用, 2020, 40(10): 2881-2889.

基于联合损失胶囊网络的换衣行人重识别

Cloth-changing person re-identification based on joint loss capsule network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 3

参考文献 32

相关文章 8

编辑推荐

Metrics