Cloth-changing person re-identification based on joint loss capsule network

doi:10.11772/j.issn.1001-9081.2021061090

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (12): 3596-3601.DOI: 10.11772/j.issn.1001-9081.2021061090

Special Issue: 第十八届中国机器学习会议(CCML 2021)

• The 18th China Conference on Machine Learning • Previous Articles Next Articles

Cloth-changing person re-identification based on joint loss capsule network

Qian LIU¹, Hongyuan WANG¹(), Liang CAO¹, Boyan SUN¹, Yu XIAO², Ji ZHANG¹

^1.School of Computer Science and Artificial Intelligence/ Aliyun School of Big Data，Changzhou University，Changzhou Jiangsu 213164，China
^2.Design Art College，Changzhou Vocational Institute of Engineering，Changzhou Jiangsu 213164，China

Received:2021-05-12 Revised:2021-08-05 Accepted:2021-08-17 Online:2021-12-28 Published:2021-12-10
Contact: Hongyuan WANG
About author:LIU Qian， born in 1993， M. S. candidate. His research interests include computer vision.
CAO Liang， born in 1996， M. S. candidate. His research interests include computer vision.
SUN Boyan， born in 1996， M. S. candidate. His research interests include computer vision.
XIAO Yu， born in 1981， M. S.， associate professor. Her research interests include digital media technology， graphics and image processing.
ZHANG Ji， born in 1981， M. S.， lecturer. His research interests include pattern recognition， image processing， machine vision.
Supported by:
the Surface Program of National Natural Science Foundation of China(61976028)

基于联合损失胶囊网络的换衣行人重识别

刘乾¹, 王洪元¹(), 曹亮¹, 孙博言¹, 肖宇², 张继¹

^1.常州大学计算机与人工智能学院、阿里云大数据学院，江苏常州 213164
^2.常州工程职业技术学院设计艺术学院，江苏常州 213164

通讯作者: 王洪元
作者简介:刘乾（1993—），男，江苏南京人，硕士研究生，主要研究方向：计算机视觉
曹亮（1996—），江苏盐城人，硕士研究生，主要研究方向：计算机视觉
孙博言（1996—），黑龙江双鸭山人，硕士研究生，主要研究方向：计算机视觉
肖宇（1981—），女，黑龙江伊春人，副教授，硕士，主要研究方向：数字媒体技术、图形图像处理
张继（1981—），男，江苏常州人，讲师，硕士，主要研究方向：模式识别、图像处理、机器视觉。
基金资助:
国家自然科学基金面上项目(61976028)

Abstract

Abstract:

Current research on Person Re-Identification （Re-ID） mainly concentrates on short-term situations with person’s clothing usually unchanged. However， more common practical cases are long-term situations， in which a person has higher possibility to change his clothes， which should be considered by Re-ID models. Therefore， a method of person re-identification with cloth changing based on joint loss capsule network was proposed. The proposed method was based on ReIDCaps， a capsule network for cloth-changing person re-identification. In the method， vector-neuron capsules that contain more information than traditional scalar neurons were used. The length of the vector-neuron capsule was used to represent the identity information of the person， and the direction of the capsule was used to represent the clothing information of the person. Soft Embedding Attention （SEA） was used to avoid the model over-fitting. Feature Sparse Representation （FSR） mechanism was adopted to extract discriminative features. The joint loss of label smoothing regularization cross-entropy loss and Circle Loss was added to improve the generalization ability and robustness of the model. Experimental results on three datasets including Celeb-reID， Celeb-reID-light and NKUP prove that the proposed method has certain advantages compared with the existing person re-identification methods.

Key words: cloth-changing person re-identification, capsule network, vector-neuron capsule, label smoothing regularization cross-entropy loss, Circle Loss

摘要：

目前的行人重识别（Re-ID）研究主要集中在短时间情形，即一个人的衣着不太可能发生改变的情况。然而现实中更常见的是长时间的情况，这时一个人有很大的机会更换衣服，Re-ID模型应该考虑这种情况。为此，研究了一种基于联合损失胶囊网络的换衣行人重识别方法。所提方法基于换衣行人重识别胶囊网络ReIDCaps，使用与传统的标量神经元相比包含更多信息的矢量胶囊，用其长度表示行人身份信息，用其方向表示行人衣着信息；采用软嵌入注意力（SEA）防止模型过拟合；使用特征稀疏表示（FSR）机制提取具有判别性的特征；增加标签平滑正则化交叉熵损失与Circle Loss的联合损失以提高模型的泛化能力和鲁棒性。在三个换衣行人重识别数据集Celeb-reID、Celeb-reID-light和NKUP上进行实验，实验结果表明所提方法与目前已有的Re-ID方法相比具有一定优势。

关键词: 换衣行人重识别, 胶囊网络, 矢量胶囊, 标签平滑正则化交叉熵损失, Circle Loss

CLC Number:

TP391.41

Qian LIU, Hongyuan WANG, Liang CAO, Boyan SUN, Yu XIAO, Ji ZHANG. Cloth-changing person re-identification based on joint loss capsule network[J]. Journal of Computer Applications, 2021, 41(12): 3596-3601.

刘乾, 王洪元, 曹亮, 孙博言, 肖宇, 张继. 基于联合损失胶囊网络的换衣行人重识别[J]. 《计算机应用》唯一官方网站, 2021, 41(12): 3596-3601.

Figures/Tables 3

References 32

1	GONG S G， CRISTANI M， LOY C C， et al. The re-identification challenge［M］// GONG S G， CRISTANI M， YAN S C， et al. Person Re-Identification. London： Springer， 2014： 1-20. 10.1007/978-1-4471-6296-4_1
2	GRAY D， TAO H. Viewpoint invariant pedestrian recognition with an ensemble of localized features［C］// Proceedings of the 2008 European Conference on Computer Vision， LNCS5302. Berlin： Springer， 2008： 262-275.
3	LI W， ZHAO R， XIAO T， et al. DeepReID： deep filter pairing neural network for person re-identification［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2014： 152-159. 10.1109/cvpr.2014.27
4	ZHENG L， SHEN L Y， TIAN L， et al. Scalable person re-identification： a benchmark［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1116-1124. 10.1109/iccv.2015.133
5	ZHENG Z D， ZHENG L， YANG Y. Unlabeled samples generated by GAN improve the person re-identification baseline in vitro［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 3774-3782. 10.1109/iccv.2017.405
6	LUO H， JIANG W， GU Y Z， et al. A strong baseline and batch normalization neck for deep person re-identification［J］. IEEE Transactions on Multimedia， 2020， 22（10）： 2597-2609. 10.1109/tmm.2019.2958756
7	LUO H， JIANG W， ZHANG X， et al. AlignedReID++： dynamically matching local information for person re-identification［J］. Pattern Recognition， 2019， 94： 53-61. 10.1016/j.patcog.2019.05.028
8	WANG G S， YUAN Y F， CHEN X， et al. Learning discriminative features with multiple granularities for person re-identification［C］// Proceedings of the 26th ACM International Conference on Multimedia. New York： ACM， 2018： 274-282. 10.1145/3240508.3240552
9	HUANG Y， XU J S， WU Q， et al. Beyond scalar neuron： adopting vector-neuron capsules for long-term person re-identification［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2020， 30（10）： 3459-3471. 10.1109/tcsvt.2019.2948093
10	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2818-2826. 10.1109/cvpr.2016.308
11	SUN Y F， CHENG C M， ZHANG Y H， et al. Circle loss： a unified perspective of pair similarity optimization［C］// Proceedings of the 2020 IEEE/CVF International Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 6397-6406. 10.1109/cvpr42600.2020.00643
12	WANG K， MA Z， CHEN S Y， et al. A benchmark for clothes variation in person re-identification［J］. International Journal of Intelligent Systems， 2020， 35（12）： 1881-1898. 10.1002/int.22276
13	HAQUE A， ALAHI A， LI F F. Recurrent attention models for depth-based person identification［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1229-1238. 10.1109/cvpr.2016.138
14	MUNARO M， BASSO A， FOSSATI A， et al. 3D reconstruction of freely moving persons for re-identification with a depth sensor［C］// Proceedings of the 2014 IEEE International Conference on Robotics and Automation. Piscataway： IEEE， 2014： 4512-4519. 10.1109/icra.2014.6907518
15	MUNARO M， FOSSATI A， BASSO A， et al. One-shot person re-identification with a consumer depth camera［M］// GONG S G， CRISTANI M， YAN S C， et al. Person Re-Identification. London： Springer， 2014： 161-181. 10.1007/978-1-4471-6296-4_8
16	BARBOSA I B， CRISTANI M， DEL BUE A， et al. Re-identification with RGB-D sensors［C］// Proceedings of the 2012 European Conference on Computer Vision， LNCS7583. Berlin： Springer， 2012： 433-442.
17	YANG Q Z， WU A C， ZHENG W S. Person re-identification by contour sketch under moderate clothing change［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（6）： 2029-2046. 10.1109/tpami.2019.2960509
18	YU S J， LI S H， CHEN D P， et al. COCAS： a large-scale clothes changing person dataset for re-identification［C］// Proceedings of the 2020 IEEE/CVF International Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 3397-3406. 10.1109/cvpr42600.2020.00346
19	QIAN X L， WANG W X， ZHANG L， et al. Long-term cloth-changing person re-identification［C］// Proceedings of the 2020 Asian Conference on Computer Vision， LNCS12624. Cham： Springer， 2021： 71-88.
20	SABOUR S， FROSST N， HINTON G E. Dynamic routing between capsules［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017：3859-3869.
21	LaLONDE R， BAGCI U. Capsules for object segmentation［EB/OL］. （2018-04-11）［2021-03-12］.. 10.1016/j.media.2020.101889
22	JAISWAL A， ABDALMAGEED W， WU Y， et al. CapsuleGAN： Generative adversarial capsule network［C］// Proceedings of the 2018 European Conference on Computer Vision. Berlin： Springer， 2018： 526-535. 10.1007/978-3-030-11015-4_38
23	HINTON G， SABOUR S， FROSST N. Matrix capsules with EM routing［EB/OL］. ［2021-03-12］.. 10.4324/9781003091127-2
24	HUANG G， LIU Z， MAATEN L VAN DER， et al. Densely connected convolutional networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2261-2269. 10.1109/cvpr.2017.243
25	SRIVASTAVA N， HINTON G， KRIZHEVSKY A， et al. Dropout： a simple way to prevent neural networks from overfitting［J］. Journal of Machine Learning Research， 2014， 15： 1929-1958.
26	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
27	HUANG Y， WU Q， XU J S， et al. Celebrities-ReID： a benchmark for clothes variation in long-term person re-identification［C］// Proceedings of the 2019 International Joint Conference on Neural Networks. Piscataway： IEEE， 2019： 1-8. 10.1109/ijcnn.2019.8851957
28	KINGMA D P， BA J L. Adam： a method for stochastic optimization［EB/OL］. （2017-01-30）［2021-03-12］.
29	YU Q， CHANG X B， SONG Y Z， et al. The devil is in the middle： exploiting mid-level representations for cross-domain instance matching［EB/OL］. （2018-04-04）［2021-03-12］..
30	ZHENG Z D， ZHENG L， YANG Y. A discriminatively learned CNN embedding for person reidentification［J］. ACM Transactions on Multimedia Computing， Communications， and Applications， 2018， 14（1）： No.13. 10.1145/3159171
31	CHANG X B， HOSPEDALES T M， XIANG T. Multi-level factorisation net for person re-identification［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2109-2118. 10.1109/cvpr.2018.00225
32	SUN Y F， ZHENG L， YANG Y， et al. Beyond part models： person retrieval with refined part pooling （and a strong convolutional baseline）［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11208. Cham： Springer， 2018： 501-518.

方法	Celeb-reID			Celeb-reID-light			NKUP
方法	mAP	Rank-1	Rank-5	mAP	Rank-1	Rank-5	mAP	Rank-1	Rank-5
Baseline（ReIDCaps）^［9］	9.7	51.1	65.2	10.2	20.6	45.6	5.3	6.5	8.3
Baseline+Label Smooth	10.4	53.1	66.8	10.4	20.3	47.6	6.3	7.4	12.0
Baseline+Circle Loss	10.5	52.1	67.0	10.9	20.8	48.1	6.8	5.6	11.1
Baseline+Circle Loss+Label Smooth	10.8	51.6	66.8	11.1	21.5	51.9	7.5	8.3	13.0

方法	Celeb-reID			Celeb-reID-light			NKUP
方法	mAP	Rank-1	Rank-5	mAP	Rank-1	Rank-5	mAP	Rank-1	Rank-5
Baseline（ReIDCaps）^［9］	9.7	51.1	65.2	10.2	20.6	45.6	5.3	6.5	8.3
Baseline+Label Smooth	10.4	53.1	66.8	10.4	20.3	47.6	6.3	7.4	12.0
Baseline+Circle Loss	10.5	52.1	67.0	10.9	20.8	48.1	6.8	5.6	11.1
Baseline+Circle Loss+Label Smooth	10.8	51.6	66.8	11.1	21.5	51.9	7.5	8.3	13.0

方法	Celeb-reID			Celeb-reID-light			NKUP
方法	mAP	rank-1	rank-5	mAP	rank-1	rank-5	mAP	rank-1	rank-5
IDE+ （DenseNet-121）	5.9	42.9	56.4	5.3	10.5	24.8	4.3	6.9	10.8
ResNet-Mid^［29］	5.8	43.3	54.6	6.0	10.3	28.0	—	—	—
Two-Stream ^［30］	7.8	36.3	54.5	—	—	—	—	—	—
MLFN ^［31］	6.0	41.4	54.7	6.3	10.6	31.0	—	—	—
PCB ^［32］	8.2	37.1	57.0	—	—	—	4.1	9.9	15.9
ReIDCaps^［9］	9.8	51.2	65.4	10.2	20.6	45.6	5.3	6.5	8.3
本文方法	10.8	51.6	66.8	11.1	21.5	51.9	7.5	8.3	13.0

方法	Celeb-reID			Celeb-reID-light			NKUP
方法	mAP	rank-1	rank-5	mAP	rank-1	rank-5	mAP	rank-1	rank-5
IDE+ （DenseNet-121）	5.9	42.9	56.4	5.3	10.5	24.8	4.3	6.9	10.8
ResNet-Mid^［29］	5.8	43.3	54.6	6.0	10.3	28.0	—	—	—
Two-Stream ^［30］	7.8	36.3	54.5	—	—	—	—	—	—
MLFN ^［31］	6.0	41.4	54.7	6.3	10.6	31.0	—	—	—
PCB ^［32］	8.2	37.1	57.0	—	—	—	4.1	9.9	15.9
ReIDCaps^［9］	9.8	51.2	65.4	10.2	20.6	45.6	5.3	6.5	8.3
本文方法	10.8	51.6	66.8	11.1	21.5	51.9	7.5	8.3	13.0

[1]	Zexi JIN, Lei LI, Ji LIU. Transfer learning model based on improved domain separation network [J]. Journal of Computer Applications, 2023, 43(8): 2382-2389.
[2]	Qinghai XU, Shifei DING, Tongfeng SUN, Jian ZHANG, Lili GUO. Improved capsule network based on multipath feature [J]. Journal of Computer Applications, 2023, 43(5): 1330-1335.
[3]	Hao SUN, Jian CAO, Haisheng LI, Dianhui MAO. Session-based recommendation model based on enhanced capsule network [J]. Journal of Computer Applications, 2023, 43(4): 1043-1049.
[4]	Jianhua ZHONG, Chuangyi QIU, Jianshu CHAO, Ruicheng MING, Jianfeng ZHONG. Cloth-changing person re-identification model based on semantic-guided self-attention network [J]. Journal of Computer Applications, 2023, 43(12): 3719-3726.
[5]	Kai WEN, Xiao XUE, Juan JI. Shared transformation matrix capsule network for complex image classification [J]. Journal of Computer Applications, 2023, 43(11): 3411-3417.
[6]	Heng CHEN, Siyi WANG, Zhengguang LI, Guanyu LI, Xin LIU. Capsule network knowledge graph embedding model based on relational memory [J]. Journal of Computer Applications, 2022, 42(7): 1985-1992.
[7]	Jiana MENG, Pin LYU, Yuhai YU, Shichang SUN, Hongfei LIN. Aspect-level cross-domain sentiment analysis based on capsule network [J]. Journal of Computer Applications, 2022, 42(12): 3700-3707.
[8]	Xiaopeng WANG, Yuanyuan SUN, Hongfei LIN. Encoding-decoding relationship extraction model based on criminal Electra [J]. Journal of Computer Applications, 2022, 42(1): 87-93.
[9]	Guihui CHEN, Huikang LIU, Zhongbing LI, Jiao PENG, Shaotian WANG, Jinyu LIN. Improved algorithm of generative adversarial network based on arbitration mechanism [J]. Journal of Computer Applications, 2021, 41(11): 3185-3191.
[10]	Mingwei GAO, Nan SANG, Maolin YANG. IPTV video-on-demand recommendation model based on capsule network [J]. Journal of Computer Applications, 2021, 41(11): 3171-3177.
[11]	LI Kai, YUE Bingjie. Capsule network based on sharing transformation matrix and its cross-view gait recognition fused with view features [J]. Journal of Computer Applications, 2021, 41(1): 157-163.
[12]	YIN Chunyong, HE Miao. Text classification based on improved capsule network [J]. Journal of Computer Applications, 2020, 40(9): 2525-2530.
[13]	WANG Minrui, GAO Shu, YUAN Ziyong, YUAN Lei. Sequence generation model with dynamic routing for multi-label text classification [J]. Journal of Computer Applications, 2020, 40(7): 1884-1890.
[14]	CAO Xiaowei, QU Zhijian, XU Lingling, LIU Xiaohong. 3D model recognition based on capsule network [J]. Journal of Computer Applications, 2020, 40(5): 1309-1314.
[15]	CHEN Lichao, ZHENG Jiamin, CAO Jianfang, PAN Lihu, ZHANG Rui. Intelligent traffic sign recognition method based on capsule network [J]. Journal of Computer Applications, 2020, 40(4): 1045-1049.

Cloth-changing person re-identification based on joint loss capsule network

基于联合损失胶囊网络的换衣行人重识别

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 3

References 32

Related Articles 15

Recommended Articles

Metrics