基于语义引导自注意力网络的换衣行人重识别模型

doi:10.11772/j.issn.1001-9081.2022121875

《计算机应用》唯一官方网站 ›› 2023, Vol. 43 ›› Issue (12): 3719-3726.DOI: 10.11772/j.issn.1001-9081.2022121875

基于语义引导自注意力网络的换衣行人重识别模型

钟建华¹, 邱创一¹^,², 巢建树², 明瑞成², 钟剑锋¹()

^1.福州大学先进制造学院，福建泉州 362000
^2.中国科学院海西研究院泉州装备制造研究中心，福建泉州 362000

收稿日期:2022-12-26 修回日期:2023-02-23 接受日期:2023-02-28 发布日期:2023-03-13 出版日期:2023-12-10
通讯作者: 钟剑锋
作者简介:钟建华（1985—），男，福建龙岩人，副教授，博士，主要研究方向：图像处理、模式识别
邱创一（1998—），男，福建福州人，硕士研究生，主要研究方向：图像处理、行人重识别
巢建树（1984—），男，江苏江阴人，研究员，博士，主要研究方向：图像处理、深度学习
明瑞成（1994—），男，湖北十堰人，工程师，硕士，主要研究方向：图像处理、深度学习；
基金资助:
国家自然科学基金资助项目(52275523)

Cloth-changing person re-identification model based on semantic-guided self-attention network

Jianhua ZHONG¹, Chuangyi QIU¹^,², Jianshu CHAO², Ruicheng MING², Jianfeng ZHONG¹()

^1.School of Advanced Manufacturing，Fuzhou University，Quanzhou Fujian 362000，China
^2.Quanzhou Institute of Equipment Manufacturing，Haixi Institute，Chinese Academy of Sciences，Quanzhou Fujian 362000，China

Received:2022-12-26 Revised:2023-02-23 Accepted:2023-02-28 Online:2023-03-13 Published:2023-12-10
Contact: Jianfeng ZHONG
About author:ZHONG Jianhua， born in 1985， Ph. D.， associate professor. His research interests include image processing， pattern recognition.
QIU Chuangyi， born in 1998， M. S. candidate. His research interests include image processing， person re-identification.
CHAO Jianshu， born in 1984， Ph. D.， research fellow. His research interests include image processing， deep learning.
MING Ruicheng， born in 1994， M. S.， engineer. His research interests include image processing， deep learning.
Supported by:
National Natural Science Foundation of China(52275523)

摘要/Abstract

摘要：

针对换衣行人重识别（ReID）任务中有效信息提取困难的问题，提出一种基于语义引导自注意力网络的换衣ReID模型。首先，利用语义信息将图像分割出无服装图像，和原始图像一起输入双分支多头自注意力网络进行计算，分别得到衣物无关特征和完整行人特征。其次，利用全局特征重建模块（GFR），重建两种全局特征，得到的新特征中服装区域包含换衣任务中鲁棒性更好的头部特征，使得全局特征中的显著性信息更突出；利用局部特征重组重建模块（LFRR），在完整图像特征和无服装图像特征中提取头部和鞋部局部特征，强调头部和鞋部特征的细节信息，并减少换鞋造成的干扰。最后，除了使用行人重识别中常用的身份损失和三元组损失，提出特征拉近损失（FPL），拉近局部与全局特征、完整图像特征与无服装图像特征之间的距离。在PRCC（Person ReID under moderate Clothing Change）和VC-Clothes（Virtually Changing-Clothes）数据集上，与基于衣物对抗损失（CAL）模型相比，所提模型的平均精确率均值（mAP）分别提升了4.6和0.9个百分点；在Celeb-reID和Celeb-reID-light数据集上，与联合损失胶囊网络（JLCN）模型相比，所提模型的mAP分别提升了0.2和 5.0个百分点。实验结果表明，所提模型在换衣场景中突出有效信息表达方面具有一定优势。

关键词: 换衣行人重识别, 多头自注意力网络, 语义分割, 特征重建, 特征重组

Abstract:

Focused on the difficulty of extracting effective information in the cloth-changing person Re-identification （ReID） task， a cloth-changing person re-identification model based on semantic-guided self-attention network was proposed. Firstly， semantic information was used to segment an original image into a cloth-free image. Both images were input into a two-branch multi-head self-attention network to extract cloth-independent features and complete person features， respectively. Then， a Global Feature Reconstruction module （GFR） was designed to reconstruct two global features， in which the clothing region contained head features with better robustness， which made the saliency information in the global features more prominent. And a Local Feature Reorganization and Reconstruction module （LFRR） was proposed to extract the head and shoe features from the original image and the cloth-free image， emphasizing the detailed information about the head and shoe features and reducing the interference caused by changing shoes. Finally， in addition to the identity loss and triplet loss commonly used in person re-identification， Feature Pull Loss （FPL） was proposed to close the distances among local and global features， complete image features and costume-free image features. On the PRCC （Person ReID under moderate Clothing Change） and VC-Clothes （Virtually Changing-Clothes） datasets， the mean Average Precision （mAP） of the proposed model improved by 4.6 and 0.9 percentage points respectively compared to the Clothing-based Adversarial Loss （CAL） model. On the Celeb-reID （Celebrities re-IDentification） and Celeb-reID-light （a light version of Celebrities re-IDentification） datasets， the mAP of the proposed model improved by 0.2 and 5.0 percentage points respectively compared with the Joint Loss Capsule Network （JLCN） model. The experimental results show that the proposed method has certain advantages in highlighting effective information expression in the cloth-changing scenarios.

Key words: cloth-changing person re-identification, multi-head self-attention network, semantic segmentation, feature reconstruction, feature reorganization

中图分类号:

TP391.41

钟建华, 邱创一, 巢建树, 明瑞成, 钟剑锋. 基于语义引导自注意力网络的换衣行人重识别模型[J]. 计算机应用, 2023, 43(12): 3719-3726.

Jianhua ZHONG, Chuangyi QIU, Jianshu CHAO, Ruicheng MING, Jianfeng ZHONG. Cloth-changing person re-identification model based on semantic-guided self-attention network[J]. Journal of Computer Applications, 2023, 43(12): 3719-3726.

图/表 10

图1 本文模型的整体框架

Fig.1 Overall framework of proposed model

图2 本文模型的训练流程

Fig.2 Training flow of proposed model

图3 全局特征重建模块

Fig.3 Global feature reconstruction module

图4 局部特征重组重建模块

Fig.4 Local feature reorganization and reconstruction module

表1 在PRCC数据集上的消融实验结果 ( %)

Tab.1 Ablation experimental results on PRCC dataset

模型	Rank-1	Rank-5	Rank-10	mAP
Baseline	51.8	58.0	61.2	51.7
SGN	58.5	67.2	71.6	58.8
SGN+GFR	60.0	68.2	71.9	58.9
SGN+GFR+LFRR	61.0	69.2	72.8	59.5
SGN+GFR+LFRR+FPL	63.7	70.6	73.8	60.4

表2 特征平均分块和特征重组重建模块在PRCC数据集上的性能比较 (%)

Tab. 2 Performance comparison of feature averagely chunking and feature reorganization and reconstruction modules on PRCC dataset

模块策略	Rank-1	Rank-5	Rank-10	mAP
特征平均分块	62.9	70.3	73.8	60.4
特征重组重建	63.7	70.6	73.8	60.4

表3 不同特征拉近损失函数策略在PRCC数据集的性能比较 (%)

Tab.3 Performance comparison of different FPL function strategies on PRCC dataset

损失函数策略	Rank-1	Rank-5	Rank-10	mAP
No FPL	61.0	69.2	72.8	59.5
FPL-LG	61.8	69.8	73.7	59.7
FPL-TG	63.3	70.5	73.3	59.6
FPL	63.7	70.6	73.8	60.4

表4 不同方法在PRCC和VC-Clothes数据集上的性能比较 (%)

Tab.4 Performance comparison of different methods on PRCC and VC-Clothes datasets

方法	PRCC		VC-Clothes
方法	Rank-1	mAP	Rank-1	mAP
HACNN^［27］	21.8	—	—	—
PCB^［5］	41.8	38.7	62.0	62.2
TransReID^［28］	51.8	51.7	84.5	76.3
SPT^［2］	34.4	—	—	—
Part-aligned^［24］	—	—	69.4	67.3
GI-ReID^［11］	33.3	—	64.5	57.8
3DSL^［12］	51.3	—	79.9	81.2
FSAM^［13］	54.5	—	78.6	78.9
CAL^［14］	55.2	55.8	81.4	81.7
SGSNet	63.7	60.4	88.9	82.6

表5 不同方法在Celeb-reID和Celeb-reID-light数据集上性能比较 (%)

Tab.5 Performance comparison of different methods on Celeb-reID and Celeb-reID-light datasets

方法	Celeb-reID		Celeb-reID-light
方法	Rank-1	mAP	Rank-1	mAP
HACNN^［27］	47.6	9.5	16.2	11.5
PCB^［5］	37.1	8.2	—	—
MGN^［29］	49.0	10.8	21.5	13.9
ReIDCaps^［9］	51.2	9.8	20.6	10.2
JLCN^［8］	51.6	10.8	21.5	11.1
SGSNet	53.0	11.0	25.8	16.1

图5 使用Baseline模型与本文模型返回的前5张检索图像对比

Fig. 5 Comparison of top five retrieved images returned by Baseline model and proposed model

参考文献 29

1	罗浩，姜伟，范星，等.基于深度学习的行人重识别研究进展［J］. 自动化学报， 2019， 45（11）： 2032-2049. 10.16383/j.aas.c180154
	LUO H， JIANG W， FAN X， et al. A survey on deep learning based on person re-identification ［J］. Acta Automatica Sinica， 2019， 45（11）： 2032-2049. 10.16383/j.aas.c180154
2	YANG Q， WU A， ZHENG W-S. Person re-identification by contour sketch under moderate clothing change ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（6）： 2029-2046. 10.1109/tpami.2019.2960509
3	LUO H， JIANG W， GU Y， et al. A strong baseline and batch normalization neck for deep person re-identification ［J］. IEEE Transactions on Multimedia， 2019， 22（10）： 2597-2609. 10.1109/tmm.2019.2958756
4	ZHONG Z， ZHENG L， KANG G， et al. Random erasing data augmentation ［C］// Proceedings of the 2020 AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 13001-13008. 10.1609/aaai.v34i07.7000
5	SUN Y， ZHENG L， YANG Y， et al. Beyond part models： person retrieval with refined part pooling （and a strong convolutional baseline）［C］// Proceedings of the 2018 European Conference on Computer Vision. Cham： Springer， 2018： 501-518. 10.1007/978-3-030-01225-0_30
6	WANG G， YUAN Y， CHEN X， et al. Learning discriminative features with multiple granularities for person re-identification ［C］// Proceedings of the 26th ACM International Conference on Multimedia. New York： ACM， 2018： 274-282. 10.1145/3240508.3240552
7	龚云鹏，曾智勇，叶锋. 基于灰度域特征增强的行人重识别方法［J］. 计算机应用， 2021， 41（12）： 3590-3595. 10.11772/j.issn.1001-9081.2021061011
	GONG Y P， ZENG Z Y， YE F. Person re-identification method based on grayscale feature enhancement ［J］. Journal of Computer Applications， 2021， 41（12）： 3590-3595. 10.11772/j.issn.1001-9081.2021061011
8	刘乾，王洪元，曹亮，等.基于联合损失胶囊网络的换衣行人重识别［J］. 计算机应用， 2021， 41（12）： 3596-3601.
	LIU Q， WANG H Y， CAO L， et al. Cloth-changing person re-identification based on joint loss capsule network ［J］. Journal of Computer Applications， 2021， 41（12）： 3596-3601.
9	HUANG Y， XU J， WU Q， et al. Beyond scalar neuron： adopting vector-neuron capsules for long-term person re-identification ［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2019， 30（10）： 3459-3471. 10.1109/tcsvt.2019.2948093
10	SUN Y， CHENG C， ZHANG Y， et al. Circle loss： a unified perspective of pair similarity optimization ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 6397-6406. 10.1109/cvpr42600.2020.00643
11	JIN X， HE T， ZHENG K， et al. Cloth-changing person re-identification from a single image with gait prediction and regularization ［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 14258-14267. 10.1109/cvpr52688.2022.01388
12	CHEN J， JIANG X， WANG F， et al. Learning 3D shape feature for texture-insensitive person re-identification ［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 8142-8151. 10.1109/cvpr46437.2021.00805
13	HONG P， WU T， WU A， et al. Fine-grained shape-appearance mutual learning for cloth-changing person re-identification ［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 10508-10517. 10.1109/cvpr46437.2021.01037
14	GU X， CHANG H， MA B， et al. Clothes-changing person re-identification with RGB modality only ［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 1050-1059. 10.1109/cvpr52688.2022.00113
15	DOSOVITSKIY A， BEYER L， KOLESNIKOV A， et al. An image is worth 16×16 words： Transformers for image recognition at scale ［EB/OL］. （2021-06-03）［2022-12-13］. .
16	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition ［EB/OL］. （2015-04-10）［2022-12-13］. .
17	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
18	HUANG G， LIU Z， VAN DER MAATEN L， et al. Densely connected convolutional networks ［C］// Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2261-2269. 10.1109/cvpr.2017.243
19	SUN Y， ZHENG L， DENG W， et al. SVDNet for pedestrian retrieval ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 3820-3828. 10.1109/iccv.2017.410
20	LAWEN H， BEN-COHEN A， PROTTER M， et al. Compact network training for person ReID ［C］// Proceedings of the 2020 International Conference on Multimedia Retrieval. New York： ACM， 2020： 164-171. 10.1145/3372278.3390686
21	SCHROFF F， KALENICHENKO D， PHILBIN J. FaceNet： a unified embedding for face recognition and clustering ［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 815-823. 10.1109/cvpr.2015.7298682
22	HERMANS A， BEYER L， LEIBE B. In defense of the triplet loss for person re-identification ［EB/OL］. （2018-03-24）［2022-12-13］. . 10.21203/rs.3.rs-1501673/v1
23	IOFFE S， SZEGEDY C. Batch normalization： accelerating deep network training by reducing internal covariate shift ［C］// Proceedings of the 32nd International Conference on Machine Learning. New York： ACM， 2015： 448-456.
24	WAN F， WU Y， QIAN X， et al. When person re-identification meets changing clothes ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 3620-3628. 10.1109/cvprw50498.2020.00423
25	GONG K， LIANG X， ZHANG D， et al. Look into person： self-supervised structure-sensitive learning and a new benchmark for human parsing ［C］// Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 6757-6765. 10.1109/cvpr.2017.715
26	LI P， XU Y， WEI Y， et al. Self-correction for human parsing ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 44（6）： 3260-3271.
27	LI W， ZHU X， GONG S. Harmonious attention network for person re-identification ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2285-2294. 10.1109/cvpr.2018.00243
28	HE S， LUO H， WANG P， et al. TransReID： Transformer-based object re-identification ［C］// Proceedings of the 2021 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2021： 14993-15002. 10.1109/iccv48922.2021.01474
29	WANG G， YUAN Y， CHEN X， et al. Learning discriminative features with multiple granularities for person re-identification ［C］// Proceedings of the 26th ACM International Conference on Multimedia. New York： ACM， 2018： 274-282. 10.1145/3240508.3240552

[1]	周迪, 张自力, 陈佳, 胡新荣, 何儒汉, 张俊. 基于EfficientNetV2和物体上下文表示的胃癌图像分割方法[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2955-2962.
[2]	郑帅, 张晓龙, 邓鹤, 任宏伟. 基于多尺度特征融合和网格注意力机制的三维肝脏影像分割方法[J]. 《计算机应用》唯一官方网站, 2023, 43(7): 2303-2310.
[3]	鲁斌, 柳杰林. 基于特征增强的三维点云语义分割[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1818-1825.
[4]	袁泉, 徐雲鹏, 唐成亮. 基于路径标签的文档级关系抽取方法[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1029-1035.
[5]	何雪东, 宣士斌, 王款, 陈梦楠. 融合累积分布函数和通道注意力机制的DeepLabV3+图像分割算法[J]. 《计算机应用》唯一官方网站, 2023, 43(3): 936-942.
[6]	廉飞宇, 张良, 王杰栋, 靳于康, 柴玉. 基于图模型与注意力机制的室外场景点云分割模型[J]. 《计算机应用》唯一官方网站, 2023, 43(12): 3911-3917.
[7]	虞资兴, 瞿绍军, 何鑫, 王卓. 高低维特征引导的实时语义分割网络[J]. 《计算机应用》唯一官方网站, 2023, 43(10): 3077-3085.
[8]	郝雯, 汪洋, 魏海南. 基于多特征融合的点云场景语义分割[J]. 《计算机应用》唯一官方网站, 2023, 43(10): 3202-3208.
[9]	王娟, 袁旭亮, 武明虎, 郭力权, 刘子杉. 基于压缩提炼网络的实时语义分割方法[J]. 《计算机应用》唯一官方网站, 2022, 42(7): 1993-2000.
[10]	鲁永帅, 唐英杰, 马鑫然. 基于深度特征融合的无纺布低对比度浆丝缺陷检测方法[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1440-1446.
[11]	余娜, 刘彦, 魏雄炬, 万源. 基于注意力机制和金字塔融合的RGB-D室内场景语义分割[J]. 《计算机应用》唯一官方网站, 2022, 42(3): 844-853.
[12]	吴奇文, 王建华, 郑翔, 冯居, 姜洪岩, 王昱博. 基于改进U-Net的水草图像分割方法[J]. 《计算机应用》唯一官方网站, 2022, 42(10): 3177-3183.
[13]	杨贞, 彭小宝, 朱强强, 殷志坚. 基于Deeplab V3 Plus的自适应注意力机制图像分割算法[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 230-238.
[14]	冯兴杰, 张天泽. 基于分组卷积进行特征融合的全景分割算法[J]. 计算机应用, 2021, 41(7): 2054-2061.
[15]	胡嵽, 冯子亮. 基于深度学习的轻量级道路图像语义分割算法[J]. 计算机应用, 2021, 41(5): 1326-1331.

基于语义引导自注意力网络的换衣行人重识别模型

Cloth-changing person re-identification model based on semantic-guided self-attention network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 29

相关文章 15

编辑推荐

Metrics