Abstract:

Focused on the difficulty of extracting effective information in the cloth-changing person Re-identification （ReID） task， a cloth-changing person re-identification model based on semantic-guided self-attention network was proposed. Firstly， semantic information was used to segment an original image into a cloth-free image. Both images were input into a two-branch multi-head self-attention network to extract cloth-independent features and complete person features， respectively. Then， a Global Feature Reconstruction module （GFR） was designed to reconstruct two global features， in which the clothing region contained head features with better robustness， which made the saliency information in the global features more prominent. And a Local Feature Reorganization and Reconstruction module （LFRR） was proposed to extract the head and shoe features from the original image and the cloth-free image， emphasizing the detailed information about the head and shoe features and reducing the interference caused by changing shoes. Finally， in addition to the identity loss and triplet loss commonly used in person re-identification， Feature Pull Loss （FPL） was proposed to close the distances among local and global features， complete image features and costume-free image features. On the PRCC （Person ReID under moderate Clothing Change） and VC-Clothes （Virtually Changing-Clothes） datasets， the mean Average Precision （mAP） of the proposed model improved by 4.6 and 0.9 percentage points respectively compared to the Clothing-based Adversarial Loss （CAL） model. On the Celeb-reID （Celebrities re-IDentification） and Celeb-reID-light （a light version of Celebrities re-IDentification） datasets， the mAP of the proposed model improved by 0.2 and 5.0 percentage points respectively compared with the Joint Loss Capsule Network （JLCN） model. The experimental results show that the proposed method has certain advantages in highlighting effective information expression in the cloth-changing scenarios.

Key words: cloth-changing person re-identification, multi-head self-attention network, semantic segmentation, feature reconstruction, feature reorganization

摘要：

针对换衣行人重识别（ReID）任务中有效信息提取困难的问题，提出一种基于语义引导自注意力网络的换衣ReID模型。首先，利用语义信息将图像分割出无服装图像，和原始图像一起输入双分支多头自注意力网络进行计算，分别得到衣物无关特征和完整行人特征。其次，利用全局特征重建模块（GFR），重建两种全局特征，得到的新特征中服装区域包含换衣任务中鲁棒性更好的头部特征，使得全局特征中的显著性信息更突出；利用局部特征重组重建模块（LFRR），在完整图像特征和无服装图像特征中提取头部和鞋部局部特征，强调头部和鞋部特征的细节信息，并减少换鞋造成的干扰。最后，除了使用行人重识别中常用的身份损失和三元组损失，提出特征拉近损失（FPL），拉近局部与全局特征、完整图像特征与无服装图像特征之间的距离。在PRCC（Person ReID under moderate Clothing Change）和VC-Clothes（Virtually Changing-Clothes）数据集上，与基于衣物对抗损失（CAL）模型相比，所提模型的平均精确率均值（mAP）分别提升了4.6和0.9个百分点；在Celeb-reID和Celeb-reID-light数据集上，与联合损失胶囊网络 （JLCN）模型相比，所提模型的mAP分别提升了0.2和 5.0个百分点。实验结果表明，所提模型在换衣场景中突出有效信息表达方面具有一定优势。

关键词: 换衣行人重识别, 多头自注意力网络, 语义分割, 特征重建, 特征重组