Abstract:

To address the problem of inaccurate 3D hand pose estimation from a single RGB image due to occlusion and self-similarity， a 3D hand pose estimation network combining attention mechanism and multi-scale feature fusion was proposed. Firstly， Sensory Enhancement Module （SEM） was proposed， which combined dilated convolution and CBAM （Convolutional Block Attention Module） attention mechanism， and it was used to replace the Basicblock of HourGlass Network （HGNet） to expand the receptive field and enhance the sensitivity to spatial information， so as to improve the ability of extracting hand features. Secondly， a multi-scale information fusion module SS-MIFM （SPCNet and Soft-attention-Multi-scale Information Fusion Module） combining SPCNet （Spatial Preserve and Content-aware Network） and Soft-Attention enhancement was designed to aggregate multi-level features effectively and improve the accuracy of 2D hand keypoint detection significantly with full consideration of the spatial content awareness mechanism. Finally， a 2.5D pose conversion module was proposed to convert 2D pose into 3D pose， thereby avoiding the problem of spatial loss caused by the direct regression of 2D keypoint coordinates to calculate 3D pose information. Experimental results show that on InterHand2.6M dataset， the two?hand Mean Per Joint Position Error （MPJPE）， the single?hand MPJPE， and the Mean Relative-Root Position Error （MRRPE） of the proposed algorithm reach 12.32， 9.96 and 29.57 mm， respectively； on RHD （Rendered Hand pose Dataset）， compared with InterNet and QMCG-Net algorithms， the proposed algorithm has the End-Point Error （EPE） reduced by 2.68 and 0.38 mm， respectively. The above results demonstrate that the proposed algorithm can estimate hand pose more accurately and is more robust in some two-hand interaction and occlusion scenarios.

Key words: hand pose estimation, multi-scale feature fusion, attention mechanism, High-Resolution Net (HRNet), HourGlass Network (HGNet)

摘要：

针对因遮挡和自相似性导致的从单张RGB图像估计三维手部姿态不精确的问题，提出结合注意力机制和多尺度特征融合的三维手部姿态估计算法。首先，提出结合扩张卷积和CBAM （Convolutional Block Attention Module）注意力机制的感受强化模块（SEM），以替换沙漏网络（HGNet）中的基本块（Basicblock），在扩大感受野的同时增强对空间信息的敏感性，从而提高手部特征的提取能力；其次，设计一种结合SPCNet （Spatial Preserve and Content-aware Network）和Soft-Attention改进的多尺度信息融合模块SS-MIFM （SPCNet and Soft-attention-Multi-scale Information Fusion Module），在充分考虑空间内容感知机制的情况下，有效地聚合多级特征，并显著提高二维手部关键点检测的准确性；最后，利用2.5D姿态转换模块将二维姿态转换为三维姿态，从而避免二维关键点坐标直接回归计算三维姿态信息导致的空间丢失问题。实验结果表明，在InterHand2.6M数据集上，所提算法的双手关节点平均误差（MPJPE）、单手MPJPE和根节点平均误差（MRRPE）分别达到了12.32、9.96和29.57 mm；在RHD（Rendered Hand pose Dataset）上，与InterNet和QMGR-Net算法相比，所提算法的终点误差（EPE）分别降低了2.68和0.38 mm。以上结果说明了所提算法能够更准确地估计手部姿态，且在一些双手交互和遮挡的场景下有更高的鲁棒性。

关键词: 手部姿态估计, 多尺度特征融合, 注意力机制, 高分辨率网络, 沙漏网络