关键词: 人物交互检测, 聚焦扩散特征增强, 语义特征, 交叉注意力, 类解耦特征增强

Abstract: Aiming at the problems existing in current human-object interaction detection methods, such as insufficient feature expression and utilization, and weak perception and discrimination ability of interaction instances with fewer training samples, this paper proposed a human-object interaction detection method based on class decoupling feature enhancement. To address the issue of insufficient feature expression, a Focus-Diffusion Feature Enhancement Network (FDFENet) was proposed, which adaptively enhances the extracted middle and high-level features to improve the expression ability of the features in the model. For the problem of low detection accuracy for difficult-to-classify interaction categories, a Feature Enhancement Algorithm based on Class Decoupling (FEACD) was proposed.  This algorithm first utilizes the vision-semantic fusion module to fully fuse visual features and semantic features. After decoupling the fused visual features and semantic features, three types of features, namely human, object and action, are obtained, and the similarity between them is calculated based on the corresponding visual and semantic features. Based on the obtained similarity, loss functions for the three categories were respectively designed, and feedback was added to each category during the training process.  In addition, focus loss was added to the loss function in this method. Focus loss helps the model pay more attention to difficult-to-classify samples and reduces the focus and enhancement on easy-to-classify samples. Experimental results show that the proposed method achieves the highest mean Average Precision (mAP) under the Scenario 1 setting of the standard human-object interaction detection dataset V-COCO, as well as under the Full (Default) setting of the HICO-DET dataset, demonstrating its effectiveness.

Key words: human-object interaction detection, focus diffusion feature enhancement, semantic feature, cross attention, class decoupling feature enhancement