Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (9): 2552-2559.DOI: 10.11772/j.issn.1001-9081.2020111743

Special Issue: 人工智能

• Artificial intelligence • Previous Articles     Next Articles

Micro-expression recognition algorithm based on convolutional block attention module and dual path networks

NIU Ruihua, YANG Jun, XING Lanxin, WU Renbiao   

  1. Tianjin Key Laboratory for Advanced Signal Processing(Civil Aviation University of China), Tianjin 300300, China
  • Received:2020-11-09 Revised:2021-01-05 Online:2021-09-10 Published:2021-09-15
  • Supported by:
    This work is partially supported by the Fundamental Research Funds for the Central Universities (3122019185).


牛瑞华, 杨俊, 邢斓馨, 吴仁彪   

  1. 天津市智能信号与图像处理重点实验室(中国民航大学), 天津 300300
  • 通讯作者: 吴仁彪
  • 作者简介:牛瑞华(1995-),女,河北衡水人,硕士研究生,主要研究方向:微表情识别、图像处理、深度学习;杨俊(1989-),男,浙江温州人,助理研究员,博士研究生,主要研究方向:图像处理、深度学习;邢斓馨(1998-),女,海南乐东人,硕士研究生,主要研究方向:微表情视频处理、神经网络;吴仁彪(1966-),男,湖北武汉人,教授,博士生导师,博士,主要研究方向:自适应信号处理,现代谱分析及其在雷达、卫星导航和空管中的应用。
  • 基金资助:

Abstract: Micro-expression is a facial movement that humans make when they are trying to hide their true emotions. It has the typical characteristics of short duration and small amplitude. Concerning the problems of the difficulty in recognition and the unsatisfactory recognition effect of micro-expression, a micro-expression recognition algorithm based on Convolutional Block Attention Module (CBAM) and Dual Path Networks (DPN), namely CBAM-DPN, was proposed. Firstly, data fusion of typical micro-expression datasets was performed. Then, the change values of pixels in the sequence frames were analyzed to determine the position of the apex frame, after that, image enhancement was performed to the apex frame. Finally, based on the CBAM-DPN network, the features of the enhanced micro-expression apex frame was effectively extracted, and a classifier was constructed to recognize the micro-expression. The Unweighted F1-score (UF1) and Unweighted Average Recall (UAR) of the model after optimization can reach 0.720 3 and 0.729 3 respectively, which are improved by 0.048 9 and 0.037 9 respectively compared with those of the DPN model, and are improved by 0.068 3 and 0.078 7 respectively compared with those of the CapsuleNet model. Experimental results show that the CBAM-DPN algorithm combined with the advantages of CBAM and DPN can enhance the information extraction ability of small features, and effectively improve the performance of micro-expression recognition.

Key words: micro-expression recognition, Dual Path Networks (DPN), Convolutional Block Attention Module (CBAM), apex frame, structure optimization

摘要: 微表情是一种人类在试图隐藏自己真实情感时作出的面部动作,具有持续时间短、幅度小的典型特点。针对微表情识别难度大、识别效果不理想的问题,提出一种基于卷积注意力模块(CBAM)和双通道网络(DPN)的微表情识别算法——CBAM-DPN。首先,进行典型微表情数据集的数据融合;然后,分析序列帧中像素的变化值以确定顶点帧位置,再对顶点帧进行图像增强处理;最后,基于CBAM-DPN对图像增强后的微表情顶点帧进行特征的有效提取,并构建分类器对微表情进行识别。优化后模型的未加权F1值(UF1)和未加权平均召回率(UAR)分别可以达到0.720 3和0.729 3,相较于DPN模型分别提高了0.048 9和0.037 9,相较于CapsuleNet模型分别提高了0.068 3和0.078 7。实验结果表明,CBAM-DPN算法融合了CBAM和DPN的共同优势,可增强微小特征的信息提取能力,有效改善微表情识别性能。

关键词: 微表情识别, 双通道网络, 卷积注意力模块, 顶点帧, 结构优化

CLC Number: