Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (5): 1282-1287.DOI: 10.11772/j.issn.1001-9081.2018102090

• Artificial intelligence • Previous Articles     Next Articles

Micro-expression recognition based on local region method

ZHANG Yanliang1, LU Bing1, HONG Xiaopeng2, ZHAO Guoying2, ZHANG Weitao3   

  1. 1. School of Physics and Electronic Information, Henan Polytechnic University, Jiaozuo Henan 454150, China;
    2. Center for Machine Vision and Signal Analysis, University of Oulu, Oulu FI-90014, Finland;
    3. School of Electronic Engineering, Xidian University, Xi'an Shaanxi 710071, China
  • Received:2018-10-16 Revised:2018-11-29 Online:2019-05-10 Published:2019-05-14
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61571339), the Open Project of State Key Laboratory of Network and Switching Technology (SKLNST-2016-1-02), the Doctoral Fund of Henan Polytechnic University (B2012-100).

基于局部区域方法的微表情识别

张延良1, 卢冰1, 洪晓鹏2, 赵国英2, 张伟涛3   

  1. 1. 河南理工大学 物理与电子信息学院, 河南 焦作 454150;
    2. 奥卢大学 机器视觉和信号分析研究中心, 芬兰 奥卢 FI-90014;
    3. 西安电子科技大学 电子工程学院, 西安 710071
  • 通讯作者: 张延良
  • 作者简介:张延良(1979-),男,河南平顶山人,副教授,博士,主要研究方向:人工智能、机器学习;卢冰(1992-),女,河南邓州人,硕士研究生,主要研究方向:智能信号处理、机器学习;洪晓鹏(1983-),男,广东揭阳人,讲师,博士生导师,博士,主要研究方向:计算机视觉、模式识别;赵国英(1977-),女,山东聊城人,教授,博士,主要研究方向:计算机视觉、情感计算;张伟涛(1983-),男,陕西西安人,副教授,博士,主要研究方向:盲信号处理、机械故障诊断。
  • 基金资助:
    国家自然科学基金资助项目(61571339);网络与交换技术国家重点实验室开放课题(SKLNST-2016-1-02);河南理工大学博士基金资助项目(B2012-100)。

Abstract: Micro-Expression (ME) occurrence is only related to local region of face, with very short time and subtle movement intensity. There are also some unrelated muscle movements in the face during the occurrence of micro-expressions. By using existing global method of micro-expression recognition, the spatio-temporal patterns of these unrelated changes were extracted, thereby reducing the representation capability of feature vectors, and thus affecting the recognition performance. To solve this problem, the local region method was proposed to recognize micro-expression. Firstly, according to the region with the Action Units (AU) related to the micro-expression, seven local regions related to the micro-expression were partitioned by facial key coordinates. Then, the spatio-temporal patterns of these local regions were extracted and connected in series to form feature vectors for micro-expression recognition. The experimental results of leave-one-subject-out cross validation show that the micro-expression recognition accuracy of local region method is 9.878% higher than that of global region method. The analysis of the confusion matrix of each region's recognition result shows that the proposed method makes full use of the structural information of each local region of face, effectively eliminating the influence of unrelated regions of the micro-expression on the recognition performance, and its performance of micro-expression recognition can be significantly improved compared with the global region method.

Key words: micro-expression recognition, feature vector, Action Unit (AU), global region method, local region method

摘要: 微表情(ME)的发生只牵涉到面部局部区域,具有动作幅度小、持续时间短的特点,但面部在产生微表情的同时也存在一些无关的肌肉动作。现有微表情识别的全局区域方法会提取这些无关变化的时空模式,从而降低特征向量对于微表情的表达能力,进而影响识别效果。针对这个问题,提出使用局部区域方法进行微表情识别。首先,根据微表情发生时所牵涉到的动作单元(AU)所在区域,通过面部关键点坐标将与微表情相关的七个局部区域划分出来;然后,提取这些局部区域组合的时空模式并串联构成特征向量,进行微表情识别。留一交叉验证的实验结果表明局部区域方法较全局区域方法进行微表情识别的识别率平均提高9.878%。而通过对各区域识别结果的混淆矩阵进行分析表明所提方法充分利用了面部各局部区域的结构信息,并有效摒除与微表情无关区域对识别性能的影响,较全局区域方法可以显著提高微表情识别的性能。

关键词: 微表情识别, 特征向量, 动作单元, 全局区域方法, 局部区域方法

CLC Number: