基于局部区域方法的微表情识别

doi:10.11772/j.issn.1001-9081.2018102090

计算机应用 ›› 2019, Vol. 39 ›› Issue (5): 1282-1287.DOI: 10.11772/j.issn.1001-9081.2018102090

基于局部区域方法的微表情识别

张延良¹, 卢冰¹, 洪晓鹏², 赵国英², 张伟涛³

1. 河南理工大学物理与电子信息学院, 河南焦作 454150;
2. 奥卢大学机器视觉和信号分析研究中心, 芬兰奥卢 FI-90014;
3. 西安电子科技大学电子工程学院, 西安 710071

收稿日期:2018-10-16 修回日期:2018-11-29 出版日期:2019-05-10 发布日期:2019-05-14
通讯作者: 张延良
作者简介:张延良(1979-),男,河南平顶山人,副教授,博士,主要研究方向:人工智能、机器学习;卢冰(1992-),女,河南邓州人,硕士研究生,主要研究方向:智能信号处理、机器学习;洪晓鹏(1983-),男,广东揭阳人,讲师,博士生导师,博士,主要研究方向:计算机视觉、模式识别;赵国英(1977-),女,山东聊城人,教授,博士,主要研究方向:计算机视觉、情感计算;张伟涛(1983-),男,陕西西安人,副教授,博士,主要研究方向:盲信号处理、机械故障诊断。
基金资助:
国家自然科学基金资助项目（61571339）；网络与交换技术国家重点实验室开放课题（SKLNST-2016-1-02）；河南理工大学博士基金资助项目（B2012-100）。

Micro-expression recognition based on local region method

ZHANG Yanliang¹, LU Bing¹, HONG Xiaopeng², ZHAO Guoying², ZHANG Weitao³

1. School of Physics and Electronic Information, Henan Polytechnic University, Jiaozuo Henan 454150, China;
2. Center for Machine Vision and Signal Analysis, University of Oulu, Oulu FI-90014, Finland;
3. School of Electronic Engineering, Xidian University, Xi'an Shaanxi 710071, China

Received:2018-10-16 Revised:2018-11-29 Online:2019-05-10 Published:2019-05-14
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61571339), the Open Project of State Key Laboratory of Network and Switching Technology (SKLNST-2016-1-02), the Doctoral Fund of Henan Polytechnic University (B2012-100).

摘要/Abstract

摘要： 微表情（ME）的发生只牵涉到面部局部区域，具有动作幅度小、持续时间短的特点，但面部在产生微表情的同时也存在一些无关的肌肉动作。现有微表情识别的全局区域方法会提取这些无关变化的时空模式，从而降低特征向量对于微表情的表达能力，进而影响识别效果。针对这个问题，提出使用局部区域方法进行微表情识别。首先，根据微表情发生时所牵涉到的动作单元（AU）所在区域，通过面部关键点坐标将与微表情相关的七个局部区域划分出来；然后，提取这些局部区域组合的时空模式并串联构成特征向量，进行微表情识别。留一交叉验证的实验结果表明局部区域方法较全局区域方法进行微表情识别的识别率平均提高9.878%。而通过对各区域识别结果的混淆矩阵进行分析表明所提方法充分利用了面部各局部区域的结构信息，并有效摒除与微表情无关区域对识别性能的影响，较全局区域方法可以显著提高微表情识别的性能。

关键词: 微表情识别, 特征向量, 动作单元, 全局区域方法, 局部区域方法

Abstract: Micro-Expression (ME) occurrence is only related to local region of face, with very short time and subtle movement intensity. There are also some unrelated muscle movements in the face during the occurrence of micro-expressions. By using existing global method of micro-expression recognition, the spatio-temporal patterns of these unrelated changes were extracted, thereby reducing the representation capability of feature vectors, and thus affecting the recognition performance. To solve this problem, the local region method was proposed to recognize micro-expression. Firstly, according to the region with the Action Units (AU) related to the micro-expression, seven local regions related to the micro-expression were partitioned by facial key coordinates. Then, the spatio-temporal patterns of these local regions were extracted and connected in series to form feature vectors for micro-expression recognition. The experimental results of leave-one-subject-out cross validation show that the micro-expression recognition accuracy of local region method is 9.878% higher than that of global region method. The analysis of the confusion matrix of each region's recognition result shows that the proposed method makes full use of the structural information of each local region of face, effectively eliminating the influence of unrelated regions of the micro-expression on the recognition performance, and its performance of micro-expression recognition can be significantly improved compared with the global region method.

Key words: micro-expression recognition, feature vector, Action Unit (AU), global region method, local region method

中图分类号:

TP391.41

张延良, 卢冰, 洪晓鹏, 赵国英, 张伟涛. 基于局部区域方法的微表情识别[J]. 计算机应用, 2019, 39(5): 1282-1287.

ZHANG Yanliang, LU Bing, HONG Xiaopeng, ZHAO Guoying, ZHANG Weitao. Micro-expression recognition based on local region method[J]. Journal of Computer Applications, 2019, 39(5): 1282-1287.

参考文献

[1] CLANCY M. The Philosophy of Deception[M]. Oxford: Oxford University Press, 2009: 118-133.
[2] SALTER F, GRAMMER K, RIKOWSKI A. Sex differences in negotiating with powerful males[J]. Human Nature, 2005, 16(3):306-321.
[3] WHITEHILL J, SERPELL Z, LIN Y C, et al. The faces of engagement: automatic recognition of student engagementfrom facial expressions[J]. IEEE Transactions on Affective Computing, 2014, 5(1):86-98.
[4] POOL L D, QUALTER P. Improving emotional intelligence and emotional self-efficacy through a teaching intervention for university students[J]. Learning & Individual Differences, 2012, 22(3):306-312.
[5] STEWART P A, WALLER B M, SCHUBERT J N. Presidential speechmaking style: emotional response to micro-expressions of facial affect[J]. Motivation & Emotion, 2009, 33(2):125-135.
[6] EKMAN P. Micro Expression Training Tool (METT)[M]. San Francisco: University of California, 2002: 1877-1903.
[7] FRANK M G, HERBASZ M, SINUK K, et al. I see how you feel: training laypeople and professionals to recognize fleeting emotions[C]// Proceedings of the 2009 Annual Meeting of the International Communication Association. New York:[s. n.], 2009: 3515-3522.
[8] OJALA T, PIETIKAINEN M, HARWOOD D. A comparative study of texture measures with classification based on featured distributions[J]. Pattern Recognition, 1996, 29(1): 51-59.
[9] ZHAO G, PIETIKAINEN M. Dynamic texture recognition using local binary patterns with an application to facial expressions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6): 915-928.
[10] HUANG X, ZHAO G, HONG X, et al. Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns[J]. Neurocomputing, 2016, 175: 564-578.
[11] WANG Y, SEE J, PHAN R C, et al. LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition[C]// Proceedings of the 12th Asian Conference on Computer Vision. Berlin: Springer, 2014: 525-537.
[12] FU X, WEI W. Centralized binary patterns embedded with image Euclidean distance for facial expression recognition[C]// Proceedings of the 20084th International Conference on Natural Computation. Washington, DC: IEEE Computer Society, 2008:115-119.
[13] SUN B, LI L, WU X, et al. Combining feature-level and decision-level fusion in a hierarchical classifier for emotion recognition in the wild[J]. Journal on Multimodal User Interfaces, 2016, 10(2):125-137.
[14] LI X, HONG X, MOILANEN A, et al. Towards reading hidden emotions: a comparative study of spontaneous micro-expression spotting and recognition methods[J]. IEEE Transactions on Affective Computing, 2018, 9(4): 563-577.
[15] COHN J F, AMBADAR Z, EKMAN P. Observer-based measurement of facial expression with the facial action coding system[J]. Neuroscience Letters, 2007, 394(3): 203-221.
[16] MARTINEZ B, VALSTAR M F, JIANG B, et al. Automatic analysis of facial actions: a survey[J/OL]. IEEE Transactions on Affective Computing, 2017[2018-06-20].https://ieeexplore.ieee.org/document/7990582.
[17] YAN W J, LI X, WANG S J, et al. CASME Ⅱ: an improved spontaneous micro-expression database and the baseline evaluation[J]. PLoS One, 2014, 9(1): 1-8.
[18] ASTHANA A, ZAFEIRIOU S, CHENG S, et al. Incremental face alignment in the wild[C]// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2014:1859-1866.
[19] COOTES T F, EDWARDS G J, TAYLOR C J. Active appearance models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6): 681-685.
[20] 刘丽,谢毓湘,魏迎梅,等.局部二进制模式方法综述[J]. 中国图象图形学报,2014,19(12): 1696-1720.(LIU L,XIE Y X,WEI Y M,et al. Survey of local binary pattern method[J].Journal of Image and Graphics, 2014, 19(12): 1696-1720.)
[21] OH Y, NGO A C, SEE J, et al. Monogenic Riesz wavelet representation for micro-expression recognition[C]// Proceedings of the 2015 IEEE International Conference on Digital Signal Processing. Piscataway, NJ: IEEE, 2015: 1237-1241.
[22] LIONG S, SEE J, PHAN R C, et al. Subtle expression recognition using optical strain weighted features[C]// Proceedings of the 12th Asian Conference on Computer Vision. Berlin: Springer, 2014: 644-657.
[23] NGO A C, PHAN R C, SEE J, et al. Spontaneous subtle expression recognition: imbalanced databases and solutions[C]// Proceedings of the 12th Asian Conference on Computer Vision. Berlin: Springer, 2014: 33-48.

基于局部区域方法的微表情识别

Micro-expression recognition based on local region method

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	牛瑞华, 杨俊, 邢斓馨, 吴仁彪. 基于卷积注意力模块和双通道网络的微表情识别算法[J]. 计算机应用, 2021, 41(9): 2552-2559.
[2]	魏哲, 王小华. 淋巴结转移检测的八度卷积方法[J]. 计算机应用, 2020, 40(3): 723-727.
[3]	崔艺馨, 陈晓东. Spark框架优化的大规模谱聚类并行算法[J]. 计算机应用, 2020, 40(1): 168-172.
[4]	田保军, 杨浒昀, 房建东. 融合信任和基于概率矩阵分解的推荐算法[J]. 计算机应用, 2019, 39(10): 2834-2840.
[5]	肖春景, 夏克文, 乔永卫, 张宇翔. 基于动态聚类的旅游线路推荐[J]. 计算机应用, 2017, 37(8): 2395-2400.
[6]	戴晓明, 张薇, 郑志恒, 李镇林. 基于容错学习的GSW-型全同态层次型IBE方案[J]. 计算机应用, 2016, 36(7): 1856-1860.
[7]	郭倩, 杨红菊, 梁新彦. 基于新的空间关系特征的图像检索方法[J]. 计算机应用, 2016, 36(7): 1918-1922.
[8]	张嫱嫱, 黄廷磊, 张银明. 基于聚类分析的二分网络社区挖掘[J]. 计算机应用, 2015, 35(12): 3511-3514.
[9]	王斌斌张妍琰张学林. 基于区域的无线传感器网络混合密钥管理方案[J]. 计算机应用, 2014, 34(1): 90-94.
[10]	杨彤瑶王彬李川何弼熊新. 基于改进主元分析方法的隧道应变实时监测预警系统[J]. 计算机应用, 2013, 33(11): 3284-3287.
[11]	张志杰. 基于测距特征向量匹配和模糊控制的室内清洁机器人路径规划[J]. 计算机应用, 2012, 32(09): 2527-2529.
[12]	王铮李兴民. 基于四元数和SOM神经网络的彩色图像边缘检测[J]. 计算机应用, 2012, 32(02): 510-513.
[13]	杨文柱卢素魁王思乐. 基于多类支持向量机的棉花异性纤维分类方法[J]. 计算机应用, 2011, 31(12): 3446-3448.
[14]	郑美珠赵景秀. 基于区域一致性测度的彩色图像边缘检测[J]. 计算机应用, 2011, 31(09): 2485-2488.
[15]	刘丛唐坚刚张丽红. 基于智能选择多维特征的肺部CT图像检索[J]. 计算机应用, 2010, 30(07): 1867-1869.