Motor imagery electroencephalogram signal recognition method based on convolutional neural network in time-frequency domain

doi:10.11772/j.issn.1001-9081.2018122553

Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (8): 2480-2483.DOI: 10.11772/j.issn.1001-9081.2018122553

• Frontier & interdisciplinary applications • Previous Articles Next Articles

Motor imagery electroencephalogram signal recognition method based on convolutional neural network in time-frequency domain

HU Zhangfang, ZHANG Li, HUANG Lijia, LUO Yuan

National Engineering Research and Development Center for Information Accessibility, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Received:2018-12-27 Revised:2019-03-04 Online:2019-08-10 Published:2019-04-08
Supported by:
This work is partially supported by the National Natural Science Fundation of China (61703067).

基于时频域的卷积神经网络运动想象脑电信号识别方法

胡章芳, 张力, 黄丽嘉, 罗元

重庆邮电大学国家信息无障碍工程研发中心, 重庆 400065

通讯作者: 张力
作者简介:胡章芳(1969-),女,重庆人,教授,硕士,主要研究方向:智能信号处理、数字图像处理;张力(1994-),男,湖北孝感人,硕士研究生,主要研究方向:脑机接口、人机交互;黄丽嘉(1999-),女,重庆人,主要研究方向:脑机接口、人机交互;罗元(1972-),女,湖北宜昌人,教授,博士,主要研究方向:数字图像处理、智能信号处理。
基金资助:
国家自然科学基金资助项目（61703067）。

Abstract

Abstract: To solve the problem of low recognition rate of motor imagery ElectroEncephaloGram (EEG) signals, considering that EEG signals contain abundant time-frequency information, a recognition method based on Convolutional Neural Network (CNN) in time-frequency domain was proposed. Firstly, Short-Time Fourier Transform (STFT) was applied to preprocess the relevant frequency bands of EEG signals to construct a two-dimensional time-frequency domain map composed of multiple time-frequency maps of electrodes, which was regarded as the input of the CNN. Secondly, focusing on the time-frequency characteristic of two-dimensional time-frequency domain map, a novel CNN structure was designed by one-dimensional convolution method. Finally, the features extracted by CNN were classified by Support Vector Machine (SVM). Experimental results based on BCI dataset show that the average recognition rate of the proposed method is 86.5%, which is higher than that of traditional motor imagery EEG signal recognition method, and the proposed method has been applied to the intelligent wheelchair, which proves its effectiveness.

Key words: motor imagery, ElectroEncephaloGram (EEG), time-frequency domain, Convolutional Neural Network (CNN), intelligent wheelchair

摘要： 针对目前运动想象脑电（EEG）信号识别率较低的问题，考虑到脑电信号蕴含着丰富的时频信息，提出一种基于时频域的卷积神经网络（CNN）运动想象脑电信号识别方法。首先，利用短时傅里叶变换（STFT）对脑电信号的相关频带进行预处理，并将多个电极的时频图组合构造出一种二维时频图；然后，针对二维时频图的时频特性，通过一维卷积的方法设计了一种新颖的CNN结构；最后，通过支持向量机（SVM）对CNN提取的特征进行分类。基于BCI数据集的实验结果表明，所提方法的平均识别率为86.5%，优于其他传统运动想象脑电信号识别方法；同时将该方法应用在智能轮椅上，验证了其有效性。

关键词: 运动想象, 脑电, 时频域, 卷积神经网络, 智能轮椅

CLC Number:

TP242.6

HU Zhangfang, ZHANG Li, HUANG Lijia, LUO Yuan. Motor imagery electroencephalogram signal recognition method based on convolutional neural network in time-frequency domain[J]. Journal of Computer Applications, 2019, 39(8): 2480-2483.

胡章芳, 张力, 黄丽嘉, 罗元. 基于时频域的卷积神经网络运动想象脑电信号识别方法[J]. 计算机应用, 2019, 39(8): 2480-2483.

References

[1] 王行愚,金晶,张宇,等.脑控:基于脑——机接口的人机融合控制[J].自动化学报,2013,39(3):208-221. (WANG X Y, JIN J, ZHANG Y, et al. Brain control:human-computer integration control based on brain-computer interface[J]. Acta Automatica Sinica, 2013, 39(3):208-221.)
[2] MINGUILLON J, LOPEZ-GORDO M A, PELAYO F. Trends in EEG-BCI for daily-life:requirements for artifact removal[J]. Biomedical Signal Processing and Control, 2017, 31:407-418.
[3] AVILÉS-CRUZ C, VILLEGAS-CORTEZ J, FERREYRA-RAMíREZ A, et al. EEG pattern recognition:an efficient improvement combination of ERD/ERS/laterality features to create a self-paced BCI system[C]//Proceedings of the 2016 Mexican Conference on Pattern Recognition, LNCS 9703. Cham:Springer, 2016:231-240.
[4] RAMOSER H, MVLLER-GERKING J, PFURTSCHELLER G. Optimal spatial filtering of single trial EEG during imagined hand movement[J]. IEEE Transactions on Rehabilitation Engineering, 2000, 8(4):441-446.
[5] KAI K A, ZHENG Y C, ZHANG H, et al. Filter Bank Common Spatial Pattern (FBCSP) in brain-computer interface[C]//Proceedings of the 2008 IEEE International Joint Conference on Neural Networks. Piscataway, NJ:IEEE, 2008:2390-2397.
[6] IQBAL H, AQIL M. A QR decomposition based RLS algorithm with forgetting factor for adaptation of AR EEG features[C]//Proceedings of the 2016 International Conference on Emerging Technologies. Piscataway, NJ:IEEE, 2016:1-5.
[7] 唐智川,张克俊,李超,等.基于深度卷积神经网络的运动想象分类及其在脑控外骨骼中的应用[J].计算机学报,2017,40(6):1367-1378. (TANG Z C, ZHANG K J, LI C, et al. Motor imagery classification based on deep convolutional neural network and its application in exoskeleton controlled by EEG[J]. Chinese Journal of Computers, 2017, 40(6):1367-1378.)
[8] PÉREZ-ZAPATA A F, CARDONA-ESCOBAR A F, JARAMILLO-GARZÓN J A, et al. Deep convolutional neural networks and power spectral density features for motor imagery classification of EEG signals[C]//Proceedings of the 2018 International Conference on Augmented Cognition. Berlin:Springer, 2018:158-169.
[9] GRAMFORT A, STROHMEIER D, HAUEISEN J, et al. Time-frequency mixed-norm estimates:sparse M/EEG imaging with non-stationary source activations[J]. Neuroimage, 2013, 70(2):410-422.
[10] SHAO S Y, SHEN K Q, ONG C J, et al. Automatic identification and removal of artifacts in EEG using a probabilistic multi-class SVM approach with error correction[C]//Proceedings of the 2008 IEEE Internatioanl Conference on Systems, Man and Cybernetics. Piscataway, NJ:IEEE, 2008:1134-1139.
[11] BHARDWAJ A, GUPTA A, JAIN P, et al. Classification of human emotions from EEG signals using SVM and LDA classifiers[C]//Proceedings of the 2015 2nd International Conference on Signal Processing and Integrated Networks. Piscataway, NJ:IEEE, 2015:180-185.
[12] NG S C, LEUNG S H, LUK A. Fast convergent generalized back-propagation algorithm with constant learning rate[J]. Neural Processing Letters, 1999, 9(1):13-23.
[13] KENG A K, YANG C Z, WANG C, et al. Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b[J]. Frontiers in Neuroscience, 2012, 6:39.
[14] NEU D, MAIRESSE O, VERBANCK P, et al. Non-REM sleep EEG power distribution in fatigue and sleepiness[J]. Journal of Psychosomatic Research, 2014, 76(4):286-291.

[1]	SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion [J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.
[2]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[3]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[4]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[5]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[6]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[7]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[8]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[9]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[10]	ZENG Xiangyin, ZHENG Bochuan, LIU Dan. Detection of left and right railway tracks based on deep convolutional neural network and clustering [J]. Journal of Computer Applications, 2021, 41(8): 2324-2329.
[11]	TAN Daoqiang, ZENG Cheng, QIAO Jinxia, ZHANG Jun. Shadow detection method based on hybrid attention model [J]. Journal of Computer Applications, 2021, 41(7): 2076-2081.
[12]	YANG Su, OUYANG Zhi, DU Nisuo. Unsupervised parallel hash image retrieval based on correlation distance [J]. Journal of Computer Applications, 2021, 41(7): 1902-1907.
[13]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[14]	WU Guangli, LI Leiting, GUO Zhenzhou, WANG Chengxiang. Video summarization generation model based on improved bi-directional long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1908-1914.
[15]	ZHANG Hui, ZHANG Nana, HUANG Jun. Multi-angle head pose estimation method based on optimized LeNet-5 network [J]. Journal of Computer Applications, 2021, 41(6): 1667-1672.

Motor imagery electroencephalogram signal recognition method based on convolutional neural network in time-frequency domain

基于时频域的卷积神经网络运动想象脑电信号识别方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics