Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (11): 3625-3632.DOI: 10.11772/j.issn.1001-9081.2021091701
• Artificial intelligence • Previous Articles
Yu PENG, Yaolian SONG(), Jun YANG
Received:
2021-09-30
Revised:
2022-01-05
Accepted:
2022-01-28
Online:
2022-11-14
Published:
2022-11-10
Contact:
Yaolian SONG
About author:
PENG Yu, born in 1995, M. S. candidate. His research interests include brain information decoding, deep learning.通讯作者:
宋耀莲
作者简介:
彭禹(1995—),男,四川泸州人,硕士研究生,CCF会员,主要研究方向:脑信息解码、深度学习CLC Number:
Yu PENG, Yaolian SONG, Jun YANG. Motor imagery electroencephalography classification based on data augmentation[J]. Journal of Computer Applications, 2022, 42(11): 3625-3632.
彭禹, 宋耀莲, 杨俊. 基于数据增强的运动想象脑电分类[J]. 《计算机应用》唯一官方网站, 2022, 42(11): 3625-3632.
Add to citation manager EndNote|Ris|BibTeX
URL: http://www.joca.cn/EN/10.11772/j.issn.1001-9081.2021091701
受试者 | FBCSP | MSFBCNN[ | EEGNet[ | 3DCNN[ | L‑Net | LH‑Net | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | |
均值 | 66.18 | 54.88 | 74.91 | 66.50 | 73.42 | 64.53 | 76.22 | 68.30 | 74.22 | 65.62 | 77.83 | 70.44 |
A01 | 73.44 | 64.57 | 81.60 | 75.41 | 83.33 | 77.73 | 81.22 | 74.96 | 81.14 | 74.86 | 79.36 | 72.48 |
A02 | 56.08 | 41.41 | 64.15 | 52.20 | 63.80 | 51.68 | 67.95 | 57.27 | 64.66 | 52.89 | 64.31 | 52.46 |
A03 | 80.42 | 73.88 | 86.98 | 82.60 | 88.76 | 84.98 | 84.09 | 78.78 | 84.98 | 79.96 | 91.57 | 88.76 |
A04 | 57.68 | 43.59 | 68.14 | 57.42 | 62.41 | 49.92 | 65.69 | 54.25 | 71.05 | 61.31 | 70.18 | 60.19 |
A05 | 57.38 | 43.13 | 71.27 | 61.58 | 58.72 | 44.97 | 70.58 | 60.78 | 71.37 | 61.87 | 74.63 | 66.18 |
A06 | 49.48 | 32.63 | 63.37 | 51.11 | 58.51 | 44.62 | 67.57 | 56.76 | 56.28 | 41.63 | 66.97 | 55.97 |
A07 | 81.25 | 74.97 | 90.54 | 87.36 | 84.81 | 79.72 | 85.87 | 81.16 | 83.03 | 77.42 | 85.56 | 80.78 |
A08 | 73.52 | 64.63 | 77.87 | 70.47 | 82.12 | 76.08 | 84.94 | 79.91 | 77.86 | 70.48 | 83.02 | 77.36 |
A09 | 66.36 | 55.12 | 70.31 | 60.35 | 78.30 | 71.06 | 78.10 | 70.80 | 77.65 | 70.20 | 84.84 | 79.78 |
Tab. 1 Comparison of classification performance of each method on test set
受试者 | FBCSP | MSFBCNN[ | EEGNet[ | 3DCNN[ | L‑Net | LH‑Net | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | Accuracy | Kappa | |
均值 | 66.18 | 54.88 | 74.91 | 66.50 | 73.42 | 64.53 | 76.22 | 68.30 | 74.22 | 65.62 | 77.83 | 70.44 |
A01 | 73.44 | 64.57 | 81.60 | 75.41 | 83.33 | 77.73 | 81.22 | 74.96 | 81.14 | 74.86 | 79.36 | 72.48 |
A02 | 56.08 | 41.41 | 64.15 | 52.20 | 63.80 | 51.68 | 67.95 | 57.27 | 64.66 | 52.89 | 64.31 | 52.46 |
A03 | 80.42 | 73.88 | 86.98 | 82.60 | 88.76 | 84.98 | 84.09 | 78.78 | 84.98 | 79.96 | 91.57 | 88.76 |
A04 | 57.68 | 43.59 | 68.14 | 57.42 | 62.41 | 49.92 | 65.69 | 54.25 | 71.05 | 61.31 | 70.18 | 60.19 |
A05 | 57.38 | 43.13 | 71.27 | 61.58 | 58.72 | 44.97 | 70.58 | 60.78 | 71.37 | 61.87 | 74.63 | 66.18 |
A06 | 49.48 | 32.63 | 63.37 | 51.11 | 58.51 | 44.62 | 67.57 | 56.76 | 56.28 | 41.63 | 66.97 | 55.97 |
A07 | 81.25 | 74.97 | 90.54 | 87.36 | 84.81 | 79.72 | 85.87 | 81.16 | 83.03 | 77.42 | 85.56 | 80.78 |
A08 | 73.52 | 64.63 | 77.87 | 70.47 | 82.12 | 76.08 | 84.94 | 79.91 | 77.86 | 70.48 | 83.02 | 77.36 |
A09 | 66.36 | 55.12 | 70.31 | 60.35 | 78.30 | 71.06 | 78.10 | 70.80 | 77.65 | 70.20 | 84.84 | 79.78 |
受试者 | L‑Net | L‑Net(NA) | L‑Net+(STFT+NA) | LH‑Net | LH‑Net(NA) | LH‑Net+(STFT+NA) | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | |
均值 | 74.22 | 65.62 | 75.14 | 66.97 | 79.12 | 72.17 | 77.83 | 70.44 | 79.28 | 72.39 | 82.67 | 76.90 |
A01 | 81.14 | 74.86 | 81.85 | 75.81 | 82.56 | 76.73 | 79.36 | 72.48 | 80.07 | 73.43 | 85.05 | 80.06 |
A02 | 64.66 | 52.89 | 67.49 | 56.69 | 67.84 | 57.16 | 64.31 | 52.46 | 70.32 | 60.46 | 72.79 | 63.74 |
A03 | 84.98 | 79.96 | 86.45 | 81.92 | 94.14 | 92.19 | 91.57 | 88.76 | 92.31 | 89.74 | 95.24 | 93.65 |
A04 | 71.05 | 61.31 | 69.30 | 60.02 | 74.56 | 66.05 | 70.18 | 60.19 | 72.36 | 63.11 | 75.88 | 67.83 |
A05 | 71.37 | 61.87 | 71.38 | 61.92 | 77.54 | 70.04 | 74.63 | 66.18 | 76.09 | 68.13 | 80.07 | 73.43 |
A06 | 56.28 | 41.63 | 57.21 | 42.95 | 63.26 | 51.06 | 66.97 | 55.97 | 69.30 | 59.08 | 71.16 | 61.56 |
A07 | 83.03 | 77.42 | 84.84 | 79.82 | 86.64 | 82.20 | 85.56 | 80.78 | 87.36 | 83.17 | 91.33 | 88.44 |
A08 | 77.86 | 70.48 | 80.44 | 73.92 | 83.76 | 78.35 | 83.02 | 77.36 | 81.24 | 75.10 | 86.35 | 81.80 |
A09 | 77.65 | 70.20 | 77.27 | 69.66 | 81.82 | 75.72 | 84.84 | 79.78 | 84.47 | 79.30 | 86.20 | 81.59 |
Tab. 2 Comparison of classification performance of L?Net and LH?Net with different data augmentation
受试者 | L‑Net | L‑Net(NA) | L‑Net+(STFT+NA) | LH‑Net | LH‑Net(NA) | LH‑Net+(STFT+NA) | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | Acc | Kappa | |
均值 | 74.22 | 65.62 | 75.14 | 66.97 | 79.12 | 72.17 | 77.83 | 70.44 | 79.28 | 72.39 | 82.67 | 76.90 |
A01 | 81.14 | 74.86 | 81.85 | 75.81 | 82.56 | 76.73 | 79.36 | 72.48 | 80.07 | 73.43 | 85.05 | 80.06 |
A02 | 64.66 | 52.89 | 67.49 | 56.69 | 67.84 | 57.16 | 64.31 | 52.46 | 70.32 | 60.46 | 72.79 | 63.74 |
A03 | 84.98 | 79.96 | 86.45 | 81.92 | 94.14 | 92.19 | 91.57 | 88.76 | 92.31 | 89.74 | 95.24 | 93.65 |
A04 | 71.05 | 61.31 | 69.30 | 60.02 | 74.56 | 66.05 | 70.18 | 60.19 | 72.36 | 63.11 | 75.88 | 67.83 |
A05 | 71.37 | 61.87 | 71.38 | 61.92 | 77.54 | 70.04 | 74.63 | 66.18 | 76.09 | 68.13 | 80.07 | 73.43 |
A06 | 56.28 | 41.63 | 57.21 | 42.95 | 63.26 | 51.06 | 66.97 | 55.97 | 69.30 | 59.08 | 71.16 | 61.56 |
A07 | 83.03 | 77.42 | 84.84 | 79.82 | 86.64 | 82.20 | 85.56 | 80.78 | 87.36 | 83.17 | 91.33 | 88.44 |
A08 | 77.86 | 70.48 | 80.44 | 73.92 | 83.76 | 78.35 | 83.02 | 77.36 | 81.24 | 75.10 | 86.35 | 81.80 |
A09 | 77.65 | 70.20 | 77.27 | 69.66 | 81.82 | 75.72 | 84.84 | 79.78 | 84.47 | 79.30 | 86.20 | 81.59 |
模型 | 总时间 | 均值 |
---|---|---|
L‑Net | 783 | 87 |
LH‑Net | 944 | 105 |
L‑Net+DA | 1 498 | 166 |
LH‑Net+DA | 1 942 | 216 |
Tab. 3 Comparison of training time of different models
模型 | 总时间 | 均值 |
---|---|---|
L‑Net | 783 | 87 |
LH‑Net | 944 | 105 |
L‑Net+DA | 1 498 | 166 |
LH‑Net+DA | 1 942 | 216 |
1 | LONG J, LI Y, WANG H, et al. A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2012, 20(5): 720-729. 10.1109/tnsre.2012.2197221 |
2 | ANG K K, CHIN Z Y, WANG C, et al. Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b[J]. Frontiers in Neuroscience, 2012, 6: Article 39. 10.3389/fnins.2012.00039 |
3 | TONIN L, CARLSON T, LEEB R, et al. Brain‑controlled telepresence robot by motor‑disabled people[C]// Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Piscataway: IEEE, 2011: 4227-4230. 10.1109/iembs.2011.6091049 |
4 | ACHARYA U R, SREEr S V, SWAPNA G, et al. Automated EEG analysis of epilepsy: a review[J]. Knowledge‑Based Systems, 2013, 45: 147-165. 10.1016/j.knosys.2013.02.014 |
5 | HOSSAIN M S, AMIN S U, ALSULAIMAN M, et al. Applying deep learning for epilepsy seizure detection and brain mapping visualization[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2019, 15(1): 1-17. 10.1145/3241056 |
6 | SAFI M S, SAFI S M M. Early detection of Alzheimer’s disease from EEG signals using Hjorth parameters[J]. Biomedical Signal Processing and Control, 2021, 65: 102338. 10.1016/j.bspc.2020.102338 |
7 | ZHANG R, ZONG Q, DOU L, et al. Hybrid deep neural network using transfer learning for EEG motor imagery decoding[J]. Biomedical Signal Processing and Control, 2021, 63: 102144. 10.1016/j.bspc.2020.102144 |
8 | CHIARELLI A M, CROCE P, MERLA A, et al. Deep learning for hybrid EEG‑fNIRS brain‑computer interface: application to motor imagery classification[J]. Journal of Neural Engineering, 2018, 15(3): 036028. 10.1088/1741-2552/aaaf82 |
9 | LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional neural network for EEG‑based brain–computer interfaces[J]. Journal of Neural Engineering, 2018, 15(5): 056013. 10.1088/1741-2552/aace8c |
10 | SAKHAVI S, GUAN C, YAN S. Learning temporal information for brain‑computer interface using convolutional neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(11): 5619-5629. 10.1109/tnnls.2018.2789927 |
11 | AMIN S U, ALSULAIMAN M, MUHAMMAD G, et al. Deep learning for EEG motor imagery classification based on multi‑layer CNNs feature fusion[J]. Future Generation Computer Systems, 2019, 101: 542-554. 10.1016/j.future.2019.06.027 |
12 | 杜秀丽,马振倩,邱少明,等.基于卷积注意力机制的运动想象脑电信号识别[J].计算机工程与应用,2021,57(18):181-185. 10.3778/j.issn.1002-8331.2006-0120 |
DU X L, MA Z Q, QIU S M, et al. Motor imagery EEG signal recognition based on the convolutional attention mechanism[J]. Journal of Computer Engineering and Applications, 2021, 57(18): 181-185. 10.3778/j.issn.1002-8331.2006-0120 | |
13 | SCHIRRMEISTER R T, SPRINGENBERG J T, FIEDERER L D J, et al. Deep learning with convolutional neural networks for EEG decoding and visualization[J]. Human Brain Mapping, 2017, 38(11): 5391-5420. 10.1002/hbm.23730 |
14 | WANG F, ZHONG S‑H, PENG J, et al. Data augmentation for EEG‑based emotion recognition with deep convolutional neural networks[C]// Proceedings of the 2018 International Conference on Multimedia Modeling, LNCS 10705. Cham: Springer, 2018: 82-93. |
15 | SCHWABEDAL J T C, SNYDER J C, CAKMAK A, et al. Addressing class imbalance in classification problems of noisy signals by using fourier transform surrogates[EB/OL]. (2019-01-28) [2021-06-19]. . 10.48550/arXiv.1806.08675 |
16 | ZHANG K, XU G, HAN Z, et al. Data augmentation for motor imagery signal classification based on a hybrid neural network[J]. Sensors, 2020, 20(16): 4485. 10.3390/s20164485 |
17 | HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co‑adaptation of feature detectors[EB/OL]. [2021-09-01]. . |
18 | CHAUDHARY S, TARAN S, BAJAJ V, et al. Convolutional neural network based approach towards motor imagery tasks EEG signals classification[J]. IEEE Sensors Journal, 2019, 19(12): 4494-4500. 10.1109/jsen.2019.2899645 |
19 | BAI S, KOLTER J Z, KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[EB/OL]. (2018-04-19) [2021-06-19]. . |
20 | 袁华,陈泽濠.基于时间卷积神经网络的短时交通流预测算法[J].华南理工大学学报(自然科学版),2020,48(11):107-113, 122. 10.12141/j.issn.1000-565X.190276 |
YUAN H, CHEN Z H. Short‑term traffic flow prediction algorithm based on time convolutional neural network[J]. Journal of South China University of Technology (Natural Science Edition), 2020,48(11): 107-113, 122. 10.12141/j.issn.1000-565X.190276 | |
21 | 周飞燕,金林鹏,董军.卷积神经网络研究综述[J].计算机学报,2017,40(6):1229-1251. 10.11897/SP.J.1016.2017.01229 |
ZHOU F Y, JIN L P, DONG J. A survey of convolutional neural networks[J]. Chinese Journal of Computers,2017,40(6): 1229-1251. 10.11897/SP.J.1016.2017.01229 | |
22 | TANGERMANN M, K‑R MÜLLER, AERTSEN A, et al. Review of the BCI Competition Ⅳ[J]. Frontiers in Neuroscience, 2012, 6: Article 00055. 10.3389/fnins.2012.00055 |
23 | SAMIEE K, KOVACS P, GABBOUJ M. Epileptic seizure classification of EEG time‑series using rational discrete short‑time Fourier transform[J]. IEEE Transactions on Biomedical Engineering, 2014, 62(2): 541-552. 10.1109/tbme.2014.2360101 |
24 | ZHAO X, ZHANG H, ZHU G, et al. A multi‑branch 3D convolutional neural network for EEG‑based motor imagery classification[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(10): 2164-2177. 10.1109/tnsre.2019.2938295 |
25 | WU H, NIU Y, LI F, et al. A parallel multiscale filter bank convolutional neural networks for motor imagery EEG classification[J]. Frontiers in Neuroscience, 2019, 13: Article 01275. 10.3389/fnins.2019.01275 |
[1] | Jinghan YIN, Shaojun QU, Zekai YAO, Xuanye HU, Xiaoyu QIN, Pujing HUA. Traffic sign recognition model in haze weather based on YOLOv5 [J]. Journal of Computer Applications, 2022, 42(9): 2876-2884. |
[2] | Jiaxuan WEI, Shikang DU, Zhixuan YU, Ruisheng ZHANG. Review of white-box adversarial attack technologies in image classification [J]. Journal of Computer Applications, 2022, 42(9): 2732-2741. |
[3] | Jinghu LI, Qianguo XING, Xiangyang ZHENG, Lin LI, Lili WANG. Noctiluca scintillans red tide extraction method from UAV images based on deep learning [J]. Journal of Computer Applications, 2022, 42(9): 2969-2974. |
[4] | Yining WANG, Qingshan ZHAO, Pinle QIN, Yulan HU, Chunmei ZONG. Super-resolution reconstruction algorithm of medical image based on lightweight dense neural network [J]. Journal of Computer Applications, 2022, 42(8): 2586-2592. |
[5] | Xianjie ZHANG, Zhiming ZHANG. Handwritten English text recognition based on convolutional neural network and Transformer [J]. Journal of Computer Applications, 2022, 42(8): 2394-2400. |
[6] | Nanjiang CHENG, Zhenxia YU, Lin CHEN, Hezhe QIAO. Multi-source and multi-label pedestrian attribute recognition based on domain adaptation [J]. Journal of Computer Applications, 2022, 42(8): 2401-2406. |
[7] | Yajiao LIU, Haitao YU, Jiang WANG, Lifeng YU, Chunhui ZHANG. Surface detection algorithm of multi-shape small defects for section steel based on deep learning [J]. Journal of Computer Applications, 2022, 42(8): 2601-2608. |
[8] | Wanjun LIU, Jiaming WANG, Haicheng QU, Libing DONG, Xinyu CAO. Music genre classification algorithm based on attention spectral-spatial feature [J]. Journal of Computer Applications, 2022, 42(7): 2072-2077. |
[9] | Yaru HAN, Lianshan YAN, Tao YAO. Deep hashing retrieval algorithm based on meta-learning [J]. Journal of Computer Applications, 2022, 42(7): 2015-2021. |
[10] | Zhenyu WANG, Lei ZHANG, Wenbin GAO, Weiming QUAN. Human activity recognition based on progressive neural architecture search [J]. Journal of Computer Applications, 2022, 42(7): 2058-2064. |
[11] | Tingwei QIN, Pengcheng ZHAO, Pinle QIN, Jianchao ZENG, Rui CHAI, Yongqi HUANG. Point cloud registration algorithm based on residual attention mechanism [J]. Journal of Computer Applications, 2022, 42(7): 2184-2191. |
[12] | Ning DONG, Xiaorong CHENG, Mingquan ZHANG. Intrusion detection system with dynamic weight loss function based on internet of things platform [J]. Journal of Computer Applications, 2022, 42(7): 2118-2124. |
[13] | Yang ZHANG, Jiangbo HAO. Malicious code detection method based on attention mechanism and residual network [J]. Journal of Computer Applications, 2022, 42(6): 1708-1715. |
[14] | Shan SU, Yang ZHANG, Dongwen ZHANG. Coupling related code smell detection method based on deep learning [J]. Journal of Computer Applications, 2022, 42(6): 1702-1707. |
[15] | Yumin HAN, Xiaoyan HAO. Material entity recognition based on subword embedding and relative attention [J]. Journal of Computer Applications, 2022, 42(6): 1862-1868. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||