南京大学学报(自然科学版) ›› 2022, Vol. 58 ›› Issue (1): 29–37.doi: 10.13232/j.cnki.jnju.2022.01.004

• • 上一篇    下一篇

基于注意力机制和深度学习的运动想象脑电信号分类方法

张玮1, 赵永虹2, 邱桃荣1()   

  1. 1.南昌大学信息工程学院，南昌，330029
    2.四川开放大学工程技术学院，成都，610073
  • 收稿日期:2021-06-28 出版日期:2022-01-30 发布日期:2022-02-22
  • 通讯作者: 邱桃荣 E-mail:qiutaorong@ncu.edu.cn
  • 作者简介:E⁃mail： qiutaorong@ncu.edu.cn
  • 基金资助:
    国家自然科学基金(61662045)

Classification of motor imagery EEG signals based on attention mechanism and deep learning

Wei Zhang1, Yonghong Zhao2, TaoRong Qiu1()   

  1. 1.Information Engineering School of Nanchang University, Nanchang, 330029, China
    2.Engineering and Technology School, The Open University of Sichuan, Chengdu, 610073, China
  • Received:2021-06-28 Online:2022-01-30 Published:2022-02-22
  • Contact: TaoRong Qiu E-mail:qiutaorong@ncu.edu.cn

RichHTML

328311

PDF (PC)

1287

摘要：

脑机接口（Brain Computer Interface，BCI）作为一种新型的信息沟通与控制手段，是一个涉及神经科学、信号处理以及模式识别等多个学科的交叉研究课题.基于运动想象的BCI系统被认为是最具发展前景的一种脑机接口系统.针对基于机器学习方法构建脑电特征与运动想象之间的映射关系进行分类时，现有方法仍存在无法兼顾脑电信号的时?空域特征并且分类精度难以提高的问题，提出一种基于注意力机制的双向长短时记忆网络（ABiLSTM）和卷积神经网络（Convolutional Neural Network，CNN）的运动想象脑电分析方法，通过ABiLSTM和CNN相结合的方式提取更具表征性的脑电信号深层特征.综合上述研究，最终构建的ABiLSTM?CNN分类模型在公共数据集上取得了90.72%的平均准确率，证明ABiLSTM?CNN运动想象脑电信号分类模型具有很理想的分类性能.

关键词: 深度学习, 注意力机制, 双向长短时记忆, 卷积神经网络, 脑电信号, 运动想象

Abstract:

As a new means of information communication and control，brain computer interface (BCI) is an interdisciplinary research topic involving neuroscience，signal processing and pattern recognition. The BCI system based on motor imagination is regarded as one of the most promising BCI systems. The traditional electroencephalogram (EEG) classification methods based on machine learning complete the classification task by constructing the mapping relationship between EEG features and motor imagery. However，the existing methods still have problems that cannot take into account the time?spatial feature of EEG signals，and the classification accuracy is difficult to improve. In this paper，a motor imagery EEG classification method is constructed based on deep learning，and through the way of combining Bi?directional Long Short?Term Memory (LSTM) based on attention mechanism (ABiLSTM) with Convolutional Neural Network （CNN） to extract more in?depth characteristic of EEG signals. Based on the above research，the ABiLSTM?CNN has achieved an average accuracy of 90.72% on EEG Motor Imagery dataset. The results show that the ABiLSTM?CNN motor imagery EEG signal classification model has ideal classification performance.

Key words: deep learning, attention mechanism, BiLSTM, CNN, EEG signal, motor imagery

中图分类号: 

  • TP301

图1

基于ABiLSTM?CNN的运动想象脑电信号分类模型"

图2

LSTM cell结构示意图"

图3

ABiLSTM模块结构图"

图4

CNN模块结构图"

图5

四种基于LSTM改进网络的脑电信号分类准确率对比"

表1

不同隐藏层神经元数目的ABiLSTM网络分类准确率"

Num of CellEpoch
20406080100
1646.37%48.53%50.39%51.01%52.24%
3248.85%51.92%55.02%55.11%56.43%
6452.18%56.11%58.87%60.26%71.62%
12856.29%62.42%67.08%70.15%71.28%
25661.23%71.02%74.16%77.27%78.62%
51265.36%76.93%77.91%77.09%78.73%
102474.21%77.65%79.52%78.23%79.33%

表2

不同注意力层神经元数目的ABiLSTM模型分类准确率"

Num of CellEpoch
20406080100
457.03%65.50%72.90%74.61%76.80%
861.23%71.02%74.16%77.27%78.62%
1657.24%68.52%72.61%74.32%73.56%
3257.64%65.45%71.28%74.29%76.01%
6459.09%66.53%70.65%73.34%75.39%
12855.21%64.19%69.33%73.84%72.73%
25655.54%63.41%68.45%71.63%72.28%

表3

不同卷积层数的ABiLSTM?CNN的分类准确率"

卷积

层数

卷积核大小
1×31×51×71×9
183.12%83.35%83.24%83.09%
284.75%85.10%85.47%86.33%
388.82%89.24%89.35%88.67%
488.48%88.96%87.61%88.02%
587.34%86.84%87.80%87.73%

表4

ABiLSTM?CNN在MID?a上的分类准确率、精确率、召回率和F1 score"

编号AccuracyPrecisionRecallF1?score
平均90.52%90.35%90.56%90.46%
189.05%88.40%89.96%89.17%
292.81%92.31%92.07%92.19%
390.24%90.39%90.04%90.21%
491.31%91.12%91.68%91.40%
589.21%89.53%89.07%89.30%

表5

ABiLSTM?CNN与对比算法在MID?a上的分类准确率对比"

特征提取分类器平均准确率
Kalaivani et al[15]DWTK?means72.60%
Pinheiro et al[17]FFTANN74.96%
Alomari et al[14]Coif4+IEEGANN71.6%
Wang[20]—Em?RNN88.27%
Ours—ABiLSTM78.62%
Ours—ABiLSTM?CNN90.52%

表6

ABiLSTM?CNN在MID?b上的分类准确率、精确率、召回率和F1 score"

编号AccuracyPrecisionRecallF1?score
平均89.96%89.97%89.73%89.85%
S2189.05%89.20%89.32%89.26%
S2291.43%91.06%91.06%91.06%
S2393.62%92.33%92.08%92.20%
S2485.48%86.30%86.11%86.20%
S2590.24%90.96%90.06%90.51%
1 Son J，Ku J. Development of brain computer interface based action observation program with functional electrical stimulation device (FES)∥2019 7th International Winter Conference on Brain?Computer Interface. Gangwon，Korea (South)：IEEE，2019：1-2.
2 高上凯. 脑机接口的现状与未来. 机器人产业，2019(5)：40-44.
3 Branco M P，Pels E G M，Sars R H，et al. Brain?computer interfaces for communication：Preferences of individuals with locked?in syndrome. Neurorehabilitation and Neural Repair，2021，35(3):267-279.
4 Gandevia S C，Rothwell J C. Knowledge of motor commands and the recruitment of human motoneurons. Brain，1987，110(5)：1117-1130.
5 李静雯，王秀梅. 脑机接口技术在医疗领域的应用. 信息通信技术与政策，2021，47(2)：87-91.
6 Torres E P，Torres A，Hernández?álvarez M，et al. EEG?based BCI emotion recognition：A survey. Sensors，2020，20(18)：5083.
7 K?gel J，Jox R J，Friedrich O. What is it like to use a BCI?Insights from an interview study with brain?computer interface users. BMC Medical Ethics，2020，21(1)：2.
8 Song X M，Aguilar L，Herb A，et al. Dynamic modeling and classification of epileptic EEG data∥2019 9th International IEEE/EMBS Conference on Neural Engineering. San Francisco，CA，USA：IEEE，2019：49-52.
9 Sullivan L S，Klein E，Brown T，et al. Keeping disability in mind：a case study in implantable brain–computer Interface research. Science and Engineering Ethics，2018，24(2)：479-504.
10 Ye B G，Qiu T R，Bai X M，et al. Research on recognition method of driving fatigue state based on sample entropy and kernel principal component analysis. Entropy，2018，20(9)：701.
11 Anderson C，Forney E，Hains D，et al. Reliable identification of mental tasks using time?embedded EEG and sequential evidence accumulation. Journal of Neural Engineering，2011，8(2)：025023.
12 Lemm S，Blankertz B，Curio G，et al. Spatio?spectral filters for improving the classification of single trial EEG. IEEE Transactions on Biomedical Engineering，2005，52(9)：1541-1548.
13 Subasi A，Gursoy M I. EEG signal classification using PCA，ICA，LDA and support vector machines. Expert Systems with Applications，2010，37(12)：8659-8666.
14 Alomari M H，AbuBaker A，Turani A，et al. EEG mouse：A machine learning?based brain computer interface. International Journal of Advanced Computer Science and Applications，2014，5(4)：193-198.
15 Kalaivani M，Kalaivani V，Devi V A. Analysis of EEG signal for the detection of brain abnormalities∥IJCA Proceedings of International Conference on Simulations in Computing Nexus. San Diego,CA，USA：ICSCN，2014：1-6.
16 Kim H S，Chang M H，Lee H J，et al. A comparison of classification performance among the various combinations of motor imagery tasks for brain?computer interface∥2013 6th International IEEE/EMBS Conference on Neural Engineering. San Diego，CA，USA：IEEE，2013：435-438.
17 Pinheiro O R，Alves L R G，Souza J R D. EEG signals classification：Motor imagery for driving an intelligent wheelchair. IEEE Latin America Transactions，2018，16(1)：254-259.
18 Dose H，M?ller J S，Iversen H K，et al. An end?to?end deep learning approach to MI?EEG signal classification for BCIs. Expert Systems with Applications，2018(114)：532-542.
19 Qiao W Z，Bi X J. Deep spatial?temporal neural network for classification of EEG?based motor imagery∥Proceedings of 2019 International Conference on Artificial Intelligence and Computer Science. New York，NY，USA：ACM，2019：265-272.
20 王薇. 基于深度学习的脑电分类识别方法. 硕士学位论文. 南京：南京邮电大学，2020.
Wang W. Classification and recognition method of EEG based on deep learning. Master Dissertation. Nanjing：Nanjing University of Posts and Telecommunications，2020.
21 Waldert S. Invasive vs. non?invasive neuronal signals for brain?machine interfaces：Will one prevail?. Frontiers in Neuroscience，2016(10)：295.
22 Hochreiter S，Schmidhuber J. Long short?term memory. Neural Computation，1997，9(8):1735-1780.
23 Bahdanau D，Cho K，Bengio Y. Neural machine translation by jointly learning to align and translate. 2014，arXiv:.
24 Goldberger A，Amaral L，Glass L，et al. PhysioBank，PhysioToolkit，and PhysioNet：Components of a new research resource for complex physiologic signals. Circulation，2000，101(23)：e215-e220.
[1] 樊炎, 匡绍龙, 许重宝, 孙立宁, 张虹淼. 一种同步提取运动想象信号时⁃频⁃空特征的卷积神经网络算法[J]. 南京大学学报(自然科学版), 2021, 57(6): 1064-1074.
[2] 孟浩, 刘强. 基于FPGA的卷积神经网络训练加速器设计[J]. 南京大学学报(自然科学版), 2021, 57(6): 1075-1082.
[3] 陈磊, 孙权森, 王凡海. 基于深度对抗网络和局部模糊探测的目标运动去模糊[J]. 南京大学学报(自然科学版), 2021, 57(5): 735-749.
[4] 倪斌, 陆晓蕾, 童逸琦, 马涛, 曾志贤. 胶囊神经网络在期刊文本分类中的应用[J]. 南京大学学报(自然科学版), 2021, 57(5): 750-756.
[5] 杨静, 赵文仓, 徐越, 冯旸赫, 黄金才. 一种基于少样本数据的在线主动学习与分类方法[J]. 南京大学学报(自然科学版), 2021, 57(5): 757-766.
[6] 房笑宇, 曹陈涵, 夏彬. 基于注意力机制的大规模系统日志异常检测方法[J]. 南京大学学报(自然科学版), 2021, 57(5): 785-792.
[7] 徐樱笑, 资文杰, 宋洁琼, 邵瑞喆, 陈浩. 基于多站点、多时间注意力机制的电磁强度时空关联分析与可视化[J]. 南京大学学报(自然科学版), 2021, 57(5): 838-846.
[8] 贾霄, 郭顺心, 赵红. 基于图像属性的零样本分类方法综述[J]. 南京大学学报(自然科学版), 2021, 57(4): 531-543.
[9] 普志方, 陈秀宏. 基于卷积神经网络的细胞核图像分割算法[J]. 南京大学学报(自然科学版), 2021, 57(4): 566-574.
[10] 段建设, 崔超然, 宋广乐, 马乐乐, 马玉玲, 尹义龙. 基于多尺度注意力融合的知识追踪方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 591-598.
[11] 颜志良, 丰智鹏, 刘丹, 王会青. 一种混合深度神经网络的赖氨酸乙酰化位点预测方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 627-640.
[12] 陈妍, 杨芸翰, 陶建成, 李欣. 隧道噪声干扰下疲劳影响研究[J]. 南京大学学报(自然科学版), 2021, 57(4): 690-694.
[13] 方志文, 刘青山, 周峰. 基于像素⁃目标级共生关系学习的多标签航拍图像分类方法[J]. 南京大学学报(自然科学版), 2021, 57(2): 208-216.
[14] 罗金屯, 滕飞, 周亚波, 池茂儒, 张海波. 数据驱动的高速铁路轮轨作用力反演模型[J]. 南京大学学报(自然科学版), 2021, 57(2): 299-308.
[15] 范习健, 杨绪兵, 张礼, 业巧林, 业宁. 一种融合视觉和听觉信息的双模态情感识别算法[J]. 南京大学学报(自然科学版), 2021, 57(2): 309-317.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!

版权所有 © 2021 《南京大学学报(自然科学)》

地址：江苏省南京市鼓楼区汉口路22号，南京大学学报编辑部，210093

电话：025-83592704 , 传真：025-83592704        技术支持：北京玛格泰克科技发展有限公司