南京大学学报(自然科学版) ›› 2019, Vol. 55 ›› Issue (1): 110–116.doi: 10.13232/j.cnki.jnju.2019.01.011

• • 上一篇    下一篇

基于LSTM的脑电情绪识别模型

阚 威1,李 云1,2*   

  1. 1.南京邮电大学计算机学院,南京,210023;2.江苏省大数据安全与智能处理重点实验室,南京,210023
  • 接受日期:2018-12-06 出版日期:2019-02-01 发布日期:2019-01-26
  • 通讯作者: 李 云, E-mail:liyun@njupt.edu.cn E-mail:liyun@njupt.edu.cn
  • 基金资助:
    国家自然科学基金(61603197,61772284,41571389)

Emotion recognition from EEG signals by using LSTM recurrent neural networks

Kan Wei1,Li Yun1,2*   

  1. 1. School of Computer,Nanjing University of Posts and Telecommunications,Nanjing,210023,China; 2. Jiangsu Key Laboratory of Big Data Security & Intelligent Processing,Nanjing,210023,China
  • Accepted:2018-12-06 Online:2019-02-01 Published:2019-01-26
  • Contact: Li Yun, E-mail:liyun@njupt.edu.cn E-mail:liyun@njupt.edu.cn

摘要: 已有研究表明,通过分析人类的脑电信号可以识别出其情绪信息. 近年来,机器学习技术的发展为基于脑电信号的情绪识别研究提供了可靠的技术手段. 传统的机器学习技术简单地从多个通道的脑电信号中提取特征,然后连接成单个特征向量,但是没有考虑到脑电信号中至关重要的时间动态信息. 深度学习技术中的长短时记忆(Long Short-Term Memory,LSTM)网络因其时间上的递归结构,可以很好地解决这个问题. 然而,脑电序列通常较长,直接用来训练LSTM模型所需的计算资源非常大且学习到的信息类型单一,而且忽略了许多对情绪识别非常重要的信息,如频域信息和非线性动力学信息. 为此提出一种新的基于LSTM的情绪识别模型. 脑电信号被分成多个非重叠的信号段,并从每段信号中提取多种时域、频域和非线性动力学特征,这些特征沿时间连接成特征序列并用来训练LSTM分类模型. 在DEAP数据集上验证了该模型在愉悦度、唤醒度和喜欢度上的二分类准确率,其中每个情绪维度分为低和高两类. 实验结果表明,该模型在愉悦度和喜欢度上的分类准确率均优于已有方法,在唤醒度上的分类准确率仅次于最先进的成果.

关键词: 脑电信号, 情绪识别, 机器学习, LSTM

Abstract: It has been demonstrated that it is possible to recognize human emotions by using electroencephalogram(EEG)signals. In recent few years,the development of machine learning technology has provided reliable techniques for EEG based emotion recognition research. Traditional machine learning methods extract features from multi-channel EEG signals in each channel and concatenate them into a single feature vector,ignoring critical temporal dynamic. The Long Short-Term Memory(LSTM)in deep learning technology can solve this defect well due to the recurrent structure. However,directly fitting the LSTM based model using EEG signals consumes huge computer resources and ignores important frequency domain and nonlinear dynamic information. In this paper,we present a new emotion recognition method based on LSTM. Various features in time domain,frequency domain and nonlinear dynamic are extracted from multi-channel EEG signals and constructed into feature sequences which are used to train the LSTM based model. Experiments are carried out on the DEAP benchmarking dataset for valence,arousal and liking classification respectively,and every emotional dimension is divided into two classes(low and high). Experimental results demonstrate that the classification accuracy of the proposed model outperforms the previous methods,with regard to both of valence and liking emotional dimensions,and is also comparable to the most advanced method for arousal classification.

Key words: EEG, emotion recognition, machine learning, LSTM

中图分类号: 

  • TP391
[1] Sammler D,Grigutsch M,Fritz T,et al. Music and emotion:Electrophysiological correlates of the processing of pleasant and unpleasant music. Psychophysiology,2007,44(2):293-304.
[2] Calvo R A,D’Mello S. Affect detection:An interdisciplinary review of models,methods,and their applications. IEEE Transactions on Affective Computing,2010,1(1):18-37.
[3] Yazdani A,Lee J S,Ebrahimi T. Implicit emotional tagging of multimedia using EEG signals and brain computer interface ∥ Proceedings of the first SIGMM workshop on social media. Beijing,China:ACM,2009:81-88.
[4] Nie D,Wang X W,Shi L C,et al. EEG-based emotion recognition during watching movies ∥ 2011 5th International IEEE/EMBS Conference on Neural Engineering. Cancun,Mexico:IEEE,2011:667-670.
[5] Li M,Lu B. Emotion classification based on gamma-band EEG ∥ 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology. Minneapolis,MN,USA:IEEE,2009:1223-1226.
[6] Valenzi S,Islam T,Jurica P,et al. Individual classification of emotions using EEG. Journal of Biomedical Science and Engineering,2014,7(8):604-620.
[7] Martinez H P,Bengio Y,Yannakakis G N. Learning deep physiological models of affect. IEEE Computational Intelligence Magazine,2013,8(2):20-33.
[8] Hu X,Yu J W,Song M D,et al. EEG correlates of ten positive emotions. Frontiers in Human Neuroscience,2017,11:26.
[9] Li X W,Hu B,Sun S T,et al. EEG-based mild depressive detection using feature selection methods and classifiers. Computer Methods and Programs In Biomedicine,2016,136:151-161.
[10] Chai X,Wang Q S,Zhao Y P,et al. Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition. Computers in Biology and Medicine,2016,79:205-214.
[11] Hochreiter S,Schmidhuber J. Long short-term memory. Neural Computation,1997,9(8):1735-1780.
[12] Stam C J. Nonlinear dynamical analysis of EEG and MEG:Review of an emerging field. Clinical Neurophysiology,2005,116(10):2266-2301.
[13] Zhang A H,Yang B,Huang L. Feature extraction of EEG signals using power spectral entropy ∥ 2008 International Conference on BioMedical Engineering and Informatics. Sanya,China:IEEE,2008:435-439.
[14] 张小鹏,范影乐,杨 勇. 基于奇异谱熵的脑电意识任务识别方法的研究. 计算机工程与科学,2009,31(12):117-120.(Zhang X P,Fan Y L,Yang Y. On the classification of consciousness tasks based on the EEG singular spectrum entropy. Computer Engineering & Science,2009,31(12):117-120.)
[15] Koelstra S,Muhl C,Soleymani M,et al. Deap:A database for emotion analysis;using physiological signals. IEEE Transactions on Affective Computing,2012,3(1):18-31.
[16] Mert A,Akan A. Emotion recognition from EEG signals by using multivariate empirical mode decomposition. Pattern Analysis and Applica-tions,2018,21(1):81-89.
[17] Daimi S N,Saha G. Classification of emotions induced by music videos and correlation with participants’rating. Expert Systems with Applications,2014,41(13):6057-6065.
[18] Gupta R,Laghari K U R,Falk T H. Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization. Neurocomputing,2016,174:875-884.
[19] Atkinson J,Campos D. Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers. Expert Systems with Applications,2016,47:35-41.
[1] 曹欣怡,李鹤,王蔚. 基于语料库的语音情感识别的性别差异研究[J]. 南京大学学报(自然科学版), 2019, 55(5): 758-764.
[2] 秦 娅, 申国伟, 赵文波, 陈艳平. 基于深度神经网络的网络安全实体识别方法[J]. 南京大学学报(自然科学版), 2019, 55(1): 29-40.
[3] 朱亚奇1,邓维斌1 ,2*. 一种基于不平衡数据的聚类抽样方法[J]. 南京大学学报(自然科学版), 2015, 51(2): 421-429.
[4] 朱亚奇1,邓维斌1,2*. 一种基于不平衡数据的聚类抽样方法[J]. 南京大学学报(自然科学版), 2015, 51(2): 421-429.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 孙 玫,张 森,聂培尧,聂秀山. 基于朴素贝叶斯的网络查询日志session划分方法研究[J]. 南京大学学报(自然科学版), 2018, 54(6): 1132 -1140 .
[2] 魏 桐,童向荣. 基于加权启发式搜索的鲁棒性信任路径生成[J]. 南京大学学报(自然科学版), 2018, 54(6): 1161 -1170 .
[3] 周星星,张海平,吉根林. 具有时空特性的区域移动模式挖掘算法[J]. 南京大学学报(自然科学版), 2018, 54(6): 1171 -1182 .
[4] 韩明鸣, 郭虎升, 王文剑. 面向非平衡多分类问题的二次合成QSMOTE方法[J]. 南京大学学报(自然科学版), 2019, 55(1): 1 -13 .
[5] 刘 素, 刘惊雷. 基于特征选择的CP-nets结构学习[J]. 南京大学学报(自然科学版), 2019, 55(1): 14 -28 .
[6] 王伯伟, 聂秀山, 马林元, 尹义龙. 基于语义相似度的无监督图像哈希方法[J]. 南京大学学报(自然科学版), 2019, 55(1): 41 -48 .
[7] 孔 颉, 孙权森, 纪则轩, 刘亚洲. 基于仿射不变离散哈希的遥感图像快速目标检测新方法[J]. 南京大学学报(自然科学版), 2019, 55(1): 49 -60 .
[8] 贾海宁, 王士同. 面向重尾噪声的模糊规则模型[J]. 南京大学学报(自然科学版), 2019, 55(1): 61 -72 .
[9] 严云洋, 瞿学新, 朱全银, 李 翔, 赵 阳. 基于离群点检测的分类结果置信度的度量方法[J]. 南京大学学报(自然科学版), 2019, 55(1): 102 -109 .
[10] 狄 岚, 何锐波, 梁久祯. 基于可能性聚类和卷积神经网络的道路交通标识识别算法[J]. 南京大学学报(自然科学版), 2019, 55(2): 238 -250 .