基于随机化融合和CNN的多模态肺部肿瘤图像识别

南京大学学报(自然科学版) ›› 2018, Vol. 54 ›› Issue (4): 775–.

基于随机化融合和CNN的多模态肺部肿瘤图像识别

梁蒙蒙1，周　涛1，2*，夏　勇3，张飞飞1，杨　健1

出版日期:2018-04-30
作者简介:1.宁夏医科大学公共卫生与管理学院，银川，750004； 2.宁夏医科大学理学院，银川，750004； 3. 西北工业大学计算机学院，西安，710072
基金资助:
基金项目：国家自然科学基金(61561040)，宁夏高等学校科学研究项目(NGY2016084) 收稿日期：2018－05－07 *通讯联系人，E－mail：zhoutaonxmu@126.com

Multimodal lung tumor image recognition based on randomized fusion and CNN

Liang Mengmeng1，Zhou Tao1，2*，Xia Yong3，Zhang Feifei1，Yang Jian1

Online:2018-04-30
About author:1.School of Public Health and Management，Ningxia Medical University，Yinchuan，750004，China； 2．School of Science，Ningxia Medical University，Yinchuan，750004，China； 3．School of Computer Science，Northwestern Polytechnical University，Xi’an，710072，China

摘要/Abstract

摘要： 在多模态医学图像背景下，针对单模态图像识别存在目标模糊、边界不清等问题，提出一种基于随机化特征融合的卷积神经网络(Convolutional Neural Network，CNN)目标识别方法. 首先使用参数迁移法构造卷积神经网络模型，利用自建的多模态医学图像数据库对CNN模型进行微调；然后，分别用CT(Computed Tomography)，PET(Positron Emission Computed Tomography)和PET/CT三个模态的数据并行地训练网络，并提取全连接层的特征向量；其次，构造随机函数，将三个模态的全连接层数据进行随机化融合；最后，通过另一个全连接层和分类器对融合后的特征进行分类识别. 通过批次大小、迭代次数和网络层数三个角度验证该方法的有效性，实验结果表明，随机化融合效果优于单模态CNN，且特异性和灵敏度也较高，因此该方法对临床肺部肿瘤识别具有良好的适应性．

Abstract: In the background of multimodal medical images，in order to solve the problem of blurred target and unclear boundary in single mode image recognition，a convolution neural network(CNN)target recognition method based on randomization feature fusion is proposed. In the first place，a CNN model is constructed by using the parameter migration method，and it is fine－tuning using self－built multimodal medical image database. Secondly，the three modal data of CT，PET and PET/CT are used to train the network in parallel. Each network structure is adapted to different models，and it is extracted that the feature vectors and weights of the full connection layer. Thirdly，a random function is constructed，and it is used to randomize the three feature vectors of fully connected layers and weights. At the same time，it follows the rules of fusion where the corresponding positions remain unchanged. The fusion feature vector is consistent with the size of single mode feature vector. At the end of this paper，another fully connected layer and classifier are constructed，and the fused features are classified and recognized using them. The validity of the method is verified through three angles：batch sizes，the numbers of iteration and the numbers of network layer. The experimental results show that the effect of randomization fusion is better than the single mode CNN，and the specificity and sensitivity of randomization fusion are higher than the single mode CNN to a certain extent. As the numbers of iteration layer increase，the recognition error rates of randomized fusion and single mode CNN are gradually decreased. When the batches of input data are decreased，the recognition error rates of randomized fusion and single mode CNN are gradually reduced. With the deepening of network layers，the recognition error rates of randomization fusion and single mode CNN gradually decrease. Therefore，the method of three single modal randomization fusion has good adaptability to identification of benign and malignant tumor of lung.

梁蒙蒙1，周　涛1，2*，夏　勇3，张飞飞1，杨　健1. 基于随机化融合和CNN的多模态肺部肿瘤图像识别[J]. 南京大学学报(自然科学版), 2018, 54(4): 775–.

Liang Mengmeng1，Zhou Tao1，2*，Xia Yong3，Zhang Feifei1，Yang Jian1. Multimodal lung tumor image recognition based on randomized fusion and CNN[J]. Journal of Nanjing University(Natural Sciences), 2018, 54(4): 775–.

参考文献

[1]　潘　浩，王　昭. 基于深度学习的肺癌细胞检测方法研究. 自动化与仪器仪表，2017(3)：196－197，200.(Pan H，Wang Z. Research on the approach for robust lung cancer cell detection based on deep learning. Automation & Instrumentation，2017(3)：196－197，200.) [2] 梁远凤，李　琦，罗天友. 能谱CT平扫定量分析鉴别诊断周围型肺癌与结核球. 中国医学影像技术，2017，33(8)：1206－1210.(Liang Y F，Li Q，Luo T Y. Quantitative analysis of energy spectrum CT in differential diagnosis of peripheral lung cancer and tuberculosis in unenhanced phase. Chinese Journal of Medical Imaging Technology，2017，33(8)：1206－1210.) [3] 何克磊，史颖欢，高　阳等. 一种基于原型学习的多示例卷积神经网络. 计算机学报，2017，40(6)：1265－1274.(He K L，Shi Y H，Gao Y，et al. A prototype learning based multi－instance convolutional neural network. Chinese Journal of Computers，2017，40(6)：1265－1274.) [4] 罗小刚，张承丹，侯长军等. 基于卟啉传感器阵列系统的肺癌标志物识别算法. 传感器与微系统，2017，36(4)：134－137.(Luo X G，Zhang C D，Hou C J，et al. Recognition algorithm for lung cancer markers based on porphyrin sensor array system. Transducer and Microsystem Technologies，2017，36(4)：134－137.) [5] Gao X W，Hui R，Tian Z M. Classification of CT brain images based on deep learning networks. Computer Methods and Programs in Biomedicine，2017，138：49－56. [6] Chen H，Ni D，Qin J，et al. Standard plane localization in fetal ultrasound via domain transferred deep neural networks. IEEE Journal of Biomedical and Health Informatics，2015，19(5)：1627－1636. [7] Jie C，Qi X B，Tervonen O，et al. Thorax disease diagnosis using deep convolutional neural network ∥ Proceedings of the 38th Annual International Conference of the Engineering in Medicine and Biology Society(EMBC). Orlando，FL，USA：IEEE，2016：2287－2290. [8] Mishra M，Schmitt S，Wang L C，et al. Structure－based assessment of cancerous mitochondria using deep networks ∥ Proceedings of the 13th International Symposium on Biomedical Imaging(ISBI). Prague，Czech Republic：IEEE，2016：545－548. [9] Zhao W Z，Du S H. Learning multiscale and deep representations for classifying remotely sensed imagery. ISPRS Journal of Photogrammetry and Remote Sensing，2016，113：155－165. [10] 李彦冬，郝宗波，雷　航. 卷积神经网络研究综述. 计算机应用，2016，36(9)：2508－2515，2565.(Li Y D，Hao Z B，Lei H. Survey of convolutional neural network. Journal of Computer Applications，2016，36(9)：2508－2515，2565.) [11] 常　亮，邓小明，周明全等. 图像理解中的卷积神经网络. 自动化学报，2016，42(9)：1300－1312.(Chang L，Deng X M，Zhou M Q，et al. Convolutional neural networks in image understanding. Acta Automatica Sinica，2016，42(9)：1300－1312.)　 [12] 王媛媛，周　涛，陆惠玲等. 基于集成卷积神经网络的肺部肿瘤计算机辅助诊断模型. 生物医学工程学杂志，2017，34(4)：543－551.(Wang Y Y，Zhou T，Lu H L，et al. Computer aided diagnosis model for lung tumor based on ensemble convolutional neural network. Journal of Biomedical Engineering，2017，34(4)：543－551.) [13] Lu H，Fu X，Liu C，et al. Cultivated land information extraction in UAV imagery based on deep convolutional neural network and transfer learning. Journal of Mountain Science，2017，14(4)：731－741. [14] Wang J，Sun J Q，Lin H F，et al. Convolutional neural networks for expert recommendation in community question answering. Science China Information Sciences，2017，60(11)：110102. [15] 刘　峰，沈同圣，马新星. 特征融合的卷积神经网络多波段舰船目标识别. 光学学报，2017，37(10)：1015002.(Liu F，Shen T S，Ma X X. Convolutional neural network based multi－band ship target recognition with feature fusion. Acta Optica Sinica，2017，37(10)：1015002.) [16] 冯家文，张立民，邓向阳. 双通道卷积神经网络在静态手势识别中的应用. 计算机工程与应用，2017：1－7.(Feng J W，Zhang L M，Deng X Y. Application of doublechannel convolution neural network in static gesture recognition. Computer Engineering and Applications，2017：1－7.)

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed