南京大学学报(自然科学版) ›› 2021, Vol. 57 ›› Issue (5): 757–766.doi: 10.13232/j.cnki.jnju.2021.05.005

• • 上一篇    

一种基于少样本数据的在线主动学习与分类方法

杨静1,2(), 赵文仓3, 徐越4, 冯旸赫1, 黄金才1   

  1. 1.国防科技大学系统工程学院，长沙，410073
    2.海军潜艇学院作战指挥系，青岛，266041
    3.青岛科技大学自动化与电子工程学院，青岛，266061
    4.解放军31102部队，南京，210016
  • 收稿日期:2021-06-29 出版日期:2021-09-29 发布日期:2021-09-29
  • 通讯作者: 杨静 E-mail:estella126@126.com
  • 作者简介:E⁃mail：estella126@126.com
  • 基金资助:
    国家自然科学基金(71701205)

An on⁃line active learning and classification method based on small sample data

Jing Yang1,2(), Wencang Zhao3, Yue Xu4, Yanghe Feng1, Jincai Huang1   

  1. 1.College of Computer Science and Technology，National University of Defense and Technology，Changsha, 410073，China
    2.Combat Command Department，Submarine College，Qingdao, 266041，China
    3.College of Automation and Electronic Engineering，Tsingtao University of Science and Technology，Qingdao，266061, China
    4.PLA 31102 Troops，Nanjing, 210016, China
  • Received:2021-06-29 Online:2021-09-29 Published:2021-09-29
  • Contact: Jing Yang E-mail:estella126@126.com

RichHTML

4353

PDF (PC)

1295

摘要：

深度学习技术往往需要大量的标记训练样本，然而真实环境中获得大量标记样本的代价昂贵，甚至难以获得.提出一种新的深度主动识别框架，借鉴人类逐步学习并获取新知识的认知过程，在基于少量样本分析模型认知错误的基础上首先定义认知错误转化，得到相应的知识来主动增强模型的认知信息.基于认知知识，选取敏感样本对模型进行在线微调，同时为了避免遗忘先前学习的知识选取先前的训练样本作为刷新样本.在通用数据集上的实验表明，敏感样本有利于提高目标识别能力，提出的认知学习机制能有效提高深度模型的性能；认知信息的特征化能有效抑制其他样本对模型认知的干扰，在线训练方法能明显节省大量训练时间.提供了一个有效的认知学习在少量数据样本情境中的应用.

关键词: 深度学习, 主动认知, 感知学习, 少量样本, 感知信息

Abstract:

Deep learning requires a large number of labeled training samples，however，it is expensive or even impossible to obtain a large number of labeled data at first in real world. In this paper，a new deep active recognition framework is proposed，which is based on the cognitive process of how we learning and acquiring new knowledge step by step. On the basis of analyzing the cognitive errors of the model based on a small number of samples，the transformation of cognitive errors is defined，and the corresponding knowledge is obtained to actively enhance the cognitive information of the model. Based on cognitive knowledge，sensitive samples are selected to fine tune the model online. At the same time，in order to avoid forgetting the previously learned knowledge，the previous training samples are selected as refresh samples. Experiments on general datasets show that the Target Sensitive Samples (TSSs) can improve the performamce of target recognition，and the proposed cognitive learning mechanism can effectively improve the performance of deep model. The characterization of cognitive information effectively restrains the interference of other samples on the model cognition，and the online training method significantly saves a lot of training time，which provides an effective application of cognitive learning in the situation of a small number of data samples.

Key words: deep learning, active recognition, cognitive learning, small sample, cognitive information

中图分类号: 

  • TP301

图1

基于主动学习的感知学习框架"

图2

主动感知过程的框架"

图3

CISs的选择过程"

图4

敏感样本（TSSs）的选择过程"

图5

训练前的样本选择"

图6

不同微调次数下四种模型的测试精度对比"

表1

MNIST手写数据集的识别结果"

方法T9T16T23T30T44T51T58T65T72T79T86T93T100
变异比83.75%87.68%90.87%93.13%93.03%94.96%94.97%95.15%95.80%95.54%94.96%96.13%96.59%
预测熵83.53%85.04%90.30%91.05%95.09%94.67%96.23%97.11%96.93%97.94%96.85%98.03%98.01%
BALD86.73%89.06%93.75%92.77%95.85%95.20%96.76%97.39%97.48%98.04%98.32%98.20%98.15%
CISs93.25%94.93%95.87%95.13%96.03%96.96%97.97%99.15%99.80%99.54%99.96%99.13%99.25%

表2

CIFAR?10的识别结果"

方法T9T16T23T30T44T51T58T65T72T79T86T93T100
变异比51.92%55.38%57.68%59.54%60.98%62.32%63.75%64.29%64.29%65.99%66.71%67.48%68.12%
预测熵49.96%53.55%56.05%58.38%60.14%62.18%63.45%65.07%66.27%67.12%68.26%69.14%69.53%
BALD49.67%53.47%56.96%58.92%60.41%61.95%63.72%64.56%66.09%67.17%67.36%69.15%69.90%
CISs56.19%60.69%63.36%66.08%67.38%69.18%69.75%71.18%71.97%72.23%72.46%73.45%73.75%

表3

CIFAR?100的识别结果"

方法T9T16T23T30T44T51T58T65T72T79T86T93T100
变异比19.03%23.14%23.29%24.44%27.60%27.97%29.89%30.97%32.98%32.86%33.84%34.19%35.15%
预测熵21.13%23.24%26.13%27.52%30.04%30.80%32.34%33.10%33.05%34.53%35.46%35.62%36.58%
BALD23.26%25.96%27.77%29.29%31.26%31.50%33.01%33.84%34.83%35.69%36.16%35.79%37.81%
CISs28.81%31.06%32.23%33.22%35.01%36.26%37.04%37.72%39.16%39.24%40.18%38.65%40.18%

表4

MNIST,CIFAR?10,CIFAR?100在不同训练集上的时间耗费比较"

方法MNISTCIFAR?10CIFAR?100
100轮耗时(s)每轮平均耗时(s)100轮耗时(s)每轮平均耗时(s)100轮耗时(s)每轮平均耗时(s)
变异比?AD33699.99336.99520540.785205.40833933.658339.33
变异比?OD18809.54188.0921807.34218.0734354.54343.54
预测熵?AD30081.62300.81487761.184877.61813504.128135.04
预测熵?OD17943.67179.4321043.45210.4334065.86340.65
BALD?AD29989.71299.89484232.124842.32803423.558034.23
BALD?OD17353.88173.5320942.65209.4233901.54339.01
CIS?AD49023.34490.23689832.736898.321232323.5112323.23
CIS?OD21302.62213.0230231.51302.3139515.21395.15
1 程健，张会. 潜艇使用声抗器材防御鱼雷方案优化模型及模型求解策略. 指挥控制与仿真，2019,041(006):48-51.
Cheng J,Zhang H. Optimal model and model solving strategy of submarine torpedo defence using acoustic countermeasure equipment. Command Control & Simulation,2019,041(006):48-51.
2 Gal Y，Islam R，Ghahramani Z. Deep bayesian active learning with image data∥Proceedings of the 34th International Conference on Machine Learning. Sydney，Australia：ACM，2016：1183-1192.
3 Feng Y H，Shi W，Cheng G Q，et al. Benchmarking framework for command and control mission planning under uncertain environment. Soft Computing，2020，24(4)：2463-2478.
4 Master A，Meltzoff A N，Lent R. Neuroscience，psychology，and soci-ety：Translating research to improve learning. Prospects，2016，46(2)：191-198.
5 Silva D F，Giusti R，Keogh E，et al. Speeding up similarity search under dynamic time warping by pruning unpromising alignments. Data Mining and Knowledge Discovery，2018，32(4)：988-1016.
6 Yang Q，Wu X D. 10 challenging problems in data mining research. International Journal of Information Technology & Decision Making，2006，5(4)：597-604.
7 Chen L，Huang H L，Feng Y H，et al. Active one?shot learning by a deep Q?network strategy. Neurocomputing，2020(383)：324-335.
8 Bagnall A，Lines J，Bostrom A，et al. The great time series classification bake off：A review and experimental evaluation of recent algorithmic advances. Data Mining and Knowledge Discovery，2017，31(3)：606-660.
9 LeCun Y，Bengio Y，Hinton G. Deep learning. Nature，2015，521(7553)：436-444.
10 Feng Y H，Yang X H，Cheng G Q. Stability in mean for multi?dimensional uncertain differential equation. Soft Computing，2018，22(17)：5783-5789.
11 Kendall A，Gal Y. What uncertainties do we need in bayesian deep learning for computer vision?∥Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach，CA，USA：ACM，2017：5574-5584.
12 Scheffer T，Decomain C，Wrobel S. Active hidden markov models for information extraction∥The 4th International Symposium on Intelligent Data Analysis. Springer Berlin Heidelberg，2001：309-318.
Pan S J，Yang Q. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering，2010，22(10)：1345-1359.
13 Lines J，Bagnall A. Time series classification with ensembles of elastic distance measures. Data Mining and Knowledge Discovery，2015，29(3)：565-592.
14 Dayoub F，Sünderhauf N，Corke P I. Episode?based active learning with bayesian neural networks∥Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu，HI，USA：IEEE，2017：498-500.
15 Chen H H，Tang F Z，Tino P，et al. Model metric co?learning for time series classification∥Proceedings of the 24th International Conference on Artificial Intelligence. Buenos Aires，Argentina：ACM，2015：3387-3394.
16 Lalor J P，Wu H，Munkhdalai T，et al. An analysis of machine learning intelligence. 2017,arXiv:1702. 04811.
17 Yang T T，Feng Y H，Cheng G Q，et al. Critical events based resource layer structure dynamic adaptive optimization method. IEEE Access，2019(7)：36710-36721.
18 Kingma D P，Welling M. Auto?encoding variational bayes. 2013,arXiv:.
19 Nweke H F，Teh Y W，Al?garadi M A，et al. Deep learning algorithms for human activity recognition using mobile and wearable sensor networks：State of the art and research challenges. Expert Systems with Applications，2018(105)：233-261.
20 Houlsby N，Huszár F，Ghahramani Z，et al. Bayesian active learning for classification and preference learning. 2011,arXiv:.
21 Che Z P，He X R，Xu K，et al. DECADE：A deep metric learning model for multivariate time series∥Proceedings of the 3rd SIGKDD Workshop on Mining and Learning from Time Series. Halifax，Canada：ACM，2017：1-9.
22 Le Nguyen T，Gsponer S，Ifrim G. Time series classification by sequence learning in all?subsequence space∥Proceedings of 2017 IEEE 33rd International Conference on Data Engineering. San Diego，CA，USA：IEEE，2017：947-958.
23 Hattie J A C，Donoghue G M. Learning strategies：A synthesis and conceptual model. NPJ Science of Learning，2016，1(1)：16013.
24 Gong Z C，Chen H H，Yuan B，et al. Multiobjective learning in the model space for time series classification. IEEE Transactions on Cybernetics，2019，49(3)：918-932.
25 Kotsifakos A，Papapetrou P. Model?based time series classification∥The 13th International Symposium on Intelligent Data Analysis. Springer Berlin Heidelberg，2014：179-191.
26 Gal Y. Uncertainty in deep learning. Ph.D. Dissertation. Cambridge：University of Cambridge，2016.
27 Bagnall A，Janacek G. A run length transformation for discriminating between auto regressive time series. Journal of Classification，2014，31(2)：154-178.
28 Papernot N，McDaniel P. Deep k?nearest neighbors：Towards confident，interpretable and robust deep learning. 2018,arXiv：.
[1] 陈磊, 孙权森, 王凡海. 基于深度对抗网络和局部模糊探测的目标运动去模糊[J]. 南京大学学报(自然科学版), 2021, 57(5): 735-749.
[2] 倪斌, 陆晓蕾, 童逸琦, 马涛, 曾志贤. 胶囊神经网络在期刊文本分类中的应用[J]. 南京大学学报(自然科学版), 2021, 57(5): 750-756.
[3] 贾霄, 郭顺心, 赵红. 基于图像属性的零样本分类方法综述[J]. 南京大学学报(自然科学版), 2021, 57(4): 531-543.
[4] 普志方, 陈秀宏. 基于卷积神经网络的细胞核图像分割算法[J]. 南京大学学报(自然科学版), 2021, 57(4): 566-574.
[5] 段建设, 崔超然, 宋广乐, 马乐乐, 马玉玲, 尹义龙. 基于多尺度注意力融合的知识追踪方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 591-598.
[6] 罗金屯, 滕飞, 周亚波, 池茂儒, 张海波. 数据驱动的高速铁路轮轨作用力反演模型[J]. 南京大学学报(自然科学版), 2021, 57(2): 299-308.
[7] 曾宪华, 陆宇喆, 童世玥, 徐黎明. 结合马尔科夫场和格拉姆矩阵特征的写实类图像风格迁移[J]. 南京大学学报(自然科学版), 2021, 57(1): 1-9.
[8] 余方超, 方贤进, 张又文, 杨高明, 王丽. 增强深度学习中的差分隐私防御机制[J]. 南京大学学报(自然科学版), 2021, 57(1): 10-20.
[9] 张萌, 韩冰, 王哲, 尤富生, 李浩然. 基于深度主动学习的甲状腺癌病理图像分类方法[J]. 南京大学学报(自然科学版), 2021, 57(1): 21-28.
[10] 李一凡, 朱斐, 凌兴宏, 刘全. 具有窗口结构Bi⁃LSTM网络的心电图QRS波检测方法[J]. 南京大学学报(自然科学版), 2021, 57(1): 42-51.
[11] 温玉莲, 林培光. 基于行业背景差异下的金融时间序列预测方法[J]. 南京大学学报(自然科学版), 2021, 57(1): 90-100.
[12] 潘越,王骏,李文飞,张建,王炜. 基于卷积神经网络的蛋白质折叠类型最小特征提取[J]. 南京大学学报(自然科学版), 2020, 56(5): 744-753.
[13] 朱伟,张帅,辛晓燕,李文飞,王骏,张建,王炜. 结合区域检测和注意力机制的胸片自动定位与识别[J]. 南京大学学报(自然科学版), 2020, 56(4): 591-600.
[14] 李康,谢宁,李旭,谭凯. 基于卷积神经网络和几何优化的统计染色体核型分析方法[J]. 南京大学学报(自然科学版), 2020, 56(1): 116-124.
[15] 韩普,刘亦卓,李晓艳. 基于深度学习和多特征融合的中文电子病历实体识别研究[J]. 南京大学学报(自然科学版), 2019, 55(6): 942-951.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!

版权所有 © 2021 《南京大学学报(自然科学)》

地址：江苏省南京市鼓楼区汉口路22号，南京大学学报编辑部，210093

电话：025-83592704 , 传真：025-83592704        技术支持：北京玛格泰克科技发展有限公司