南京大学学报(自然科学版) ›› 2021, Vol. 57 ›› Issue (4): 531543.doi: 10.13232/j.cnki.jnju.2021.04.001
• • 下一篇
Xiao Jia1, Shunxin Guo1, Hong Zhao1,2()
摘要:
随着机器学习技术的不断发展,深度学习在许多研究领域取得了巨大的突破.然而,多数深度学习方法需要大量的有标注数据进行模型拟合,不符合现实世界的一些应用场景,而零样本学习则可有效地缓解该问题.具体地,零样本学习主要针对样本数量稀少、新样本的出现和分类任务人工标注成本高等一系列问题给出有效的解决方案,对图像分类有重要意义.系统综述基于图像属性的零样本学习方法:首先,系统概述零样本学习的定义及零样本学习的发展历程;其次,对基于图像属性的零样本分类的三类主要方法进行介绍,并讨论了各方法的区别和联系;最后,指出了零样本学习现在仍存在的问题以及未来发展的方向.
中图分类号:
1 | Agliari E,Alemanno F,Barra A,et al. Generalized Guerra's interpolation schemes for dense associative neural networks. Neural Networks,2020(128):254-267. |
2 | Changpinyo S,Chao W L,Gong B Q,et al. Classifier and exemplar synthesis for zero?shot learning. Inter?national Journal of Computer Vision,2020 (128):166-201. |
3 | Koeppe A,Bamer F,Markert B. An intelligent nonlinear meta element for elastoplastic continua:Deep learning using a new time?distributed residual |
U?Net architecture. Computer Methods in Applied Mechanics and Engineering,2020(366):113088. | |
4 | Chowdhury R,Mahdy M R C,Alam T N,et al. Predicting the stock price of frontier markets using machine learning and modified black?scholes option pricing model. Physica A:Statistical Mechanics and Its Applications,2020(555):124444. |
5 | Ben Hassine N. Machine learning for network resource management. Ph.D. Dissertation. Paris:Université Paris?Saclay (ComUE),2017. |
6 | 盛一堃. 基于深度学习的迁移学习方法研究与应用. 硕士学位论文. 合肥:合肥工业大学,2019. (Sheng Y K. Research and application of transfer learning method based on deep learning. Master |
Dissertation. Hefei:HefeiUniversity of Technology,2019. | |
7 | Guo Y C,Ding G G,Han J G,et al. Zero?shot learning with transferred samples. IEEE Transactions on Image Processing,2017,26(7):3277-3290. |
8 | Yang L,Gao X B,Gao Q X,et al. Label?activating framework for zero?shot learning. Neural Networks,2020(121):1-9. |
9 | Wu Z Q,Serie D,Xu G G,et al. PB?Net:Automatic peak integration by sequential deep learning for multiple reaction monitoring. Journal of Proteomics,2020(223):103820. |
10 | Singla N,Srivastava V. Deep learning enabled multi?wavelength spatial coherence microscope for the classification of malaria?infected stages with limited labelled data size. Optics and Laser Technology,2020(130):106335. |
11 | Kim H,Lee J,Byun H. Unseen image generating domain?free networks for generalized zero?shot learning. Neurocomputing,2020(411):67-77. |
12 | 叶翰嘉,詹德川. 基于模型分解的小样本学习. 中国科学:信息科学,2020,50(5):662-674. |
Ye H J,Zhan D C. Few?shot learning via model decomposition. Scientia Sinica Informationis,2020,50(5):662-674. | |
13 | Larochelle H,Erhan D,Bengio Y. Zero?data learning of new tasks∥Proceedings of the 23rd National Conference on Artificial Intelligence. Chicago,IL,USA:AAAI Press,2008:646-651. |
14 | Palatucci M,Pomerleau D,Hinton G,et al. Zero?shot learning with semantic output codes∥Proceedings of the 22nd International Conference on Neural Information Processing Systems. Vancouver,Canada:Curran Associates Inc.,2009:1410-1418. |
15 | Lampert C H,Nickisch H,Harmeling S. Learning to detect unseen object classes by between?class attribute transfer∥2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami,FL,USA:IEEE,2009:951-958. |
16 | Kaiser R,Kim S,Lee D. Deep data analysis for aspiration pressure estimation in a high?pressure gas atomization process using an artificial neural network. Chemical Engineering and Processing,2020(153):107924. |
17 | 周翔. 基于深度生成模型的零样本学习. 硕士学位论文. 成都:电子科技大学,2019. |
Zhou X. Zero∥shot learning research based on deep generative | |
models. Master Dissertation. Chengdu:University of Electronic Science and Technology of China,2019. | |
18 | Jiang J W,Kang L,Huang J J,et al. Deep learning based mild cognitive impairment diagnosis using structure MR images. Neuroscience Letters,2020 (730):134971. |
19 | Saleem N,Khattak M I. Deep neural networks based binary classification for single channel speaker independent multi?talker speech separation. Applied Acoustics,2020(167):107385. |
20 | Krizhevsky A,Sutskever I,Hinton G E. ImageNet classification with deep convolutional neural networks∥Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe,NV,USA:Curran Associates Inc.,2012:1097-1105. |
21 | Yin W S,Yang W H,Liu H Y. A neural network scheme for recovering scattering obstacles with limited phaseless far?field data. Journal of Computational Physics,2020(417):109594. |
22 | Agre? M,Avsec J,Stru?nik D. Entropy and exergy analysis of steam passing through an inlet steam turbine control valve assembly using artificial neural networks. International Journal of Heat and Mass Transfer,2020(156):119897. |
23 | Ampatzidis Y,Partel V,Costa L. Agroview:cloud?based application to process,analyze and visualize UAV?collected data for precision agriculture applica?tions utilizing artificial intelligence. Computers and Electronics in Agriculture,2020(174):105457. |
24 | Thai N X,Tonezzer M,Masera L,et al. Multi gas sensors using one nanomaterial,temperature gradient and machine learning algorithms for discrimination of gases and their concentration. Analytica Chimica Acta,2020(1124):85-93. |
25 | Dubey R,Gunasekaran A,Childe S J,et al. Big data analytics and artificial intelligence pathway to operational performance under the effects of entrepreneurial orientation and environmental dynamism:A study of manufacturing organisations. International Journal of Production Economics,2020 (226):107599. |
26 | Trong V H,Gwang?Hyun Y,Vu D T,et al. Late fusion of multimodal deep neural networks for weeds classification. Computers and Electronics in Agriculture,2020(175):105506. |
27 | Li G. The development of VLSI implementation?related neural network training algorithms. Master Dissertation. Vancouver,Canada:The University of British Columbia,1994. |
28 | Li A X,Lu Z W,Guan J C,et al. Transferrable feature and projection learning with class hierarchy for zero?shot learning. International Journal of Computer Vision,2020,128(12):2810-2827. |
29 | Qiu R J,Wang Y K,Wang D,et al. Water temperature forecasting based on modified artificial neural network methods:Two cases of the Yangtze River. Science of the Total Environment,2020(737):139729. |
30 | Liu Z Z,Zhang X X,Zhu Z F,et al. Convolutional prototype learning for zero?shot recognition. Image and Vision Computing,2020(98):103924. |
31 | 潘崇煜,黄健,郝建国等. 融合零样本学习和小样本学习的弱监督学习方法综述. 系统工程与电子技术,2020,42(10):2246-2256. |
Pan C Y,Huang J,Hao J G,et al. Survey of weakly supervised learning integrating zero?shot and few?shot learning. Systems Engineering and Electronics,2020,42(10):2246-2256. | |
32 | 马世龙,乌尼日其其格,李小平. 大数据与深度学习综述. 智能系统学报,2016,11(6):728-742. (Ma S L,Wuniri Q Q G,Li X P. Deep learning with big |
data:state of the art and development. CAAI | |
Transactions on Intelligent Systems,2016,11(6):728-742. | |
33 | 魏宏喜,张越. 基于生成对抗网络的零样本图像分类. 北京航空航天大学学报,2019,45(12):2345-2350. (Wei H X,Zhang Y. Zero?shot image classification based on generative adversarial network. |
Journal of Beijing University of Aeronautics and | |
Astronautics,2019,45(12):2345-2350. | |
34 | Hascoet T,Ariki Y,Takiguchi T. Semantic embeddings of generic objects for zero?shot learning. EURASIP Journal on Image and Video Processing,2019:Article number:13. |
35 | Lázaro?Gredilla M,Lin D H,Guntupalli J S,et al. Beyond imitation:Zero?shot task transfer on robots by learning concepts as cognitive programs. Science Robotics,2019,4(26):eaav3150. |
36 | 巩萍,程玉虎,王雪松. 基于属性关系图正则化特征选择的零样本分类. 中国矿业大学学报,2015,44(6):1097-1104. |
Gong P,Cheng Y H,Wang X S. Zero?shot classification based on attribute correlation graph regularized feature selection. Journal of China University of Mining and Technology,2015,44(6):1097-1104. | |
37 | 徐戈,肖永强,汪涛等. 基于视觉误差与语义属性的零样本图像分类. 计算机应用,2020,40(4):1016-1022. (Xu G,Xiao Y Q,Wang T,et al. Zero?shot |
image classification based on visual error and semantic attributes. Journal of Computer Applications,2020,40(4):1016-1022. | |
38 | 林武旭,成科扬,张建明. 基于属性学习的图像分类研究. 计算机科学,2014,41(5):288-291. (Lin W X,Cheng K Y,Zhang J M. Research on image classification based on attribute learning. Computer |
Science,2014,41(5):288-291. | |
39 | 张旗. 基于属性的图像分类研究. 博士学位论文. 大连:大连海事大学,2004. |
Zhang Q. Characteris?tics?based image classification. Ph.D. Dissertation. Dalian:Dalian Maritime University,2004. | |
40 | Yu Y L,Ji Z,Guo J C,et al. Zero?shot learning via latent space encoding. IEEE Transactions on Cybernetics,2018,49(10):3755-3766. |
41 | 潘兴会. 基于语义属性的零样本图像分类. 硕士学位论文. 徐州:中国矿业大学,2017. (Pan X H. |
Zero?shot image classification based on semantic attribute. Master Dissertation. Xuzhou:China University of Mining and Technology,2017. | |
42 | Socher R,Ganjoo M,Manning C D,et al. Zero?shot learning through cross?modal transfer∥Proceedings of the 26th International Conference on Neural Information Processing Systems. Stateline,NV,USA:Curran Associates Inc.,2013:935-943. |
43 | Norouzi M,Mikolov T,Bengio S,et al. Zero?shot learning by convex combination of semantic embeddings. 2014,arXiv:. |
44 | Dauphin Y N,Tur G,Hakkani?Tur D,et al. Zero?shot learning for semantic utterance classification. 2014,arXiv:. |
45 | Fu Z Y,Xiang T,Gong S G. Semantic graph for zero?shot learning. 2015,arXiv:. |
46 | Fu Y W,Hospedales T M,Xiang T,et al. Transductive multi?view zero?shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(11):2332-2345. |
47 | Xu X,Hospedales T,Gong S G. Semantic embedding space for zero?shot action recognition∥2015 IEEE International Conference on Image Processing. Quebec City,Canada:IEEE,2015:63-67. |
48 | Changpinyo S,Chao W L,Gong B Q,et al. Synthesized classifiers for zero?shot learning∥2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas,NV,USA:IEEE,2016:5327-5336. |
49 | Jiang H J,Wang R P,Shan S G,et al. Transferable contrastive network for generalized zero?shot learning∥2019 IEEE/CVF International Conference on Computer Vision. Seoul,Korea (South):IEEE,2019:9764-9773. |
50 | Biswas S,Annadani Y. Preserving semantic relations for zero?shot learning∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,UT,USA:IEEE,2018:7603-7612. |
51 | Lampert C H,Nickisch H,Harmeling S. Attribute?based classification for zero?shot visual object categorization. IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(3):453-465. |
52 | Mishra A,Pandey A,Murthy H A. Zero?shot learning for action recognition using synthesized features. Neurocomputing,2020,390:117-130. |
53 | Jetley S,Romera?Paredes B,Jayasumana S,et al. Prototypical priors:From improving classification to zero?shot learning. 2018,arXiv:. |
54 | Sung F,Yang Y X,Zhang L,et al. Learning to compare:Relation network for few?shot learning∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,UT,USA:IEEE,2018:1199-1208. |
55 | Wang X S,Chen C,Chen Y H,et al. Zero?shot learning based on deep weighted attribute prediction. IEEE Transactions on Systems,Man,and Cybernetics:Systems,2020,50(8):2948-2957. |
56 | Sylvain T,Petrini L,Hjelm D. Locality and compositionality in zero?shot learning. 2019,arXiv:. |
57 | Oreshkin B N,Carpov D,Chapados N,et al. Meta?learning framework with applications to zero?shot time?series forecasting. arXiv:,2020. |
58 | Qiao R Z,Liu L Q,Shen C H,et al. Less is more:Zero?shot learning from online textual documents with noise suppression∥2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas,NV,USA:IEEE,2016:2249-2257. |
59 | Yu Y L,Ji Z,Guo J C,et al. Transductive zero?shot learning with adaptive structural embedding. IEEE Transactions on Neural Networks and Learning Systems,2018,29(9):4116-4127. |
60 | Ji Z,Sun Y X,Yu Y L,et al. Semantic softmax loss for zero?shot learning. Neurocomputing,2018,319:369-375. |
61 | 程玉虎,乔雪,王雪松. 基于混合属性的零样本图像分类. 电子学报,2017,45(6):1462-1468. |
Cheng Y H,Qiao X,Wang X S. Hybrid attribute?based zero?shot image classification. Acta Electronica Sinica,2017,45(6):1462-1468. | |
62 | Xu X,Hospedales T,Gong S G. Transductive zero?shot action recognition by word?vector embedding. International Journal of Computer Vision,2017,123(3):309-333. |
63 | Yu Y L,Ji Z,Li X,et al. Transductive zero?shot learning with a self?training dictionary approach. IEEE Transactions on Cybernetics,2018,48(10):2908-2919. |
64 | Zhu Y Z,Xie J W,Liu B C,et al. Learning feature?to?feature translator by alternating back?propagation for generative zero?shot learning∥2019 IEEE/CVF International Conference on Computer Vision. Seoul,Korea (South):IEEE,2019:9843-9853. |
65 | Zhang J Y,Lv F M,Yang G W,et al. Learning cross?domain semantic?visual relation for transductive zero?shot learning. 2020,arXiv:. |
66 | Ji Z,Chen K X,Wang J Y,et al. Multi?modal generative adversarial network for zero?shot learning. Knowledge?Based Systems,2020,197:105847. |
67 | Jayaraman D,Grauman K. Zero?shot recognition with unreliable attributes∥Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge,MA,USA:MIT Press,2014:3464-3472. |
68 | Tang C W,Yang X,Lv J C,et al. Zero?shot learning by mutual information estimation and maximization. Knowledge?Based Systems,2020(194):105490. |
69 | 杨晨曦,左劼,孙频捷. 基于自编码器的零样本学习方法研究进展. 现代计算机,2020(1):48-52. |
Yang C X,Zuo J,Sun P J. Research progress of | |
zero?shot learning method based on AutoEncoder. Modern Computer,2020(1):48-52. | |
70 | Liu H,Yao L,Zheng Q H,et al. Dual?stream generative adversarial networks for distributionally robust zero?shot learning. Information Sciences,2020(519):407-422. |
71 | 刘欢,郑庆华,罗敏楠等. 基于跨域对抗学习的零样本分类. 计算机研究与发展,2019,56(12):2519-2535. (Liu H,Zheng Q H,Luo M N,et al. Cross? |
domain adversarial learning for zero?shot classification. Journal of Computer Research and Development,2019,56(12):2519-2535. | |
72 | 马保雷,宋颖惠,刘亚维. 基于概念漂移检测的自适应流量识别的研究. 智能计算机与应用,2013,3(6):50-53,56. |
Ma B L,Song Y H,Liu Y W. Research on adaptive traffic identification based on concept drifting detection. Intelligent Computer and Applications,2013,3(6):50-53,56. | |
73 | 文益民,强保华,范志刚. 概念漂移数据流分类研究综述. 智能系统学报,2013(2):95-104. |
Wen Y M,Qiang B H,Fan Z G. A survey of the classification of data streams with concept drift. CAAI Transactions on Intelligent Systems,2013(2):95-104. | |
74 | 尹志武,黄上腾. 一种自适应局部概念漂移的数据流分类算法. 计算机科学,2008,35(2):138-139,143. |
Yin Z W,Huang S T. Adaptive method for handling local concept drift of data streams classification. Computer Science,2008,35(2):138-139,143. | |
75 | 魏晓聪,林鸿飞. 面向迁移学习的文本特征对齐算法. 计算机工程,2017,43(2):215-219,226. |
Wei X C,Lin H F. Transfer learning oriented text feature alignment algorithm. Computer Engineering,2017,43(2):215-219,226. | |
76 | 葛伟,朱金福,吴薇薇等. 基于无容量限制的p?枢纽中位问题的随机优化. 系统工程理论与实践,2013,33(10):2674-2678. |
Ge W,Zhu J F,Wu W W,et al. Stochastic optimization for uncapacitated p?hub median problems. Systems Engineering:Theory and Practice,2013,33(10):2674-2678. | |
77 | 翟晓燕,张新政. 有向网络中具有一个枢纽点的最小支撑树的计算方法. 系统科学与数学,2005,25(6):649-657. |
Zhai X Y,Zhang X Z. An algorithm of the minimum spanning tree with a pivotal point in the directed network. Journal of Systems Science and Mathematical Sciences,2005,25(6):649-657. | |
78 | 何超,张玉峰. 融合领域本体的中文文本语义特征提取算法研究. 情报理论与实践,2013,36(9):96-99. |
He C,Zhang Y F. Research on the Chinese text semantic feature extraction algorithm in combination with domain ontology. Information Studies:Theory & Application,2013,36(9):96-99. | |
79 | 张霞,郑逢斌. 基于多层次视觉语义特征融合的图像检索算法. 包装工程,2018,39(19):223-232. |
Zhang X,Zheng F B. The image retrieval algorithm based on multi?level visual semantic feature fusion. Packaging Engineering,2018,39(19):223-232. | |
80 | 付岩,王耀威,王伟强等. SVM用于基于内容的自然图像分类和检索. 计算机学报,2003,26(10):1261-1265. |
Fu Y,Wang Y W,Wang W Q,et al. Content?based natural image classification and retrieval using SVM. Chinese Journal of Computers,2003,26(10):1261-1265. | |
81 | Guadarrama S,Krishnamoorthy N,Malkarnenkar G,et al. YouTube2Text:Recognizing and describing arbitrary activities using semantic hierarchies and |
zero?shot recognition∥2013 IEEE International Conference on Computer Vision. Sydney,Australia:IEEE,2013:2712-2719. | |
82 | 宋闯,赵佳佳,王康等. 面向智能感知的小样本学习研究综述. 航空学报,2020,41(S1):723756-1-723756-14. (Song C,Zhao J J,Wang K,et al. A survey of few shot learning based on intelligent perception. Acta Aeronautica et Astronautica Sinica,2020,41(S1):723756-1-723756-14.) |
83 | 王洁欣. 基于零样本学习的图像分类研究. 硕士学位论文. 上海:上海交通大学,2018. (Wang J X. |
Research on image classification based on zero?shot learning. Master Dissertation. Shanghai:Shanghai | |
Jiaotong University,2018. | |
84 | Zhao D P,Deng S,Zhao L,et al. Overview on artificial intelligence in design of organic rankine cycle. Energy and AI,2020(1):100011. |
85 | Cao Z,Lu J,Cui S,et al. Zero?shot handwritten Chinese character recognition with hierarchical decomposition embedding. Pattern Recognition,2020(107):107488. |
86 | 汪纯燕. 零样本学习方法在图像分类中的研究与应用. 硕士学位论文. 合肥:安徽大学,2020. |
Wang C Y. Research and application of zero?shot learning method in image classification. Master Dissertation. Hefei:Anhui University,2020. | |
87 | 王珏,石纯一. 机器学习研究. 广西师范大学学报(自然科学版),2003,21(2):1-15. (Wang Y,Shi C Y. Investigations on machine learning. Journal of Guangxi Normal University (Natural Science |
Edition),2003,21(2:1-15. | |
88 | Su Y,Ng T,Li Z G,et al. Sparse scattered high performance computing data driven artificial neural networks for multi?dimensional optimization of buoyancy driven heat and mass transfer in porous structures. Chemical Engineering Journal,2020,doi:10.1016/j.cej.2020.125257. |
[1] | 普志方, 陈秀宏. 基于卷积神经网络的细胞核图像分割算法[J]. 南京大学学报(自然科学版), 2021, 57(4): 566-574. |
[2] | 段建设, 崔超然, 宋广乐, 马乐乐, 马玉玲, 尹义龙. 基于多尺度注意力融合的知识追踪方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 591-598. |
[3] | 崔鹤, 刘昆, 瞿晓磊. 基于紫外⁃可见光谱和机器学习方法的溶解性有机质吸附预测模型研究[J]. 南京大学学报(自然科学版), 2021, 57(3): 356-363. |
[4] | 罗金屯, 滕飞, 周亚波, 池茂儒, 张海波. 数据驱动的高速铁路轮轨作用力反演模型[J]. 南京大学学报(自然科学版), 2021, 57(2): 299-308. |
[5] | 曾宪华, 陆宇喆, 童世玥, 徐黎明. 结合马尔科夫场和格拉姆矩阵特征的写实类图像风格迁移[J]. 南京大学学报(自然科学版), 2021, 57(1): 1-9. |
[6] | 余方超, 方贤进, 张又文, 杨高明, 王丽. 增强深度学习中的差分隐私防御机制[J]. 南京大学学报(自然科学版), 2021, 57(1): 10-20. |
[7] | 张萌, 韩冰, 王哲, 尤富生, 李浩然. 基于深度主动学习的甲状腺癌病理图像分类方法[J]. 南京大学学报(自然科学版), 2021, 57(1): 21-28. |
[8] | 李一凡, 朱斐, 凌兴宏, 刘全. 具有窗口结构Bi⁃LSTM网络的心电图QRS波检测方法[J]. 南京大学学报(自然科学版), 2021, 57(1): 42-51. |
[9] | 温玉莲, 林培光. 基于行业背景差异下的金融时间序列预测方法[J]. 南京大学学报(自然科学版), 2021, 57(1): 90-100. |
[10] | 潘越,王骏,李文飞,张建,王炜. 基于卷积神经网络的蛋白质折叠类型最小特征提取[J]. 南京大学学报(自然科学版), 2020, 56(5): 744-753. |
[11] | 朱伟,张帅,辛晓燕,李文飞,王骏,张建,王炜. 结合区域检测和注意力机制的胸片自动定位与识别[J]. 南京大学学报(自然科学版), 2020, 56(4): 591-600. |
[12] | 李康,谢宁,李旭,谭凯. 基于卷积神经网络和几何优化的统计染色体核型分析方法[J]. 南京大学学报(自然科学版), 2020, 56(1): 116-124. |
[13] | 李子龙,周勇,鲍蓉. AdaBoost图像到类距离学习的图像分类方法[J]. 南京大学学报(自然科学版), 2020, 56(1): 51-56. |
[14] | 韩普,刘亦卓,李晓艳. 基于深度学习和多特征融合的中文电子病历实体识别研究[J]. 南京大学学报(自然科学版), 2019, 55(6): 942-951. |
[15] | 张家精,夏巽鹏,陈金兰,倪友聪. 基于张量分解和深度学习的混合推荐算法[J]. 南京大学学报(自然科学版), 2019, 55(6): 952-959. |
|