南京大学学报(自然科学版) ›› 2021, Vol. 57 ›› Issue (5): 810–817.doi: 10.13232/j.cnki.jnju.2021.05.011

• • 上一篇    

分离表示学习下的严重缺失静脉信息高质量生成

王军, 申政文(), 李玉莲, 潘在宇   

  1. 中国矿业大学信息与控制工程学院，徐州，221116
  • 收稿日期:2021-06-23 出版日期:2021-09-29 发布日期:2021-09-29
  • 通讯作者: 申政文 E-mail:szwfast@163.com
  • 作者简介:E⁃mail：szwfast@163.com
  • 基金资助:
    科技部科技创新2030?“新一代人工智能”重大项目(2020AAA0107300)

Disentangled representation learning network for high⁃quality vein image inpainting

Jun Wang, Zhengwen Shen(), Yulian Li, Zaiyu Pan   

  1. College of Information and Control Engineering，China University of Mining and Technology，Xuzhou，221116，China
  • Received:2021-06-23 Online:2021-09-29 Published:2021-09-29
  • Contact: Zhengwen Shen E-mail:szwfast@163.com

RichHTML

1

PDF (PC)

535

摘要：

为解决在识别过程中存在手背静脉图像信息严重缺失而造成识别效率低下的问题，提出基于分离表示学习严重缺失手背静脉图像的修复算法.基于图像到图像转换的互信息估计表示学习的原理，通过一个共享属性部分编码网络和一个独占属性部分的编码网络来进行特征信息的分离表示，学习静脉关键点与完整静脉骨架图像之间的映射，进而实现基于部分关键点对静脉严重缺失图像的良好修复.为保证生成图像的质量，采用对抗损失与感知损失保证图像的语义真实性与信息完整性，采用循环一致性损失对分离表示网络得到的分离内容和属性表示的循环重建进行约束.实验结果表明，生成图像在视觉效果、峰值信噪比（Peak Signal to Noise Ratio，PSNR）、结构相似性（Structural Similarity Index，SSIM）等方面的表现优于经典算法，有效地实现了对严重缺失静脉图像的良好修复.

关键词: 手背静脉图像, 图像修复, 图像转换, 分离表示学习, 循环一致性损失

Abstract:

To establish a stable and effective hand vein image identification system and solve the problem of low recognition efficiency caused by the serious lack of hand vein image information in the recognition process，it is necessary to inpainting the missing information of hand?dorsal vein image. Based on the disentangled representation learning of mutual information estimation in the image?to?image translation，we propose a method that uses a shared part coding network and an exclusive attribute part coding network to separate representation information of image features，and learn the mapping between the key points and the complete skeleton of vein image to achieve good inpainting of the seriously missing vein image based on some key points. To ensure the quality of the generated image，the adversarial loss and perceptual loss are used to ensure the semantic authenticity and information integrity of the image，and cycle consistency loss is used to constrain the separated content and attribute representation of the separated representation network. The experimental results show that the visual effect，Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) of the generated image is better than the existing classical algorithms，which effectively tests the good inpainting result of the seriously missing vein image.

Key words: hand?dorsal vein image, image inpainting, image?to?image translation, disentangled representation learning, cycle consistency loss

中图分类号: 

  • TP391

图1

手背静脉图像"

图2

严重缺失的手背静脉图像"

图3

严重缺失静脉图像修复的示意图"

图4

分离表示学习网络的整体框架"

表1

编码与生成网络参数"

卷积核步长输出
Conv17×71×164
Conv23×32×2128
Conv33×32×2256
Resblock13×31×1512
3×31×1512
3×31×1512
3×31×1512
Resblock23×31×1512
3×31×1512
3×31×1512
3×31×1512
Conv43×32×2256
Conv53×32×2128
Conv61×11×13

表2

判别网络参数"

卷积核步长输出
Conv13×32×264
Conv23×32×2128
Conv33×32×2256
Conv43×32×2512
Conv53×32×21024
Sigmoid2×21×11

图5

编码网络与生成网络结构"

图6

判别器网络结构"

图7

手背静脉的原始图像（上）和分割图像（下）"

图8

手背静脉的原始图像（上）和加权特征热力图（下）"

图9

手背静脉的原始图像（上）和选取的关键点（下）"

表3

不同损失函数对网络的影响"

损失PSNRSSIM
Add KL?loss24.470.858
Add perceptual loss25.120.865
KL?loss+perceptual loss25.700.871

图10

本文算法与几种经典的图像转换算法的实验结果对比"

表4

不同算法的PSNR与SSIM"

算法PSNRSSIM
CycleGAN5.0700.366
DistanceGAN15.760.626
UNIT22.860.823
本文算法25.700.871

图11

手背静脉修复图像结果"

表5

本文算法修复静脉图像的PSNR与SSIM"

图像PSNRSSIM
静脉修复图像14.630.767
1 曾军英，谌瑶，秦传波等. 级联优化CNN的手指静脉图像质量评估. 中国图像图形学报，2019，24(6)：902-913.
Zeng J Y，Chen Y，Qin C B，et al. Finger vein image quality assessment based on cascaded fine?tuning convolutional neural network. Chinese Journal of Image Graphics，2019，24(6)：902-913.
2 Joshi V S. Analysis of retinal vessel networks using quantitative descriptors of vascular morphology. Ph.D. Dissertation. Iowa：University of Iowa，2012：150-164.
3 Al?Diri B，Hunter A，Steel D，et al. Joining retinal vessel segments∥2008 8th IEEE International Conference on Bioinformatics and Bioengineering. Athens，Greece：IEEE,2008：1-6.
4 Calivá F，Hunter A，Chudzik P，et al. A fluid?dynamic based approach to reconnect the retinal vessels in fundus photography∥2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Jeju，Korea (South)：IEEE，2017：360-364.
5 李振娟，贾桂敏，杨金锋等. 手指静脉图像血管网分形修复方法. 信号处理，2019，35(8)：1334-1342.
Li Z J，Jia G M，Yang J F，et al. Vascular network restoration method for finger?vein images based on fractal. Journal of Signal Processing，2019，35(8)：1334-1342.
6 Isola P，Zhu J Y，Zhou T，et al. Image?to?image translation with conditional adversarial networks∥2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu，HI，USA：IEEE，2017：5967-5976.
7 Zhu J Y，Park T，Isola P，et al. Unpaired image?to?image translation using cycle?consistent adversarial networks∥2017 IEEE International Conference on Computer Vision. Venice,Italy：IEEE，2017：2242-2251.
8 Kingma D P，Welling M. Auto?encoding variational bayes. 2013，arXiv：.
9 Higgins I，Matthey L，Pal A，et al. beta?VAE：Learning basic visual concepts with a constrained variational framework∥The 5th International Conference on Learning Representations. Toulon，France：OpenReview.net，2017.
10 Chen X，Duan Y，Houthooft R，et al. InfoGAN：Interpretable representation learning by information maximizing generative adversarial nets∥The 30th Conference on Neural Information Processing Systems. Barcelona，Spain：MIT Press，2016：2180-2188.
11 Gonzalez?Garcia A，Van De Weijer J，Bengio Y. Image?to?image translation for cross?domain disentanglement. 2018，arXiv：.
12 Tran L，Yin X，Liu X M. Disentangled representation learning GAN for pose?invariant face recognition∥2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu，HI，USA：IEEE，2017：1283-1292.
13 Bao J M，Chen D，Wen F，et al. Towards open?set identity preserving face synthesis∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City，UT，USA：IEEE，2018：6713-6722.
14 Liu Y，Wei F Y，Shao J，et al. Exploring disentangled feature representation beyond face identification∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City，UT，USA：IEEE，2018：2080-2089.
15 Zhu J Y，Zhang R，Pathak D，et al. Toward multimodal image?to?image translation∥The 31st Annual Conference on Neural Information Processing Systems. Long Beach，CA，USA：MIT Press,2017：466-477.
16 Bao J M，Chen D，Wen F，et al. CVAE?GAN：Fine?grained image generation through asymmetric training∥2017 IEEE International Conference on Computer Vision. Venice，Italy：IEEE，2017：2764-2773.
17 Liu M Y，Breuel T，Kautz J. Unsupervised image?to?image translation networks. 2017，arXiv：.
18 Simonyan K，Zisserman A. Very deep convolutional networks for large?scale image recognition. 2014，arXiv：.
19 Pan Z Y，Wang J，Shen Z W，et al. Multi?layer convolutional features concatenation with semantic feature selector for vein recognition. IEEE Access，2019(7)：90608-90619.
20 Benaim S，Wolf L. One?sided unsupervised domain mapping. 2017，arXiv：.
[1] 张礼, 王嘉瑞. 基于图同构网络的自闭症功能磁共振影像诊断算法[J]. 南京大学学报(自然科学版), 2021, 57(5): 801-809.
[2] 张强, 杨吉斌, 张雄伟, 曹铁勇, 梅鹏程. 基于生成对抗网络的音频目标分类对抗[J]. 南京大学学报(自然科学版), 2021, 57(5): 793-800.
[3] 李娜, 段友祥, 孙歧峰, 沈楠. 一种基于样本点距离突变的聚类方法[J]. 南京大学学报(自然科学版), 2021, 57(5): 775-784.
[4] 倪斌, 陆晓蕾, 童逸琦, 马涛, 曾志贤. 胶囊神经网络在期刊文本分类中的应用[J]. 南京大学学报(自然科学版), 2021, 57(5): 750-756.
[5] 陈磊, 孙权森, 王凡海. 基于深度对抗网络和局部模糊探测的目标运动去模糊[J]. 南京大学学报(自然科学版), 2021, 57(5): 735-749.
[6] 卢锦亮, 吴广潮, 冯夫健, 王林. 基于联合轨迹特征的徘徊行为识别方法[J]. 南京大学学报(自然科学版), 2021, 57(5): 724-734.
[7] 薛峰, 李凡, 李爽, 李华锋. 基于域分离和对抗学习的跨域行人重识别[J]. 南京大学学报(自然科学版), 2021, 57(5): 715-723.
[8] 贾霄, 郭顺心, 赵红. 基于图像属性的零样本分类方法综述[J]. 南京大学学报(自然科学版), 2021, 57(4): 531-543.
[9] 吴礼福, 徐行. 融合韵律与动态倒谱特征的语音疲劳度检测[J]. 南京大学学报(自然科学版), 2021, 57(4): 709-714.
[10] 颜志良, 丰智鹏, 刘丹, 王会青. 一种混合深度神经网络的赖氨酸乙酰化位点预测方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 627-640.
[11] 王发旺, 陈睿, 伏云发. 基于DBN和RF的跨被试情绪识别研究[J]. 南京大学学报(自然科学版), 2021, 57(4): 617-626.
[12] 段建设, 崔超然, 宋广乐, 马乐乐, 马玉玲, 尹义龙. 基于多尺度注意力融合的知识追踪方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 591-598.
[13] 杜淑颖, 侯海薇, 丁世飞. 基于多层次特征的深度集成聚类算法[J]. 南京大学学报(自然科学版), 2021, 57(4): 575-581.
[14] 黄鹤, 吴琨, 宋京, 王会峰, 茹锋, 郭璐. 融合全局与区域大气光值图的暗通道图像去雾方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 551-565.
[15] 乔宇鑫, 葛洪伟. 自适应样本加权的多视图聚类算法[J]. 南京大学学报(自然科学版), 2021, 57(4): 544-550.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!

版权所有 © 2021 《南京大学学报(自然科学)》

地址：江苏省南京市鼓楼区汉口路22号，南京大学学报编辑部，210093

电话：025-83592704 , 传真：025-83592704        技术支持：北京玛格泰克科技发展有限公司