南京大学学报(自然科学版) ›› 2023, Vol. 59 ›› Issue (3): 413–424.doi: 10.13232/j.cnki.jnju.2023.03.005

• • 上一篇    下一篇

基于U2⁃Net的金属表面缺陷检测算法

王昱翔1,2, 葛洪伟1,2()   

  1. 1.江南大学人工智能与计算机学院，无锡，214122
    2.江苏省模式识别与计算智能工程实验室(江南大学), 无锡，214122
  • 收稿日期:2023-03-17 出版日期:2023-05-31 发布日期:2023-06-09
  • 通讯作者: 葛洪伟 E-mail:ghw8601@163.com
  • 基金资助:
    国家自然科学基金(61806006);江苏高校优势学科建设工程

Metal surface defect detection algorithm based on U2⁃Net

Yuxiang Wang1,2, Hongwei Ge1,2()   

  1. 1.School of Artificial Intelligence and Computer Science，Jiangnan University, Wuxi，214122，China
    2.Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence，Jiangnan University，Wuxi，214122，China
  • Received:2023-03-17 Online:2023-05-31 Published:2023-06-09
  • Contact: Hongwei Ge E-mail:ghw8601@163.com

RichHTML

16

PDF (PC)

528

摘要：

金属表面缺陷检测旨在通过合理的算法判断工业生产中金属材料的表面图像中是否存在缺陷，是计算机视觉领域在工业应用中的重要研究内容，但现有的基于分割的金属表面缺陷检测算法存在抗干扰能力弱、容易背景误判和检测粒度不够细致等问题.针对具有各种干扰因子的金属表面图像，提出一种基于U2?Net的金属表面缺陷检测算法Attention?U2Net.首先，为了解决实际生产中金属表面图像噪点过多导致背景误判和采样层获取信息不够细致的问题，设计U型注意力编码模块，可以在编码时增大缺陷区域权重的同时，抑制背景噪声；然后，为了解决图像中存在的难检测样本和边界复杂问题，设计具有权重的损失函数并结合多层次监督方法，使检测算法更关注难检测样本和边界像素点，提高预测准确度；最后，使用根据图像结果自动计算灰度阈值进行结果优化的算法，最终生成高质量缺陷预测图.与六种常用的缺陷检测领域的像素分割方法在金属表面缺陷公开数据集上进行比较实验，实验结果表明，提出的算法的像素准确率、查准率、查全率、F?score、平均绝对误差和均方误差均取得了优良的结果，证明该算法拥有较强的抗干扰因子能力，最终缺陷预测图像素准确率高，边界明显.

关键词: 缺陷检测, 深度学习, 注意力编码, 图像优化, 图像分割

Abstract:

Metal surface defect detection aims to judge whether there are defects in the surface image of metal materials in industrial production through a reasonable algorithm，and is an important research content in the field of computer vision applied in industry. However，the existing segmentation?based metal surface defect detection algorithms still have problems such as weak anti?interference ability，easy background misjudgment and insufficient detection granularity. For metal surface images with various interference factors，this paper proposes a U2?Net?based metal surface defect detection algorithm Attention?U2Net. First of all，in order to solve the problem of misjudgment of background and insufficient information obtained by sampling layer due to excessive noise of metal surface image in actual production，a U?shaped attention coding module is designed for suppressing background noise. Then，in order to solve the complex problems of difficult?to?detect samples and boundaries in images，a weighted loss function is designed and combined with a multi?level supervision method，so that the detection algorithm pays more attention to difficult?to?detect samples and boundary pixels and improves the prediction accuracy. Finally，an algorithm that automatically calculates the grayscale threshold according to the image results is used to optimize the results,and a high?quality defect prediction map is generated finally. By comparing with six pixel segmentation methods commonly used in the field of defect detection，the experimental results on the public data set of metal surface defects show that the algorithm has good performance. The experimental results show that the algorithm has strong anti?interference factor ability，and the final defect prediction image has high pixel accuracy and obvious boundaries.

Key words: defect detection, deep learning, attention coding, image optimization, image segmentation

中图分类号: 

  • TP183

图1

U2?Net的网络结构"

图2

CBAM的结构"

图3

U型空间注意力编码模块的结构"

图4

U型通道注意力编码模块的结构"

图5

金属表面缺陷检测算法Attention?U2Net的网络结构"

图6

解码器模块的结构"

表1

本文方法同其他六种方法在KolektorSDD数据集上的性能比较"

AccPRF1⁃scoreF2⁃scoreMAEMSE
本文方法0.9980.7940.8430.8120.8330.2770.001
U⁃Net0.9770.6820.7890.7320.85010.8150.63
Att⁃Unet0.9870.7620.8120.7590.79010.3630.88
U2⁃Net0.9820.7630.8090.7850.7990.8580.794
Nested⁃UNet0.9890.7370.8230.7780.80410.1010.23

Res⁃

Unet++

0.9930.7740.8050.7890.7980.8890.908
R2U_Net0.9920.7610.8130.7860.80210.0110.28

图7

本文方法与其他方法的可视化对比"

表2

不同模块的消融实验"

USEUCEOTSUAccPRF1⁃scoreF2⁃scoreMAEMSE
√√√0.9980.7940.8430.8120.8330.2770.001
0.9820.7630.8090.7850.7990.8580.794
√0.9870.7650.8230.7920.8110.3700.346
√√0.9920.7900.8200.8050.8140.3470.135

表3

超参数γ的消融实验"

γAccPRF1⁃scoreF2⁃scoreMSEMAE
10.9930.7540.7990.7760.7890.6720.003
20.9980.7940.8430.8120.8330.2770.001
30.9950.8050.7490.7750.7600.3780.005
40.9930.5340.8930.6680.78710.380.013
1 Nand G K，Noopur， Neogi N. Defect detection of steel surface using entropy segmentation∥2014 Annual IEEE India Conference. Pune，India：IEEE，2014：1-6.
2 Song K C， Yan Y H. A noise robust method based on completed local binary patterns for hot?rolled steel strip surface defects. Applied Surface Science，2013(285)：858-864.
3 Ghorai S， Mukherjee A， Gangadaran M，et al. Automatic defect detection on hot?rolled flat steel products. IEEE Transactions on Instrumentation and Measurement，2013，62(3)：612-621.
4 Liu Y， Xu K， Wang D D. Online surface defect identification of cold rolled strips based on local binary pattern and extreme learning machine. Metals，2018，8(3)：197.
5 Niskanen M， Kauppinen H， Silven O. Real?time aspects of SOM?based visual surface inspection∥Proceedings Volume 4664，Machine Vision Applications in Industrial Inspection X. San Jose，CA，USA：SPIE，2002：123-134.
6 Yazdchi M， Yazdi M， Mahyari A G. Steel surface defect detection using texture segmentation based on multifractal dimension∥2009 International Conference on Digital Image Processing. Bangkok，Thailand：IEEE，2009：346-350.
7 Akram M W， Li G W， Jin Y，et al. CNN based automatic detection of photovoltaic cell defects in electroluminescence images. Energy，2019(189)：116319.
8 Shang L D， Yang Q S， Wang J N，et al. Detection of rail surface defects based on CNN image recognition and classification∥2018 20th International Conference on Advanced Communication Technology. Chun?cheon，Korea (South)：IEEE，2018：45-51.
9 Tao X， Zhang D P， Ma W Z，et al. Automatic metallic surface defect detection and recognition with convolutional neural networks. Applied Sciences，2018，8(9)：1575.
10 Zhang J B， Zhang Z T， Su H，et al. Quality inspection based on quadrangular object detection for deep aperture component. IEEE Transactions on Systems，Man，and Cybernetics：Systems，2021，51(10)：5938-5948.
11 Xue Y D， Li Y C. A fast detection method via region‐based fully convolutional neural networks for shield tunnel lining defects. Computer‐Aided Civil and Infrastructure Engineering，2018，33(8)：638-654.
12 Chen J W， Liu Z G， Wang H R，et al. Automatic defect detection of fasteners on the catenary support device using deep convolutional neural network. IEEE Transactions on Instrumentation and Measurement，2018，67(2)：257-269.
13 Aslam Y， Santhi N， Ramasamy N，et al. Localization and segmentation of metal cracks using deep learning. Journal of Ambient Intelligence and Humanized Computing，2021，12(3)：4205-4213.
14 Huang H W， Li Q T， Zhang D M. Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunnelling and Underground Space Technology，2018(77)：166-176.
15 Han H， Gao C Q， Zhao Y，et al. Polycrystalline silicon wafer defect segmentation based on deep convolutional neural networks. Pattern Recognition Letters，2020(130)：234-241.
16 Qin X B， Zhang Z C， Huang C Y，et al. U2?Net：Going deeper with nested U?structure for salient object detection. Pattern Recognition，2020(106)：107404.
17 Ronneberger O， Fischer P， Brox T. U?net：Convolu?tional networks for biomedical image segmentation∥Proceedings of the 18th International Conference on Medical Image Computing and Computer：Assisted Intervention. Springer Berlin Heidelberg，2015：234-241.
18 Hu J， Shen L， Sun G. Squeeze?and?excitation networks∥Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City，UT，USA：IEEE，2018：7132-7141.
19 Badrinarayanan V， Kendall A， Cipolla R. SegNet：A deep convolutional encoder?decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39(12)：2481-2495.
20 Simonyan K， Zisserman A. Very deep convolutional networks for large?scale image recognition∥The 3rd International Conference on Learning Representa?tions. San Diego，CA，USA：ICLR，2014：234-241.
21 He K M， Zhang X Y， Ren S Q，et al. Deep residual learning for image recognition∥Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas，NV，USA：IEEE，2016：770-778.
22 Huang G， Liu Z， Van Der Maaten L，et al. Densely connected convolutional networks∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Honolulu，HI，USA：IEEE，2017：2261-2269.
23 Otsu N. A threshold selection method from gray?level histograms. IEEE Transactions on Systems Man & Cybernetics，2007，9(1)：62-66.
24 Tabernik D， ?ela S， Skvar? J，et al. Segmentation?based deep?learning approach for surface?defect detection. Journal of Intelligent Manufacturing，2020，31(3)：759-776.
25 Oktay O， Schlemper J， Le Folgoc L，et al. Attention U?Net：Learning where to look for the pancreas. 2018，arXiv:.
26 Zhou Z W， Siddiquee M M R， Tajbakhsh N，et al. UNet++：A nested U?Net architecture for medical image segmentation∥The 4th International Workshop on Deep Learning in Medical Image Analysis. Springer Berlin Heidelberg，2018：3-11.
27 Jha D， Smedsrud P H， Riegler M A，et al. ResUNet++：An advanced architecture for medical image segmentation∥2019 IEEE International Symposium on Multimedia. San Diego，CA，USA：IEEE，2019：225-255.
28 Alom M Z， Yakopcic C， Hasan M，et al. Recurrent residual U?Net for medical image segmentation. Journal of Medical Imaging，2019，6(1)：014006.
[1] 谭嘉辰, 董永权, 张国玺. SSM: 基于孪生网络的糖尿病视网膜眼底图像分类模型[J]. 南京大学学报(自然科学版), 2023, 59(3): 425-434.
[2] 王岳宁, 郭成雯, 王育昕, 袁杰, 程茜. 基于反卷积算法的超声图像优化方法[J]. 南京大学学报(自然科学版), 2023, 59(2): 247-255.
[3] 卞苏阳, 严云洋, 龚成张, 冷志超, 祝巧巧. 基于CXANet⁃YOLO的火焰检测方法[J]. 南京大学学报(自然科学版), 2023, 59(2): 295-301.
[4] 林灏昶, 秦云川, 蔡宇辉, 李肯立, 唐卓. 基于目标检测的图形用户界面控件识别方法[J]. 南京大学学报(自然科学版), 2022, 58(6): 1012-1019.
[5] 马学森, 马吉, 蒋功辉, 许雪梅, 周天保. 基于注意力机制和多尺度特征融合的绝缘子缺陷检测方法[J]. 南京大学学报(自然科学版), 2022, 58(6): 1020-1029.
[6] 成科扬, 丁杨柳, 詹永照, 严浏阳. 基于回顾蒸馏学习的无监督工业品缺陷检测方法[J]. 南京大学学报(自然科学版), 2022, 58(6): 1030-1040.
[7] 周佳倩, 林培光, 李庆涛, 王基厚, 刘利达. MDDE：一种基于投资组合的金融市场趋势分析方法[J]. 南京大学学报(自然科学版), 2022, 58(5): 876-883.
[8] 罗思涵, 杨燕. 一种基于深度学习和元学习的出行时间预测方法[J]. 南京大学学报(自然科学版), 2022, 58(4): 561-569.
[9] 杜渊洋, 邓成伟, 张建. 基于深度卷积神经网络的RNA三维结构打分函数[J]. 南京大学学报(自然科学版), 2022, 58(3): 369-376.
[10] 陈轶洲, 刘旭生, 孙林檀, 李文中, 方立兵, 陆桑璐. 基于图神经网络的社交网络影响力预测算法[J]. 南京大学学报(自然科学版), 2022, 58(3): 386-397.
[11] 汪鹏飞, 沈庆宏, 张维利, 董文杰, 陈红梅. 基于多尺度特征图像分割的车道线提取方法[J]. 南京大学学报(自然科学版), 2022, 58(2): 336-344.
[12] 张玮, 赵永虹, 邱桃荣. 基于注意力机制和深度学习的运动想象脑电信号分类方法[J]. 南京大学学报(自然科学版), 2022, 58(1): 29-37.
[13] 孟浩, 刘强. 基于FPGA的卷积神经网络训练加速器设计[J]. 南京大学学报(自然科学版), 2021, 57(6): 1075-1082.
[14] 陈磊, 孙权森, 王凡海. 基于深度对抗网络和局部模糊探测的目标运动去模糊[J]. 南京大学学报(自然科学版), 2021, 57(5): 735-749.
[15] 倪斌, 陆晓蕾, 童逸琦, 马涛, 曾志贤. 胶囊神经网络在期刊文本分类中的应用[J]. 南京大学学报(自然科学版), 2021, 57(5): 750-756.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!

版权所有 © 2021 《南京大学学报(自然科学)》

地址：江苏省南京市鼓楼区汉口路22号，南京大学学报编辑部，210093

电话：025-83592704 , 传真：025-83592704        技术支持：北京玛格泰克科技发展有限公司