南京大学学报(自然科学版) ›› 2022, Vol. 58 ›› Issue (2): 336–344.doi: 10.13232/j.cnki.jnju.2022.02.017

• • 上一篇    

基于多尺度特征图像分割的车道线提取方法

汪鹏飞1, 沈庆宏1(), 张维利2, 董文杰2, 陈红梅2   

  1. 1.南京大学电子科学与工程学院，南京，210023
    2.江苏金晓电子信息股份有限公司，南京，210023
  • 收稿日期:2022-01-11 出版日期:2022-03-30 发布日期:2022-04-02
  • 通讯作者: 沈庆宏 E-mail:qhshen@nju.edu.cn
  • 作者简介:E⁃mail：qhshen@nju.edu.cn

Lane extraction method based on multi⁃scale feature image segmentation

Pengfei Wang1, Qinghong Shen1(), Weili Zhang2, Wenjie Dong2, Hongmei Chen2   

  1. 1.School of Electronic Sciences and Engineering, Nanjing University, Nanjing, 210023, China
    2.Genture Electronics Co. , Ltd. , Nanjing, 210023, China
  • Received:2022-01-11 Online:2022-03-30 Published:2022-04-02
  • Contact: Qinghong Shen E-mail:qhshen@nju.edu.cn

RichHTML

656420

PDF (PC)

972

摘要：

识别与检测车道线作为自动驾驶感知周围环境的一环，为自主车辆在众多复杂场景中提供交通数据信息参考.为了提取车道线本身含有的交通语义信息，按照实际含义分为不同类别，提出一种多尺度分辨率特征的图像分割方法提取车道线，生成低分辨特征，同时保持高分辨尺度子网.针对卷积神经网络无法充分探索空间信息的局限，引入全自注意力网络结构改进下采样解码部分，将特征图通过嵌入向量映射完成线性采样，再经由全自注意力网络结构提取空间上下文语义信息，最后对图像进行降采样完成最终的下采样过程.利用滑窗多头注意力机制，解决嵌入向量映射层因划分造成边界上下文语义信息的不连续问题.针对改进的模型采用交并比损失函数进行优化，能够在保持精度的情况下正确识别相应类别，交并比和F1系数分别达到49.36%和63.02%.经实际测试，在遮挡、阴影等复杂场景下的车道线识别也能更加准确，具有更好的鲁棒性.

关键词: 自动驾驶, 车道线检测, 多尺度分辨率, 图像分割

Abstract:

Lane recognization and detection is a part of autonomous driving to perceive the surrounding environment，and provides traffic data information reference for autonomous vehicles in more complex surroundings. To extract the traffic semantic information contained in the lane itself，it is divided into different categories according to the actual meaning，and a multi?scale resolution feature map image segmentation method is proposed to extract the lanes，which generates a low?resolution features map while maintaining high?resolution scales net. Aiming at the limitation of the convolutional neural network that cannot fully explore the spatial information，Transformer is introduced to improve the down?sampling for the decoding part. The feature map is converted to linear sampling through embedding，and then Transfomer is performed to extract the spatial context semantic information. Finally，the completion is completed down?sampling through patch merging. The shifted window multi?head self?attention is used to solve the problem of discontinuity and limitation of boundary context semantic information，which is caused by the embedding partition. The IoU (Intersection over Union) loss function is used to optimize the improved model，which can correctly identify the corresponding category while maintaining accuracy. The IoU results and F1 coefficients reach 49.36% and 63.02%，respectively. In the actual test，lane detection in complex scenes such as occlusion and shadow can be more accurate and has better robustness.

Key words: autonomous driving, lane detection, multi?scale resolution, image segmentation

中图分类号: 

  • TP391

图 1

HRNet 网络结构示意图"

图2

Transformer 和MSA的传播过程"

图3

模型的编解码结构过程"

图4

用Transformer 替换ConvBlock的结构"

图5

Ehualu 数据集"

图6

模型训练的前期过程"

表1

使用不同的损失函数对实验结果的影响"

MIoUAccuracyF1?score
HRNet (C)37.83%97.98%53.62%
HRTran (C)39.64%98.52%54.37%
HRNet (C+L)44.01%98.98%57.09%
HRTran (C+L)49.36%99.03%63.02%

表2

不同模型的评估结果"

MIoUAccracyF1?score
UNet++34.82%96.72%52.67%
Deeplabv343.35%98.96%57.82%
HRNet44.01%98.98%57.09%
HRTran49.36%99.03%63.02%

图7

Deeplab，HRNet和改进后的HRTran对不同场景下的车道检测结果"

图8

改进后的HRTran模型对Tusimple数据集的测试结果"

1 Yurtsever E， Lambert J， Carballo A，et al. A survey of autonomous driving：Common practices and emerging technologies. IEEE Access，2020(8)：58443-58469.
2 Xing Y， Lv C， Chen L，et al. Advances in vision?based lane detection：Algorithms，integration，assessment，and perspectives on ACP?based parallel vision. IEEE/CAA Journal of Automatica Sinica，2018，5(3)：645-661.
3 万琴，朱晓林，肖岳平，等. 面向复杂城市交通场景的一种实时车道线检测方法. 计算机测量与控制，2019，27(9)：61-65，122.
Wan Q， Zhu X L， Xiao Y P，et al. A real?time lane detection method for complex urban traffic scenes. Computer Measurement & Control，2019，27(9)：61-65，122.
4 Liang D， Guo Y C， Zhang S K，et al. Lane detection：A survey with new results. Journal of Computer Science and Technology，2020，35(3)：493-505.
5 Qin Z Q， Wang H Y， Li X. Ultra fast structure?aware deep lane detection∥The 16th European Conference on Computer Vision. Glasgow，United Kingdom：Springer，2020：276-291.
6 Tabelini L， Berriel R， Paix?o T M，et al. Keep your eyes on the lane：Real?time attention?guided lane detection. 2020，arXiv:.
7 Cai Y F， Luan T Y， Gao H B，et al. YOLOv4?5D：An effective and efficient object detector for autonomous driving. IEEE Transactions on Instrumentation and Measurement，2021(70)：4503613.
8 胡太，杨明. 结合目标检测的小目标语义分割算法. 南京大学学报(自然科学)，2019，55(1)：73-84.
Hu T， Yang M. A small object semantic segmentation algorithm combined with object detection. Journal of Nanjing University (Natural Science)，2019，55(1)：73-84.
9 Xu H， Wang S J， Cai X Y，et al. CurveLane?NAS：Unifying lane?sensitive architecture search and adaptive point blending∥The 16th European Conference on Computer Vision. Glasgow，United Kingdom：Springer，2020：689-704.
10 Pan X G， Shi J P， Luo P，et al. Spatial as deep：Spatial CNN for traffic scene understanding∥Proceedings of the 32nd AAAI Conference on Artificial Intelligence. New Orleans，LA，USA：AAAI，2018：7276-7283.
11 Neven D， Brabandere B D， Georgoulis S，et al. Towards end?to?end lane detection：An instance segmentation approach∥2018 IEEE Intelligent Vehicles Symposium (IV). Changshu，China：IEEE，2018：286-291.
12 Sun K， Xiao B， Liu D，et al. Deep high?resolution representation learning for human pose estimation∥The 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach，CA，USA：IEEE，2019：5686-5696.
13 Vaswani A， Shazeer N， Parmar N，et al. Attention is all you need∥The 31st International Conference on Neural Information Processing Systems. Red Hook，NY，USA：Curran Associates Inc.，2017：6000-6010.
14 Liu Z， Lin YT， Cao Y，et al. Swin transformer：Hierarchical vision transformer using shifted windows. 2021，arXiv:.
15 Kipf T N， Welling M. Semi?supervised classification with graph convolutional networks.2016,arXiv:1609. 02907
16 Fathullah Y， Zhang C， Woodland P C. Improved large?margin softmax loss for speaker diarisation∥2020 IEEE International Conference on Acoustics，Speech and Signal Processing. Barcelona，Spain：IEEE，2020：7104-7108.
17 Berman M， Triki A R， Blaschko M B. The Lovasz?softmax loss：A tractable surrogate for the optimization of the intersection?over?union measure in neural networks∥The 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City，UT，USA：IEEE，2018：4413-4421.
18 陈俊芬，赵佳成，韩洁，等. 基于深度特征表示的Softmax聚类算法. 南京大学学报(自然科学版)，2020，56(4)：533-540.
Chen J F， Zhao J C， Han J，et al. Softmax clustering algorithm based on deep features representation. Journal of Nanjing University (Natural Science)，2020，56(4)：533-540.
19 Milletari F， Navab N， Ahmadi S A. V?Net：Fully convolutional neural networks for volumetric medical image segmentation∥The 4th International Conference on 3D Vision. Stanford，CA，USA：IEEE，2016：565-571.
20 Roberts B， Kaltwang S， Samangooei S，et al. A dataset for lane instance segmentation in urban environments. 2018，arXiv:.
21 Liu Y， Chu L， Chen G，et al. PaddleSeg：A high?efficient development toolkit for image segmentation. 2021，arXiv:.
22 Zhou Z W， Siddiquee M M R， Tajbakhsh N，et al. UNet++：Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging，2020，39(6)：1856-1867.
23 Chen L C， Zhu Y K， Papandreou G，et al. Encoder?decoder with atrous separable convolution for semantic image segmentation∥The 15th European Conference on Computer Vision. Munich，Germany：Springer，2018：833-851.
24 Liu Y B， Zeng M， Meng Q H. Heatmap?based vanishing point boosts lane detection. 2020，arXiv:.
[1] 普志方, 陈秀宏. 基于卷积神经网络的细胞核图像分割算法[J]. 南京大学学报(自然科学版), 2021, 57(4): 566-574.
[2]  唐益明1，2*，赵跟陆1，2，任福继1，2，丰刚永1，2，胡相慧1，2.  图像分割的EMKPFC算法[J]. 南京大学学报(自然科学版), 2017, 53(3): 569-.
[3] 张学良1,2,3，冯学智1,2,3 ，肖鹏峰1,2,3
. 基于区域合并的高分辨率遥感图像多尺度分割[J]. 南京大学学报(自然科学版), 2015, 51(5): 1030-1038.
[4] 胡永月1,2,3，肖鹏峰1,2,3*，冯学智1,2,3，张学良1,2,3，袁　敏1,2,3. 面向对象变化检测中多时相图像分割模式影响评价[J]. 南京大学学报(自然科学版), 2015, 51(5): 1049-1057.
[5] 周淑媛1,2,3，肖鹏峰1,2,3*，冯学智1,2,3，朱榴骏1,2,3，郭金金1,2,3. 基于马尔可夫随机场模型的SAR图像积雪识别[J]. 南京大学学报(自然科学版), 2015, 51(5): 976-986.
[6] 于振洋1*，高尚兵1，唐嵩涛2
. 应用局部结构与方向张量的图像分割算法研究
于振洋1*，高尚兵1，唐嵩涛2
[J]. 南京大学学报(自然科学版), 2015, 51(1): 111-117.
[7]  高尚兵1**，周静波2，严云洋1.  一种新的基于超像素的谱聚类图像分割算法*[J]. 南京大学学报(自然科学版), 2013, 49(2): 169-175.
[8]  纳跃跃，于剑**
.  一种用于谱聚类图像分割的像素相似度计算方法*[J]. 南京大学学报(自然科学版), 2013, 49(2): 159-168.
[9]  李士进**，王万国，谢萍，万定生 .  面向遥感图像分割的自适应特征选择水平集模型*
[J]. 南京大学学报(自然科学版), 2012, 48(4): 482-490.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!

版权所有 © 2021 《南京大学学报(自然科学)》

地址：江苏省南京市鼓楼区汉口路22号，南京大学学报编辑部，210093

电话：025-83592704 , 传真：025-83592704        技术支持：北京玛格泰克科技发展有限公司