基于编码解码结构的移动端电力线语义分割方法

doi:10.11772/j.issn.1001-9081.2020122037

计算机应用 ›› 2021, Vol. 41 ›› Issue (10): 2952-2958.DOI: 10.11772/j.issn.1001-9081.2020122037

所属专题：多媒体计算与计算机仿真

• 多媒体计算与计算机仿真 • 上一篇下一篇

基于编码解码结构的移动端电力线语义分割方法

黄巨挺, 高宏力, 戴志坤

西南交通大学机械工程学院, 成都 610031

收稿日期:2020-12-25 修回日期:2021-03-21 出版日期:2021-10-10 发布日期:2021-07-14
通讯作者: 高宏力
作者简介:黄巨挺(1996-),男,浙江温州人,硕士研究生,主要研究方向:深度学习、机器视觉;高宏力(1971-),男,河南洛阳人,教授,博士,主要研究方向:智能化状态监测、故障诊断;戴志坤(1996-),男,浙江台州人,硕士研究生,主要研究方向:图像处理、图像分割。
基金资助:
国家自然科学基金资助项目（61801402，51775452）。

Semantic segmentation method of power line on mobile terminals based on encoder-decoder structure

HUANG Juting, GAO Hongli, DAI Zhikun

School of Mechanical Engineering, Southwest Jiaotong University, Chengdu Sichuan 610031, China

Received:2020-12-25 Revised:2021-03-21 Online:2021-10-10 Published:2021-07-14
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61801402, 51775452).

摘要/Abstract

摘要： 针对传统的视觉算法在复杂场景下检测细长电力线准确率低、受环境因素干扰大，现有基于深度学习的电线检测算法效率不高的问题，提出一种适用于移动端电力线检测的端到端全卷积神经网络模型。首先，采用一种对称的编码-解码结构，其中编码部分使用最大池化层进行下采样，以提取多尺度特征，而解码部分使用最大池化索引的非线性上采样方式逐层融合多尺度特征，以恢复图像细节；其次，针对电线像素与背景像素不平衡的问题，采用了一种加权损失函数来训练模型；最后，构建了一个背景复杂且有像素级标注的电线数据集来训练和评估模型，并重新标注了一个公开电线数据集作为不同源测试集。与现有移动端电线语义分割模型Dilated ConvNet相比，所提模型在移动端设备GPU NVIDIA JetsonTX2上对于512×512分辨率的图片的预测速度提升至Dilated ConvNet的两倍，达到8.2 frame/s所提模型在同源测试集上的平均交并比（mIoU）为0.857 3，F1分数为0.844 7，平均精度（AP）为0.927 9，这三个指标分别提升了0.011、0.014和0.008；所提模型在公开测试集上的mIoU达到0.724 4，F1分数达到0.634 1，AP达到0.664 4，这三个指标分别提升了0.004、0.007和0.032。实验结果表明，该模型具有更好的移动端电力线实时分割性能。

关键词: 深度学习, 卷积神经网络, 语义分割, 无人机, 输电线路

Abstract: The traditional vision algorithms have low accuracy and are greatly affected by environmental factors during the detection of long and slender power lines in complex scenes, and the existing power line detection algorithms based on deep learning are not efficient. In order to solve the problems, an end-to-end fully convolutional neural network model was proposed which was suitable for power line detection on mobile terminals. Firstly, a symmetrical encoder-decoder structure was adopted. In the encoder part, the max-pooling layer was used for down-sampling, so as to extract multi-scale features. In the decoder part, the max-pooling indices based non-linear up-sampling was used to fuse multi-scale features layer by layer to restore the image details. Then, a weighted loss function was adopted to train the model, thereby solving the imbalance problem between power line pixels and background pixels. Finally, a power line dataset with complex background and pixel-level labels was constructed to train and evaluate the model, and a public power line dataset was relabeled as a different source test set. Compared with a model named Dilated ConvNet for power line semantic segmentation on mobile devices, the proposed model has the prediction speed for 512×512 resolution images on the mobile device GPU NVIDIA JetsonTX2 twice that of Dilated ConvNet, which is 8.2 frame/s; the proposed model achieves a mean Intersection over Union (mIoU) of 0.857 3, F1 score of 0.844 7, Average Precision (AP) of 0.927 9 on the same source test set, which are increased by 0.011, 0.014 and 0.008 respectively; and the proposed model achieves mIoU of 0.724 4, F1 score of 0.634 1, AP of 0.664 4 on the public test set, which are increased by 0.004, 0.007 and 0.032 respectively. Experimental results show that the proposed model has better performance of real-time power line segmentation on mobile terminals.

Key words: deep learning, Convolutional Neural Network (CNN), semantic segmentation, Unmanned Aerial Vehicle (UAV), power transmission line

中图分类号:

TP242.6

黄巨挺, 高宏力, 戴志坤. 基于编码解码结构的移动端电力线语义分割方法[J]. 计算机应用, 2021, 41(10): 2952-2958.

HUANG Juting, GAO Hongli, DAI Zhikun. Semantic segmentation method of power line on mobile terminals based on encoder-decoder structure[J]. Journal of Computer Applications, 2021, 41(10): 2952-2958.

参考文献

[1] AVIZONIS P, BARRON B. Low cost wire detection system[C]//Proceedings of the 18th Digital Avionics Systems Conference. Piscataway:IEEE, 1999:3. C. 3.
[2] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural network[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge:MIT Press, 2015:91-99.
[3] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10)[2020-11-20]. https://arxiv.org/pdf/1409.1556.pdf.
[4] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:770-778.
[5] REDMON J, FARHADI A. YOLOv3:an incremental improvement[EB/OL]. (2018-04-08)[2020-11-20]. https://arxiv.org/pdf/1804.02767.pdf.
[6] LIU W, ANGUELOV D, ERHAN D, et al. SSD:single shot multibox detector[C]//Proceedings of the 2016 European Conference on Computer Vision, LNCS 9905. Cham:Springer, 2016:21-37.
[7] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2015:1331-1440.
[8] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[9] CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the 2018 European Conference on Computer Vision, LNCS 11211. Cham:Springer, 2018:833-851.
[10] ZHU L L, CAO W R, HAN J D, et al. A double-side filter based power line recognition method for UAV vision system[C]//Proceedings of the 2013 IEEE International Conference on Robotics and Biomimetics. Piscataway:IEEE, 2013:2655-2660.
[11] DU S Z, VAN WYK B J, TU C L. Heuristic Bayesian pixel classification for power line inspection[C]//Proceedings of the 3rd International Congress on Image and Signal Processing. Piscataway:IEEE, 2010:960-963.
[12] SONG B Q, LI X L. Power line detection from optical images[J]. Neurocomputing, 2014, 129:350-361.
[13] 韦盛. 基于图像处理的输电线识别与六旋翼巡检研究[D]. 合肥:合肥工业大学, 2020:14-29. (WEI S. Research on transmission line identification and helicopter inspection based on image processing[D]. Hefei:Hefei University of Technology, 2020:14-29.)
[14] RONNEBERGER O, FISCHER P, BROX T. U-Net:convolutional networks for biomedical image segmentation[C]//Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention, LNCS 9351. Cham:Springer, 2015:234-241.
[15] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL]. (2016-06-07)[2020-12-29]. https://arxiv.org/pdf/1412.7062.pdf.
[16] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[17] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. (2017-12-05)[2020-12-29]. https://arxiv.org/pdf/1706.05587.pdf.
[18] 王栩文. 基于图像的输电线路语义分割技术研究[D]. 杭州:浙江大学, 2019:29-51.(WANG X W. Research on semantic segmentation of power line based on image[D]. Hangzhou:Zhejiang University, 2019:29-51.)
[19] MADAAN R, MATURANA D, SCHERER S. Wire detection using synthetic data and dilated convolutional networks for unmanned aerial vehicles[C]//Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway:IEEE, 2017:3487-3494.
[20] ZHANG H, YANG W, YU H, et al. Detecting power lines in UAV images with convolutional features and structured constraints[J]. Remote Sensing, 2019, 11(11):No. 1342.
[21] CHOI H, KOO G, KIM B J, et al. Weakly supervised power line detection algorithm using a recursive noisy label update with refined broken line segments[J]. Expert Systems with Applications, 2021, 165:No. 113895.
[22] YETGIN Ö E, BENLIGIRAY B, GEREK Ö N. Power line recognition from aerial images with deep learning[J]. IEEE Transactions on Aerospace and Electronic Systems, 2019, 55(5):2241-2252.
[23] RUSSELL B C, TORRALBA A, MURPHY K P, et al. LabelMe:a database and web-based tool for image annotation[J]. International Journal of Computer Vision, 2008, 77(1/2/3):157-173.
[24] IOFFE S, SZEGEDY C. Batch normalization:accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning. New York:JMLR. org, 2015:448-456.
[25] NAIR V, HINTON G E. Rectified linear units improve restricted Boltzmann machines[C]//Proceedings of the 27th International Conference on Machine Learning. Madison, WI:Omnipress, 2010:807-814.
[26] BADRINARAYANAN V, HANDA A, CIPOLLA R. SegNet:a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling[EB/OL]. (2015-05-27)[2020-12-20]. https://arxiv.org/pdf/1505.07293.pdf.
[27] EIGEN D, FERGUS R. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:2650-2658.

基于编码解码结构的移动端电力线语义分割方法

Semantic segmentation method of power line on mobile terminals based on encoder-decoder structure

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[2]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[3]	宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.
[4]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[5]	张永斌, 常文欣, 孙连山, 张航. 基于字典的域名生成算法生成域名的检测方法[J]. 计算机应用, 2021, 41(9): 2609-2614.
[6]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[7]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[8]	牟长宁, 王海鹏, 周丕宇, 侯鑫行. 基于图卷积神经网络的串联质谱从头测序[J]. 计算机应用, 2021, 41(9): 2773-2779.
[9]	王贺兵, 张春梅. 基于非对称卷积-压缩激发-次代残差网络的人脸关键点检测[J]. 计算机应用, 2021, 41(9): 2741-2747.
[10]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[11]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[12]	何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431.
[13]	曾祥银, 郑伯川, 刘丹. 基于深度卷积神经网络和聚类的左右轨道线检测[J]. 计算机应用, 2021, 41(8): 2324-2329.
[14]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[15]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.