Review of image edge detection algorithms based on deep learning

doi:10.11772/j.issn.1001-9081.2020030314

Abstract

Abstract: Edge detection is the process of extracting the important information of mutations in the image. It is a research hotspot in the field of computer vision and the basis of many middle-and high-level vision tasks such as image segmentation, target detection and recognition. In recent years, in view of the problems of thick edge contour lines and low detection accuracy, edge detection algorithms based on deep learning such as spectral clustering, multi-scale fusion, and cross-layer fusion were proposed by the industry. In order to make more researchers understand the research status of edge detection, firstly, the implementation theory and methods of traditional edge detection were introduced. Then, the main edge detection methods based on deep learning in resent years were summarized, and these methods were classified according to the implementation technologies of the methods. And the analysis of the key technologies of these methods show that the multi-scale multi-level fusion and selection of loss function was the important research directions. Various methods were compared to each other through evaluation indicators. It can be seen that the Optimal Dataset Scale (ODS) of edge detection algorithm on the Berkeley Segmentation Data Set and benchmark 500 (BSDS500) was increased from 0.598 to 0.828, which was close to the level of human vision. Finally, the development direction of edge detection algorithm research was forecasted.

Key words: edge detection, deep learning, Convolutional Neural Network (CNN), loss function, multi-scale fusion

摘要： 边缘检测是将图像中的突变的重要信息提取出来的过程，是计算机视觉领域研究热点，也是图像分割、目标检测与识别等多种中高层视觉任务的基础。近几年来，针对边缘轮廓线过粗以及检测精度不高等问题，业内提出了谱聚类、多尺度融合、跨层融合等基于深度学习的边缘检测算法。为了使更多研究者了解边缘检测的研究现状，首先，介绍了传统边缘检测的实现理论及方法；然后，总结了近年来基于深度学习的主要边缘检测方法，并依据实现技术对这些方法进行了分类，对其涉及的关键技术进行分析，发现对多尺度多层次融合与损失函数的选择是重要的研究方向。通过评价指标对各类方法进行了比较，可知边缘检测算法在伯克利大学数据集（BSDS500）上的最优数据集规模（ODS）经过多年研究从0.598提高到了0.828，接近人类视觉水平。最后，展示了边缘检测算法研究的发展方向。

关键词: 边缘检测, 深度学习, 卷积神经网络, 损失函数, 多尺度融合

CLC Number:

TP391.41

LI Cuijin, QU Zhong. Review of image edge detection algorithms based on deep learning[J]. Journal of Computer Applications, 2020, 40(11): 3280-3288.

李翠锦, 瞿中. 基于深度学习的图像边缘检测算法综述[J]. 《计算机应用》唯一官方网站, 2020, 40(11): 3280-3288.

References

[1] KITTLER J. On the accuracy of the Sobel edge detector[J]. Image and Vision Computing,1983,1(1):37-42.
[2] TORRE V, POGGIO T A. On edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1986, PAMI-8(2):147-163.
[3] CANNY J. Acomputational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1986, PAMI-8(6):679-698.
[4] ARBELÁEZ P,MAIRE M,FOWLKES C,et al. Contour detection and hierarchical image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5):898-916.
[5] REN X,BO L. Discriminatively trained sparse code gradients for contour detection[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook, NY:Curran Associates Inc.,2012:593-601.
[6] DOLLÁR P,ZITNICK C L. Fast edge detection using structured forests[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(8):1558-1570.
[7] ZHENG L, SHEN L, TIAN L, et al. Scalable person reidentification:a benchmark[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:1116-1124.
[8] XIE S,TU Z. Holistically-nested edge detection[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE,2015:1395-1403.
[9] BERTASIUS G,SHI J,TORRESANI L. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE,2015:318-330.
[10] BERTASIUS G,SHI J,TORRESANI L. DeepEdge:a multiscale bifurcated deep network for top-down contour detection[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:4380-4389.
[11] WANG Z,ZHU S,LI Y,et al. Convolutional neural network based deep conditional random fields for stereo matching[J]. Journal of Visual Communication and Image Representation, 2016,40(Pt B):739-750.
[12] YU Z,FENG C,LIU M Y,et al. CASENet:deep category-aware semantic edge detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:1761-1770.
[13] CHADHA A, ANDREOPOULOS Y. Voronoi-basedcompact image descriptors:efficient region-of-interest retrieval with VLAD and deep-learning based descriptors[J]. IEEE Transactions on Multimedia,2017,19(7):1596-1608.
[14] LIU Y,CHENG M,HU X,et al. Richer convolutional features for edge detection[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:5872-5881.
[15] WANG X,MA H,CHEN X,et al. Edge preserving and multiscale contextual neural network for salient object detection[J]. IEEE Transactions on Image Processing,2018,27(1):121-134.
[16] FU K,ZHAO Q,GU I Y H,et al. Refinet:a deep segmentation assisted refinement network for salient object detection[J]. IEEE Transactions on Multimedia,2019,21(2):457-469.
[17] LUO J F,MA J W. Image segmentation with thecompetitive learning based MS model[C]//Proceedings of the 2015 IEEE International Conference on Image Processing. Piscataway:IEEE, 2015:182-186.
[18] COMANICIU D,MEER P. Mean shift:a robust approach toward feature space analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(5):603-619.
[19] WANG Y,ZHAO X,HUANG K. Deep crisp boundaries:from boundaries to higher-level tasks[J]. IEEE Transactions on Image Processing,2019,28(3):1285-1298.
[20] MARR D, HILDRETH E. Theory of edge detection[J]. Proceedings of the Royal Society B:Biological Sciences,1980, 207(1167):187-217.
[21] HARRIS C,STEPHENS M. Acombined corner and edge detector[C]//Proceedings of the 1988 British Machine Vision Conference. Durham:BMVA Press,1988:147-152.
[22] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE,1998,86(11):2278-2324.
[23] TRAJKOVIĆ M,HEDLEY M. Fast corner detection[J]. Image and Vision Computing,1998,16(2):75-87.
[24] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision,2004,60(2):91-110.
[25] REN X. Multi-scale improves boundary detection in natural images[C]//Proceedings of the 2008 European Conference on Computer Vision,LNCS 5304. Berlin:Springer,2008:533-545.
[26] ZHANG Z, XING F, SHI X, et al. SemiContour:a semisupervised learning approach for contour detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:251-259.
[27] DOLLÁR P,ZITNICK C L. Fast edge detection using structured forests[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(8):1558-1570.
[28] PATI Y C,REZAIIFAR R,KRISHNAPRASAD P S. Orthogonal matching pursuit:recursive function approximation with applications to wavelet decomposition[C]//Proceedings of the 27th Asilomar Conference on Signals, Systems and Computers. Piscataway:IEEE,1993:40-44.
[29] AHARON M,ELAD M,BRUCKSTEIN A. K-SVD:an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing,2006,54(11):4311-4322.
[30] MAIRAL J,LEORDEANU M,BACH F,et al. Discriminative sparse image models for class-specific edge detection and image interpretation[C]//Proceedings of the 2008 European Conference on Computer Vision,LNCS 5304. Berlin:Springer,2008:43-56.
[31] SILBERMAN N, HOIEM D, KOHLI P, et al. Indoor segmentation and support inference from RGBD images[C]//Proceedings of the 2012 European Conference on Computer Vision,LNCS 7576. Berlin:Springer,2012:746-760.
[32] HALLMAN S,FOWLKES C C. Oriented edge forests for boundary detection[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2015:1732-1740.
[33] SHEN W,WANG X,WANG Y,et al. DeepContour:a deep convolutional feature learned by positive-sharing loss for contour detection[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2015:3982-3991.
[34] IVAN G D. DermaKNet:Incorporating the knowledge of dermatologists to convolutional neural networks for skin lesion diagnosis[J]. IEEE Journal of Biomedical and Health Informatics, 2019,23(2):547-559.
[35] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2020-01-20]. https://arxiv.org/pdf/1409.1556.pdf.
[36] LIU Y,LEW M S. Learning relaxed deep supervision for better edge detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:231-240.
[37] BERTASIUS,SHI J,TORRESANI L. High-for-low and low-forhigh:efficient boundary detection from deep object features and its applications to high-level vision[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE,2015:504-512.
[38] GANIN Y,LEMPITSKY V. N4-fields:neural network nearest neighbor fields for image transforms[C]//Proceedings of the 12th Asian Conference on Computer Vision, LNCS 9004. Cham:Springer,2015:536-551.
[39] KRIZHEVSKY A,SUTSKEVER I,HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook, NY:Curran Associates Inc.,2012:1097-1105.
[40] MANINIS K K, PONT-TUSET J, ARBELÁEZ P, et al. Convolutional oriented boundaries[C]//Proceedings of the 2016 European Conference on Computer Vision,LNCS 9905. Cham:Springer,2016:580-596.
[41] HE K,ZHANG X,REN S,et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:770-778.
[42] XU D,OUYANG W,ALAMEDA-PINEDA X,et al. Learning deep structured multi-scale features using attention-gated CRFs for contour prediction[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc.,2017:3961-3970.
[43] FUKUSHIMA K. Neocognitron:a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological Cybernetics,1980,36(4):193-202.
[44] DENG R,SHEN C,LIU S,et al. Learning to predict crisp boundaries[C]//Proceedings of the 2018 European Conference on Computer Vision, LNCS 11210. Cham:Springer, 2018:570-586.
[45] HE J,ZHANG S,YANG M,et al. Bi-directional cascade network for perceptual edge detection[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:3823-3832.
[46] QU Z,WANG S,LIU L,et al. Visual cross-image fusion using deep neural networks for image edge detection[J]. IEEE Access, 2019,7:57604-57615.
[47] YANG J,PRICE B,COHEN S,et al. Object contour detection with a fully convolutional encoder-decoder network[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:193-202.
[48] RUMELHART D E, HINTON G, WILLIAMS R. Learning internal representations by error propagation[M]//RUMELHART D E, MCCLELLAND J L, PDP Research Group. Parallel Distributed Processing:Explorations in the Microstructure of Cognition, Volume 1:Foundations. Cambridge:MIT Press, 1986:318-362.
[49] LECUN Y,BOSER B,DENKER J S,et al. Backpropagation applied to handwritten zip code recognition[J]. Neural Computation,1989,1(4):541-551.
[50] BARRON J L,MALIK J. Intrinsic scene properties from a single RGB-D image[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2013:17-24.
[51] DOLLÁR P,TU Z,BELONGIE S. Supervised learning of edges and object boundaries[C]//Proceedings of the 2006 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2006:1964-1971.
[52] DOLLÁR P,ZITNICK C L. Fast edge detection using structured forests[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(8):1558-1570.
[53] CHU X, OUYANG W, LI H, et al. CRF-CNN:modeling structured information in human pose estimation[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc., 2016:1-10.
[54] XU D,RICCI E,OUYANG W,et al. Multi-scale continuous CRFs as sequential deep networks for monocular depth estimation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:161-169.
[55] MINKA T, WINN J. Gates[C]//Proceedings of the 22nd International Conference and Workshop on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc., 2008:1073-1080.
[56] LONG J,SHELHAMER E,DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:3431-3440.
[57] NOH H,HONG S,HAN B. Learning deconvolution network for semantic segmentation[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:1520-1528.
[58] PINHEIRO P O,LIN T Y,COLLOBERT R,et al. Learning to refine object segments[C]//Proceedings of the 2016 European Conference on Computer Vision,LNCS 9905. Cham:Springer, 2016:75-91.
[59] SHI W,CABALLERO J,HUSZÁR F,et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1874-1883.
[60] KHOREVA A, BENENSON R, OMRAN M, et al. Weakly supervised object boundaries[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:183-192.
[61] XIE S,TU Z. Holistically-nested edge detection[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE,2015:1395-1403.
[62] LI Y,PALURI M,REHG J M,et al. Unsupervised learning of edges[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1619-1627.
[63] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE,1998,86(11):2278-2324.
[64] SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2015:1-9.
[65] EVERINGHAM M,ESLAMI S M A,VAN GOOL L,et al. The pascal visual object classes challenge:a retrospective[J]. International Journal of Computer Vision,2015,111(1):98-136.
[66] MOTTAGHI R,CHEN X,LIU X,et al. The role of context for object detection and semantic segmentation in the wild[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2014:891-898.
[67] 苏松志, 李绍滋, 陈淑媛, 等. 行人检测技术综述[J]. 电子学报,2012,40(4):814-820.(SU S Z,LI S Z,CHEN S Y,et al. A survey on pedestrian detection[J]. Acta Electronica Sinica, 2012,40(4):814-820.)