Single image shadow detection method based on entropy driven domain adaptive learning
YUAN Yuan1, WU Wen1, WAN Yi2
1. Department of Information Engineering, Xinjiang Institute of Technology, Aksu Xinjiang 843100, China; 2. School of Electrical and Electronic Engineering, Wenzhou University, Wenzhou Zhejiang 325035, China
Abstract：Cross-domain discrepancy frequently hinders deep neural networks to generalize to different datasets. In order to improve the robustness of shadow detection, a novel unsupervised domain adaptive shadow detection framework was proposed. Firstly, in order to reduce the data bias between different domains, a multi-level domain adaptive model was introduced to align the feature distributions of source domain and target domain from low level to high level. Secondly, to improve the model ability of soft shadow detection, a boundary-driven adversarial branch was proposed to guarantee the structured shadow boundary was also able to be obtained by the model on the target dataset. Thirdly, the entropy adversarial branch was combined to further suppress the high uncertainty at shadow boundary of the prediction result, so as to obtain an accurate and smooth shadow mask. Compared with the existing deep learning-based shadow detection methods, the proposed method has the Balance Error Rate (BER) averagely reduced by 10.5% and 18.75% on ISTD dataset and SBU dataset respectively. The experimental results demonstrate that the shadow detection results of the proposed algorithm have better boundary structure.
[1] SUN G,HUANG H,WENG Q,et al. Combinational shadow index for building shadow extraction in urban areas from Sentinel-2A MSI imagery[J]. International Journal of Applied Earth Observation and Geoinformation,2019,78:53-65. [2] CHAI D,NEWSAM S,ZHANG H K,et al. Cloud and cloud shadow detection in Landsat imagery based on deep convolutional neural networks[J]. Remote Sensing of Environment, 2019, 225:307-316. [3] SCHEEL A,DIETMAYER K. Tracking multiple vehicles using a variational radar model[J]. IEEE Transactions on Intelligent Transportation Systems,2019,20(10):3721-3736. [4] ZHU J,SAMUEL K G G,MASOOD S Z,et al. Learning to recognize shadows in monochromatic natural images[C]//Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2010:223-230. [5] GUO R,DAI Q,HOIEM D. Paired regions for shadow detection and removal[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(12):2956-2967. [6] VICENTE T F Y,HOAI M,SAMARAS D. Leave-one-out kernel optimization for shadow detection and removal[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(3):682-695. [7] KHAN S H,BENNAMOUN M,SOHEL F,et al. Automatic shadow detection and removal from a single image[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(3):431-446. [8] VICENTE T F Y,HOU L,YU C P,et al. Large-scale training of shadow detectors with noisily-annotated shadow examples[C]//Proceedings of the 14th European Conference on Computer Vision, LNCS 9910. Cham:Springer,2016:816-832. [9] NGUYEN V,VICENTE T F Y,ZHAO M,et al. Shadow detection with conditional generative adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE,2017:4520-4528. [10] LE H,VICENTE T F Y,NGUYEN V,et al. A+D Net:training a shadow detector with adversarial shadow attenuation[C]//Proceedings of the 15th European Conference on Computer Vision. Cham:Springer,2018:680-696. [11] WANG J,LI X,YANG J. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:1788-1797. [12] MOHAJERANI S,SAEEDI P. CPNet:a context preserver convolutional neural network for detecting shadows in single RGB images[C]//Proceedings of the IEEE 20th International Workshop on Multimedia Signal Processing. Piscataway:IEEE,2018:1-5. [13] ZHANG Z,LIU Q,WANG Y. Road extraction by deep residual U-Net[J]. IEEE Geoscience and Remote Sensing Letters,2018, 15(5):749-753. [14] KAMNITSAS K,BAUMGARTNER C,LEDIG C,et al. Unsupervised domain adaptation in brain lesion segmentation with adversarial networks[C]//Proceedings of the 25th International Conference on Information Processing in Medical Imaging,LNCS 10265. Cham:Springer,2017:597-609. [15] CHEN Y,LI W,SAKARIDIS C,et al. Domain adaptive faster RCNN for object detection in the wild[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:3339-3348. [16] LONG M,CAO Y,WANG J,et al. Learning transferable features with deep adaptation networks[C]//Proceedings of the 32nd International Conference on Machine Learning. New York:JMLR.org, 2015:97-105. [17] HUANG H,HUANG Q,KRÄHENBÜHL P. Domain transfer through deep activation matching[C]//Proceedings of the 15th European Conference on Computer Vision,LNCS 11220. Cham:Springer,2018:611-626-605. [18] XIE R,YU F,WANG J,et al. Multi-level domain adaptive learning for cross-domain detection[EB/OL].[2019-03-20]. https://arxiv.org/pdf/1907.11484.pdf. [19] VU T H,JAIN H,BUCHER M,et al. ADVENT:adversarial entropy minimization for domain adaptation in semantic segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:2512-2521. [20] ISOLA P,ZHU J,ZHOU T,et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:5967-5976. [21] LOFFE S,SZEGEDY C. Batch normalization:accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning. New York:JMLR.org,2015:448-456. [22] 廖斌, 吴文. 区域配对引导的光照传播视频阴影去除方法[J]. 计算机应用,2019,39(2):556-563.(LIAO B,WU W. Video shadow removal using region matching guided by illumination transfer[J]. Journal of Computer Applications,2019,39(2):556-563.)
2020, Vol. 40 
