Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (5): 1471-1476.DOI: 10.11772/j.issn.1001-9081.2020071101

Special Issue: 多媒体计算与计算机仿真

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Application of improved DeepLabV3+ model in mural segmentation

CAO Jianfang1,2, TIAN Xiaodong1, JIA Yiming1, YAN Minmin1   

  1. 1. College of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan Shanxi 030024, China;
    2. Department of Computer, Xinzhou Teachers University, Xinzhou Shanxi 034000, China
  • Received:2020-07-11 Revised:2020-10-06 Online:2021-05-10 Published:2021-05-19
  • Supported by:
    This work is partially supported by the Program of the Humanities and Social Sciences Key Research Base of Higher Education Institutions of Shanxi (20190130).

改进DeepLabV3+模型在壁画分割中的应用

曹建芳1,2, 田晓东1, 贾一鸣1, 闫敏敏1   

  1. 1. 太原科技大学 计算机科学与技术学院, 太原 030024;
    2. 忻州师范学院 计算机系, 山西 忻州 034000
  • 通讯作者: 曹建芳
  • 作者简介:曹建芳(1976-),女,山西忻州人,教授,博士,CCF高级会员,主要研究方向:数字图像理解、大数据;田晓东(1996-),男,山西朔州人,硕士研究生,主要研究方向:数字图像处理;贾一鸣(1996-),男,山西太原人,硕士研究生,主要研究方向:深度学习、图像处理;闫敏敏(1997-),女,山西长治人,硕士研究生,主要研究方向:智能信息处理。
  • 基金资助:
    山西省高等学校人文社会科学重点研究基地项目(20190130)。

Abstract: Aiming at the problems of blurred target boundaries and low image segmentation efficiency in the image segmentation process of ancient murals, a multi-class image segmentation model fused with a lightweight convolutional neural network named MC-DM (Multi-Class DeepLabV3+MobileNetV2 (Mobile Networks Vision 2)) was proposed. In the model, DeepLabV3+ architecture and MobileNetV2 network were combined together, and the unique spatial pyramid structure of DeepLabV3+ was utilized to perform multi-scale fusion of the convolutional features of the mural to reduce the loss of image details during the mural segmentation. First of all, the features of the input image were extracted by MobileNetV2 to ensure the accurate extraction of image information and reduce the time consumption at the same time. Secondly, the image features were processed through the dilated convolution, so that the receptive field was expanded, and more semantic information was obtained without changing the number of parameters. Finally, the bilinear interpolation method was utilized to up-sample the output feature image to obtain a pixel-level prediction segmentation map, so that the accuracy of image segmentation was ensured to the greatest extent. In the JetBrains PyCharm Community Edition 2019 environment, a dataset made of 1 000 mural scanning pictures was used for testing. Experimental results showed that the MC-DM model had a 1% improvement in training accuracy compared with the traditional SegNet (Segment Network)-based image segmentation model, and had a 2% improvement in accuracy compared with the image segmentation model based on PSPNet (Pyramid Scene Parsing Network), and the Peak Signal-to-Noise Ratio (PSNR) of the MC-DM model was 3 to 8 dB higher than those of the experimental comparison models on average, which verified the effectiveness of the model in the field of mural segmentation. The proposed model provides a new idea for the segmentation of ancient mural images.

Key words: mural segmentation, multi-scale information fusion, depthwise separable convolution, inverted residual, spatial pyramid pool

摘要: 针对古代壁画图像分割过程中出现的目标边界模糊、图像分割效率低等问题,提出一种融合轻量级卷积神经网络的多分类图像分割模型MC-DM,该模型将DeepLabV3+结构和MobileNetV2相结合,利用DeepLabV3+特有的空间金字塔结构对壁画的卷积特征进行多尺度融合,从而减少壁画分割时的图像细节损失。首先,通过MobileNetV2对输入图像进行特征提取,从而在确保图像信息准确提取的同时减少耗时;其次,通过空洞卷积处理图像特征,从而扩展感受野,并在不改变参数数量的情况下得到更多的语义信息;最后,采用双线性插值的方法对输出特征图像进行上采样,以得到像素级的预测分割图,从而最大限度保证图像分割的准确性。在JetBrains PyCharm Community Edition 2019环境下,利用以1 000张壁画扫描图片制作而成的数据集进行测试,实验结果表明,MC-DM模型较传统的基于SegNet的图像分割模型在训练精确度上提升了1个百分点,较基于PSPNet的图像分割模型在精确度上提升了2个百分点,且MC-DM模型的峰值信噪比(PSNR)较实验对比模型平均提高了3~8 dB,充分验证了该模型在壁画分割领域的有效性。所提模型为古代壁画图像分割提供了新的思路。

关键词: 壁画分割, 多尺度信息融合, 深度可分离卷积, 倒转残差, 空间金字塔池

CLC Number: