Application of Inception-v3 model integrated with transfer learning in dynasty identification of ancient murals

doi:10.11772/j.issn.1001-9081.2020121924

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (11): 3219-3227.DOI: 10.11772/j.issn.1001-9081.2020121924

• Artificial intelligence • Previous Articles Next Articles

Application of Inception-v3 model integrated with transfer learning in dynasty identification of ancient murals

Jianfang CAO¹^,²(), Minmin YAN¹, Yiming JIA¹, Xiaodong TIAN¹

^1.College of Computer Science and Technology，Taiyuan University of Science and Technology，Taiyuan Shanxi 030024，China
^2.Computer Department，Xinzhou Teachers University，Xinzhou Shanxi 034000，China

Received:2020-12-09 Revised:2021-07-23 Accepted:2021-08-03 Online:2021-03-07 Published:2021-11-10
Contact: Jianfang CAO
About author:CAO Jianfang，born in 1976，Ph. D.，professor. Her research interests include digital image understanding，big data
YAN Minmin， born in 1997， M. S. candidate. Her research interests include intelligent information processing
JIA Yiming，born in 1996，M. S. candidate. His research interests include digital image processing，machine learning
TIAN Xiaodong，born in 1996，M. S. candidate.
Supported by:
the Humanities and Social Sciences Key Research Base Project of Shanxi Province Colleges and Universities(20190130)

融合迁移学习的Inception-v3模型在古壁画朝代识别中的应用

曹建芳¹^,²(), 闫敏敏¹, 贾一鸣¹, 田晓东¹

^1.太原科技大学计算机科学与技术学院，太原 030024
^2.忻州师范学院计算机系，山西忻州 034000

通讯作者: 曹建芳
作者简介:曹建芳（1976—），女，山西忻州人，教授，博士，CCF高级会员，主要研究方向：数字图像理解、大数据
闫敏敏（1997—），女，山西长治人，硕士研究生，主要研究方向：智能信息处理
贾一鸣（1996—），男，山西太原人，硕士研究生，主要研究方向：数字图像处理、机器学习
田晓东（1996—），男，山西朔州人，硕士研究生，主要研究方向：图像处理、深度学习。
基金资助:
山西省高等学校人文社会科学重点研究基地项目(20190130)

Abstract

Abstract:

Aiming at the problems of small quantity， poor quality， difficulty in feature extraction， and similarity of mural text and painting style of ancient mural images， an Inception-v3 model integrated with transfer learning was proposed to identify and classify the dynasties of ancient murals. Firstly， the Inception-v3 model was pre-trained on the ImageNet dataset to obtain the migration model. After fine-tuning the parameters of the migration model on the small mural dataset， the high-level features were extracted from the mural images. Then， the feature representation ability was enhanced by adding two fully connected layers， and the color histogram and Local Binary Pattern （LBP） texture histogram were used to extract the artistic features of murals. Finally， the high-level features were combined with the artistic features， and the Softmax classifier was used to perform the dynasty classification of murals. Experimental results show that， the training process of the proposed model was stable. On the constructed small mural dataset， the proposed model has the final accuracy of 88.70%， the recall of 88.62%， and the F1-score of 88.58%. Each evaluation index above of the proposed model is better than those of the classic network models such as AlexNet and Visual Geometry Group Net （VGGNet）. Compared with LeNet-5， AlexNet-S6 and other improved convolutional neural network models， the proposed model has the accuracy of each dynasty category improved by at least 7 percentage points on average. It can be seen that the proposed model has strong generalization ability， is not prone to overfitting， and can effectively identify the dynasty to which the murals belong.

Key words: mural classification, dynasty identification, transfer learning, Inception-v3 model, color histogram

摘要：

针对古代壁画图像数量少、质量差、特征提取困难和存在壁画文本与绘画风格相似等问题，提出了一种融合迁移学习的Inception-v3模型来对古代壁画的朝代进行识别与分类。首先，将Inception-v3模型在ImageNet数据集上进行预训练以得到迁移模型；然后，将迁移模型在小型壁画数据集上进行参数微调后对壁画图像提取高层特征；其次，增加两个全连接层来增强特征表达能力，并用颜色直方图与局部二值模式（LBP）纹理直方图提取壁画的艺术特征；最后，将高层特征与艺术特征相融合，用Softmax分类器进行壁画的朝代分类。实验结果表明，所提出的模型训练过程稳定，在构造的小型壁画数据集上，其最终准确率为88.70%，召回率为88.62%，F1值为88.58%，以上各评价指标均优于AlexNet、VGGNet等经典网络模型；与LeNet-5、AlexNet-S6等改进的卷积神经网络模型相比，该模型对各朝代类别准确率平均提升了至少7个百分点。可见，该模型泛化能力强，不易出现过拟合现象，能有效识别壁画所属朝代。

关键词: 壁画分类, 朝代识别, 迁移学习, Inception-v3模型, 颜色直方图

CLC Number:

TP391.41

Jianfang CAO, Minmin YAN, Yiming JIA, Xiaodong TIAN. Application of Inception-v3 model integrated with transfer learning in dynasty identification of ancient murals[J]. Journal of Computer Applications, 2021, 41(11): 3219-3227.

曹建芳, 闫敏敏, 贾一鸣, 田晓东. 融合迁移学习的Inception-v3模型在古壁画朝代识别中的应用[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3219-3227.

Figures/Tables 17

References 19

1	王可，王慧琴，殷颖，等.基于光谱重建技术的壁画颜色复原与评价［J］.激光技术，2019，43（2）：280-285. 10.7510/jgjs.issn.1001-3806.2019.02.024
	WANG K， WANG H Q， YIN Y， et al. Reproduction and evaluation of mural color based on spectral reconstruction technology ［J］. Laser Technology， 2019， 43（2）： 280-285. 10.7510/jgjs.issn.1001-3806.2019.02.024
2	潘云鹏，鲁东明.古代敦煌壁画的数字化保护与修复［J］.系统仿真学报，2003，15（3）：310-314. 10.3969/j.issn.1004-731X.2003.03.003
	PAN Y P， LU D M. Digital protection and restoration of Dunhuang mural ［J］. Journal of System Simulation， 2003， 15（3）： 310-314. 10.3969/j.issn.1004-731X.2003.03.003
3	ZENG Y， GONG Y. Nearest neighbor based digital restoration of damaged ancient Chinese paintings ［C］// Proceedings of the 2018 IEEE 23rd International Conference on Digital Signal Processing. Piscataway： IEEE， 2018： 1-5. 10.1109/icdsp.2018.8631553
4	WANG H， LI Q Q， ZOU Q. Inpainting of Dunhuang murals by sparsely modeling the texture similarity and structure continuity ［J］. Journal on Computing and Cultural Heritage， 2019， 12（3）： Article No.17. 10.1145/3280790
5	唐大伟，鲁东明，许端清，等.壁画图像分类中的分组多实例学习方法［J］.中国图象图形学报，2014，19（5）：708-715. 10.11834/jig.20140508
	TANG D W， LU D M， XU D Q， et al. Clustered multiple instance learning for mural image classification ［J］. Journal of Image and Graphics， 2014， 19（5）： 708-715. 10.11834/jig.20140508
6	王琦.古代壁画的语义检索技术及应用研究［D］.杭州：浙江大学，2011：71-84.
	WANG Q. Semantic retrieval technology and applications for ancient murals ［D］. Hangzhou： Zhejiang University， 2011： 71-84.
7	唐大伟，鲁东明，杨冰，等.轮廓整体结构约束的壁画图像相似性度量［J］.中国图象图形学报，2013，18（8）：968-975. 10.11834/jig.20130811
	TANG D W， LU D M， YANG B， et al. Similarity metrics between mural images with constraints of the overall structure of contours ［J］. Journal of Image and Graphics， 2013， 18（8）： 968-975. 10.11834/jig.20130811
8	ZHOU L J， LI Q W， HUO G Y， et al. Image classification using biomimetic pattern recognition with convolutional neural networks features ［J］. Computational Intelligence and Neuroscience， 2017， 2017： Article No.3792805. 10.1155/2017/3792805
9	TANG P， WANG X G， SHI B G， et al. Deep FisherNet for image classification ［J］. IEEE Transactions on Neural Networks and Learning Systems， 2019， 30（7）： 2244-2250. 10.1109/tnnls.2018.2874657
10	郝占龙，罗晓曙，赵书林.基于同层多尺度核CNN的单细胞图像分类［J］.计算机工程与应用，2018，54（15）：181-184. 10.3778/j.issn.1002-8331.1704-0294
	HAO Z L， LUO X S， ZHAO S L. Single cell image classification based on same layer multi scale kernel CNN ［J］. Computer Engineering and Applications， 2018， 54（15）： 181-184. 10.3778/j.issn.1002-8331.1704-0294
11	LI Q Q， ZOU Q， MA D， et al. Dating ancient paintings of Mogao Grottoes using deeply， learnt visual codes ［J］. Science China Information Sciences， 2018， 61（9）： Article No.092105. 10.1007/s11432-017-9308-x
12	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2818-2826. 10.1109/cvpr.2016.308
13	石祥滨，房雪键，张德园，等.基于深度学习混合模型迁移学习的图像分类［J］.系统仿真学报，2016，28（1）：167-173，182. 10.1016/j.neucom.2016.01.074
	SHI X B， PANG X J， ZHANG D Y， et al. Image classification based on mixed deep learning model transfer learning ［J］. Journal of System Simulation， 2016， 28（1）： 167-173， 182. 10.1016/j.neucom.2016.01.074
14	KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks ［C］// Proceedings of the 2012 25th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2012： 1097-1105.
15	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for largescale image recognition ［EB/OL］. （2015-04-10）［2020-04-10］. .
16	张荣梅，张琦，陈彬.基于改进LeNet-5的车牌识别算法［J］.科学技术与工程，2020，20（12）：4775-4779. 10.3969/j.issn.1671-1815.2020.12.024
	ZHANG R M， ZHANG Q， CHEN B. An improved license plate recognition algorithm based on LeNet-5 convolutional neural network ［J］. Science Technology and Engineering， 2020， 20（12）： 4775-4779. 10.3969/j.issn.1671-1815.2020.12.024
17	XIAO L S， YAN Q， DENG S Y. Scene classification with improved AlexNet model ［C］// Proceedings of the 2017 12th International Conference on Intelligent Systems and Knowledge Engineering. Piscataway： IEEE， 2017： 1-6. 10.1109/iske.2017.8258820
18	徐志京，汪毅.青光眼眼底图像的迁移学习分类方法［J］.计算机工程与应用，2021，57（3）：144-149. 10.1007/978-3-030-87000-3_14
	XU Z J， WANG Y. Glaucoma fundus images classification method based on transfer learning ［J］. Computer Engineering and Applications， 2021， 57（3）： 144-149. 10.1007/978-3-030-87000-3_14
19	钟志权，袁进.唐晓颖.基于卷积神经网络的左右眼识别［J］.计算机研究与发展，2018，55（8）：1667-1673. 10.7544/issn1000-1239.2018.20180215
	ZHONG Z Q， YUAN J， TANG X Y. Left-vs-right eye discrimination based on convolutional neural network ［J］. Journal of Computer Research and Development， 2018， 55（8）： 1667-1673. 10.7544/issn1000-1239.2018.20180215

type	patch size/stride	input size
conv	3×3/2	299×299×3
conv	3×3/1	149×149×32
conv	3×3/1	147×147×32
pool	3×3/2	147×147×64
conv	3×3/1	73×73×64
conv	3×3/2	71×71×80
conv	3×3/1	35×35×192
3×Inception	—	35×35×288
5×Inception	—	17×17×768
2×Inception	—	8×8×1 280
pool	8×8	8×8×2 048
linear	logits	1×1×2 048
Softmax	classifier	1×1×1 000

type	patch size/stride	input size
conv	3×3/2	299×299×3
conv	3×3/1	149×149×32
conv	3×3/1	147×147×32
pool	3×3/2	147×147×64
conv	3×3/1	73×73×64
conv	3×3/2	71×71×80
conv	3×3/1	35×35×192
3×Inception	—	35×35×288
5×Inception	—	17×17×768
2×Inception	—	8×8×1 280
pool	8×8	8×8×2 048
linear	logits	1×1×2 048
Softmax	classifier	1×1×1 000

朝代种类	样本总数	训练集样本数	验证集样本数	测试集样本数
石器时代	840	672	84	84
秦汉时期	1 560	1 248	156	156
隋唐时期	1 860	1 488	186	186
宋金时期	1 280	1 024	128	128
魏晋时期	2 180	1 744	218	218
明清时期	1 980	1 584	198	198

朝代种类	样本总数	训练集样本数	验证集样本数	测试集样本数
石器时代	840	672	84	84
秦汉时期	1 560	1 248	156	156
隋唐时期	1 860	1 488	186	186
宋金时期	1 280	1 024	128	128
魏晋时期	2 180	1 744	218	218
明清时期	1 980	1 584	198	198

序号	学习率	准确率/%
1	0.001	78.62
2	0.01	84.44
3	0.1	88.70

Application of Inception-v3 model integrated with transfer learning in dynasty identification of ancient murals

融合迁移学习的Inception-v3模型在古壁画朝代识别中的应用

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 17

References 19

Related Articles 15

Recommended Articles

Metrics

调整情况	准确率
原图	96.35
增加灰度值	38.76
增加饱和度	92.57
反色变换	69.48

模型	运行时间/h	准确率/%
AlexNet	2.5	78.34
ResNet	4.2	69.56
VGGNet	5.0	76.48
LeNet-5	1.7	82.98
Alex-10	2.8	75.05
R-VGGNet	5.1	78.66
AlexNet-S6	3.0	79.04
本文模型	1.5	88.70

模型	石器时代	秦汉时期	隋唐时期	宋金时期	魏晋时期	明清时期
AlexNet-10	91.23	69.36	69.43	68.86	83.32	70.46
AlexNet-S6	90.45	69.70	75.54	74.50	88.53	72.45
LeNet-5	91.57	73.76	71.87	84.12	89.60	87.60
R-VGGNet	89.95	74.21	75.58	69.80	85.65	78.39
本文模型	98.69	84.47	84.56	83.80	93.32	83.89

[1]	CHEN Zhengtao, HUANG Can, YANG Bo, ZHAO Li, LIAO Yong. Yak face recognition algorithm of parallel convolutional neural network based on transfer learning [J]. Journal of Computer Applications, 2021, 41(5): 1332-1336.
[2]	WANG Jinkai, JIA Xu. Vein recognition algorithm based on Siamese nonnegative matrix factorization with transferability [J]. Journal of Computer Applications, 2021, 41(3): 898-903.
[3]	Jin ZHAO, Wen’ai SONG, Jun TAI, Jijiang YANG, Qing WANG, Xiaodan LI, Yi LEI, Yue QIU. Review of computer-aided face diagnosis for obstructive sleep apnea in children [J]. Journal of Computer Applications, 2021, 41(11): 3394-3401.
[4]	Xiaolong LIU, Shitong WANG. Open set fuzzy domain adaptation algorithm via progressive separation [J]. Journal of Computer Applications, 2021, 41(11): 3127-3131.
[5]	HUANG Xueyu, XU Haote, TAO Jianwen. Multi-source adaptation classification framework with feature selection [J]. Journal of Computer Applications, 2020, 40(9): 2499-2506.
[6]	ZHANG Chenxi, TANG Shu, TANG Ke. Strategies of parameter fault detection for rocket engines based on transfer learning [J]. Journal of Computer Applications, 2020, 40(9): 2774-2780.
[7]	BAI Ruyi, GUO Xiaoying, JIA Chunhua. Sentiment prediction of small sample abstract painting image based on feature fusion [J]. Journal of Computer Applications, 2020, 40(8): 2207-2213.
[8]	MA Jinlin, WEI Meng, MA Ziping. Pulmonary nodule segmentation method based on deep transfer learning [J]. Journal of Computer Applications, 2020, 40(7): 2117-2125.
[9]	XIAO He, LIU Zhiqin, WANG Qingfeng, HUANG Jun, ZHOU Ying, LIU Qiyu, XU Weiyun. Mass and calcification classification method in mammogram based on multi-view transfer learning [J]. Journal of Computer Applications, 2020, 40(5): 1460-1464.
[10]	CHEN Jie, ZHANG Ting, DU Yi. Reconstruction of porous media using adaptive deep transfer learning [J]. Journal of Computer Applications, 2020, 40(4): 1231-1236.
[11]	CHEN Guihui, YI Xin, LI Zhongbing, QIAN Jiren, CHEN Wu. Third-party construction target detection in aerial images of pipeline inspection based on improved YOLOv2 and transfer learning [J]. Journal of Computer Applications, 2020, 40(4): 1062-1068.
[12]	ZHENG Zongsheng, HU Chenyu, JIANG Xiaoyi. Deep transfer adaptation network based on improved maximum mean discrepancy algorithm [J]. Journal of Computer Applications, 2020, 40(11): 3107-3112.
[13]	MAO Wentao, YANG Chao, LIU Yamin, TIAN Siyu. Deep domain adaptation model with multi-scale residual attention for incipient fault detection of bearings [J]. Journal of Computer Applications, 2020, 40(10): 2890-2898.
[14]	WU Kuan, QIN Pinle, CHAI Rui, ZENG Jianchao. Benign and malignant diagnosis of thyroid nodules based on different ultrasound imaging [J]. Journal of Computer Applications, 2020, 40(1): 77-82.
[15]	LI Minze, LI Xiaoxia, WANG Xueyuan, SUN Wei. Real-time facial expression recognition based on convolutional neural network with multi-scale kernel feature [J]. Journal of Computer Applications, 2019, 39(9): 2568-2574.

分辨率	准确率/%
299×299	96.35
897×897	90.38
1 495×1 495	87.64

分辨率	准确率/%
299×299	96.35
897×897	90.38
1 495×1 495	87.64