《计算机应用》唯一官方网站 ›› 2023, Vol. 43 ›› Issue (5): 1401-1408.DOI: 10.11772/j.issn.1001-9081.2022040581
收稿日期:
2022-04-24
修回日期:
2022-06-19
接受日期:
2022-07-01
发布日期:
2022-07-26
出版日期:
2023-05-10
通讯作者:
王彬
作者简介:
王彬(1971—),男,陕西西安人,副教授,硕士,CCF会员,主要研究方向:进化计算、神经网络 wb@xaut.edu.cn基金资助:
Bin WANG1,2(), Tian XIANG1, Yidong LYU1, Xiaofan WANG1
Received:
2022-04-24
Revised:
2022-06-19
Accepted:
2022-07-01
Online:
2022-07-26
Published:
2023-05-10
Contact:
Bin WANG
About author:
WANG Bin, born in 1971, M. S., associate professor. His research interests include evolutionary computing, neural network.Supported by:
摘要:
针对轻量型卷积神经网络(LCNN)的精确度和复杂度均衡优化问题,提出基于快速非支配排序遗传算法(NSGA-Ⅱ)的自适应多尺度特征通道分组优化算法对LCNN特征通道分组结构进行优化。首先,将LCNN中的特征融合层结构的复杂度最小化和精确度最大化作为两个优化目标,进行双目标函数建模及理论分析;然后,设计基于NSGA-Ⅱ的LCNN结构优化框架,并在原始LCNN结构的深度卷积层之上增加基于NSGA-Ⅱ的自适应分组层,构建基于NSGA-Ⅱ的自适应多尺度的特征融合网络NSGA2-AMFFNetwork。在图像分类数据集上的实验结果显示,与手工设计的网络结构M_blockNet_v1相比,NSGA2-AMFFNetwork的平均精确度提升了1.220 2个百分点,运行时间降低了41.07%。这表明所提优化算法能较好平衡LCNN的复杂度和精确度,同时还可为领域知识不足的普通用户提供更多性能表现均衡的网络结构选择方案。
中图分类号:
王彬, 向甜, 吕艺东, 王晓帆. 基于NSGA‑Ⅱ的自适应多尺度特征通道分组优化算法[J]. 计算机应用, 2023, 43(5): 1401-1408.
Bin WANG, Tian XIANG, Yidong LYU, Xiaofan WANG. Adaptive multi-scale feature channel grouping optimization algorithm based on NSGA‑Ⅱ[J]. Journal of Computer Applications, 2023, 43(5): 1401-1408.
结构 | 平均精确度/% | 运行时间/s |
---|---|---|
M_blockNet_v1 | 91.492 5 | 8 176 |
MobileNet_v1 | 80.187 8 | 1 883 |
MobileNet_v2 | 83.333 5 | 2 162 |
ShuffleNet_v2 | 89.328 7 | 1 791 |
DenseNet | 91.734 1 | 4 862 |
SSCA_MblockNet | 92.300 5 | 5 132 |
NSGA2-AMFFNetwork | 92.712 7 | 4 818 |
表1 平均测试精确度及运行时间对比
Tab. 1 Comparison of average test accuracy and running time
结构 | 平均精确度/% | 运行时间/s |
---|---|---|
M_blockNet_v1 | 91.492 5 | 8 176 |
MobileNet_v1 | 80.187 8 | 1 883 |
MobileNet_v2 | 83.333 5 | 2 162 |
ShuffleNet_v2 | 89.328 7 | 1 791 |
DenseNet | 91.734 1 | 4 862 |
SSCA_MblockNet | 92.300 5 | 5 132 |
NSGA2-AMFFNetwork | 92.712 7 | 4 818 |
解序号 | NSGA2-AMFFModule1 | NSGA2-AMFFModule2 | ||||
---|---|---|---|---|---|---|
1×1 点卷积 | 3×3 标准卷积 | 3×3 膨胀卷积 | 1×1 点卷积 | 3×3 标准卷积 | 3×3 膨胀卷积 | |
1 | 27 | 58 | 11 | 175 | 34 | 21 |
2 | 97 | 7 | 10 | 192 | 20 | 28 |
3 | 80 | 5 | 11 | 198 | 13 | 29 |
表2 NSGA2-AMFFNetwork模块的通道分组参数
Tab. 2 Channel grouping parameters of NSGA2-AMFFNetwork modules
解序号 | NSGA2-AMFFModule1 | NSGA2-AMFFModule2 | ||||
---|---|---|---|---|---|---|
1×1 点卷积 | 3×3 标准卷积 | 3×3 膨胀卷积 | 1×1 点卷积 | 3×3 标准卷积 | 3×3 膨胀卷积 | |
1 | 27 | 58 | 11 | 175 | 34 | 21 |
2 | 97 | 7 | 10 | 192 | 20 | 28 |
3 | 80 | 5 | 11 | 198 | 13 | 29 |
LCNN | 模块参数总量 | 模块计算总量/107 | 平均 精确度/% | 运行 时间/s |
---|---|---|---|---|
M_blockNet_v1 | 6 656 | 68.640 8 | 91.492 5 | 8 176 |
NSGA2-AMFFNetwork | 48 312 | 42.135 6 | 92.712 7 | 4 818 |
表3 LCNN优化前后复杂度和精确度对比结果
Tab. 3 Comparison results of complexity and accuracy before and after LCNN optimization
LCNN | 模块参数总量 | 模块计算总量/107 | 平均 精确度/% | 运行 时间/s |
---|---|---|---|---|
M_blockNet_v1 | 6 656 | 68.640 8 | 91.492 5 | 8 176 |
NSGA2-AMFFNetwork | 48 312 | 42.135 6 | 92.712 7 | 4 818 |
1 | 武文娟,李勇. Emfacenet:一种轻量级人脸识别的卷积神经网络[J/OL]. 小型微型计算机系统 (2021-12-17) [2022-03-21].. 10.3390/s21175901 |
WU W J, LI Y. Emfacenet: a lightweight convolutional neural network for face recognition[J/OL]. Journal of Chinese Computer Systems (2021-12-17) [2022-03-21].. 10.3390/s21175901 | |
2 | 钱慧芳,易剑平,付云虎. 基于深度学习的人体动作识别综述[J]. 计算机科学与探索, 2021, 15(3): 438-455. 10.3778/j.issn.1673-9418.2009095 |
QIAN H F, YI J P, FU Y H. Review of human action recognition based on deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(3): 438-455. 10.3778/j.issn.1673-9418.2009095 | |
3 | 马利,郑诗雨,牛斌. 应用区域关联自适应图卷积的动作识别方法[J]. 计算机科学与探索, 2022, 16(4):898-908. 10.3778/j.issn.1673-9418.2010070 |
MA L, ZHENG S Y, NIU B. Action recognition method on regional association adaptive graph convolution[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4):898-908. 10.3778/j.issn.1673-9418.2010070 | |
4 | SINGH S P, KUMAR A, DARBARI H, et al. Machine translation using deep learning: an overview[C]// Proceedings of the 2017 International Conference on Computer, Communications and Electronics. Piscataway: IEEE, 2017: 162-167. 10.1109/comptelix.2017.8003957 |
5 | SCHMIDHUBER J. Deep learning in neural networks: an overview[J]. Neural Networks, 2015, 61: 85-117. 10.1016/j.neunet.2014.09.003 |
6 | 徐子豪,黄伟泉,王胤. 基于深度学习的监控视频中多类别车辆检测[J]. 计算机应用, 2019, 39(3): 700-705. 10.11772/j.issn.1001-9081.2018071587 |
XU Z H, HUANG W Q, WANG Y. Multi-class vehicle detection in surveillance video based on deep learning[J]. Journal of Computer Applications, 2019, 39(3): 700-705. 10.11772/j.issn.1001-9081.2018071587 | |
7 | 周飞燕,金林鹏,董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251. 10.11897/SP.J.1016.2017.01229 |
ZHOU F Y, JIN L P, DONG J. Review of convolutional neural network[J]. Chinese Journal of Computers, 2017, 40(6): 1229-1251. 10.11897/SP.J.1016.2017.01229 | |
8 | 朱叶,余宜林,郭迎春. HRDA-Net:面向真实场景的图像多篡改检测与定位算法[J]. 通信学报, 2022, 43(1): 217-226. 10.11959/j.issn.1000-436x.2022016 |
ZHU Y, YU Y L, GUO Y C. HRDA-Net: image multiple manipulation detection and location algorithm in real scene[J]. Journal on Communications, 2022, 43(1): 217-226. 10.11959/j.issn.1000-436x.2022016 | |
9 | GHAYOUMI M. A quick review of deep learning in facial expression[J]. Journal of Communication and Computer, 2017, 14: 34-38. 10.17265/1548-7709/2017.01.004 |
10 | 孟亮,郭小燕,杜佳举,等. 一种轻量级CNN农作物病害图像识别模型[J]. 江苏农业学报, 2021, 37(5): 1143-1150. 10.3969/j.issn.1000-4440.2021.05.008 |
MENG L, GUO X Y, DU J J, et al. A lightweight CNN model for image recognition of crop diseases[J]. Jiangsu Journal of Agricultural Sciences, 2021, 37(5): 1143-1150. 10.3969/j.issn.1000-4440.2021.05.008 | |
11 | 井望,李汪根,沈公仆,等. 轻量级多信息图卷积神经网络动作识别方法[J]. 计算机应用研究, 2022, 39(4): 1247-1252. |
JING W, LI W G, SHEN G P, et al. Lightweight multi-information graph convolution neural network action recognition method[J]. Application Research of Computers, 2022, 39(4): 1247-1252. | |
12 | 毕鹏程,罗健欣,陈卫卫. 轻量化卷积神经网络技术研究[J]. 计算机工程与应用, 2019, 55(16): 25-35. |
BI P C, LUO J X, CHEN W W. Research on lightweight convolutional neural network technology[J]. Computer Engineering and Applications, 2019, 55(16): 25-35. | |
13 | 廖干洲. 轻量化卷积神经网络技术综述[J]. 新一代信息技术, 2020, 3(10): 36-40. 10.3969/j.issn.2096-6091.2020.10.007 |
LIAO G Z. Overview of lightweight convolutional neural network technology[J]. New Generation Information Technology, 2020, 3(10): 36-40. 10.3969/j.issn.2096-6091.2020.10.007 | |
14 | SUN Y N, XUE B, ZHANG M J, et al. An experimental study on hyper-parameter optimization for stacked auto-encoders[C]// Proceedings of the 2018 IEEE Congress on Evolutionary Computation. Piscataway: IEEE, 2018: 1-8. 10.1109/cec.2018.8477921 |
15 | 王伟,孙成胜,伍少梅,等. 一种轻量级文本蕴含模型[J]. 四川大学学报(自然科学版), 2021, 58(5): No.052001. 10.19907/j.0490-6756.2021.052001 |
WANG W, SUN C S, WU S M, et al. A lightweight text entailment model[J]. Journal of Sichuan University (Natural Science Edition), 2021, 58(5): No.052001. 10.19907/j.0490-6756.2021.052001 | |
16 | 许虞俊,李晨. 基于YOLO优化的轻量级目标检测网络[J]. 计算机科学, 2021, 48(11A): 265-269. 10.11896/jsjkx.201000152 |
XU Y J, LI C. Light-weight object detection network optimized based on YOLO family[J]. Computer Science, 2021, 48(11A): 265-269. 10.11896/jsjkx.201000152 | |
17 | 徐晓,蒋志翔,张杨. 基于通道剪枝和量化的卷积神经网络压缩方法[J]. 计算机工程与设计, 2021, 42(10): 2860-2866. |
XU X, JIANG Z X, ZHANG Y. Convolution neural network compression method based on channel pruning and quantization[J]. Computer Engineering and Design, 2021, 42(10): 2860-2866. | |
18 | DEB K, PRATAP A, AGARWAL S, et al. A fast and elitist multiobjective genetic algorithm: NSGA-Ⅱ[J]. IEEE Transactions on Evolutionary Computation, 2002, 6(2): 182-197. 10.1109/4235.996017 |
19 | ZITZLER E, LAUMANNS M, THIELE L. SPEA2: improving the strength Pareto evolutionary algorithm[R/OL]. (2001-05) [2022-03-22].. |
20 | LALWANI S, SINGHAL S, KUMAR R, et al. A comprehensive survey: applications of Multi-Objective Particle Swarm Optimization (MOPSO) algorithm[J]. Transactions on Combinatorics, 2013, 2(1): 39-101. |
21 | CORNE D W, JERRAM N R, KNOWLES J D, et al. PESA-Ⅱ: region-based selection in evolutionary multiobjective optimization[C]// Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation Conference. San Francisco: Morgan Kaufmann Publishers Inc., 2001:283-290. |
22 | ZHANG Q F, LI H. MOEA/D: a multiobjective evolutionary algorithm based on decomposition[J]. IEEE Transactions on Evolutionary Computation, 2007, 11(6): 712-731. 10.1109/tevc.2007.892759 |
23 | ZITZLER E, KÜNZLI S. Indicator-based selection in multi objective search[C]// Proceedings of the 2004 International Conference on Parallel Problem Solving from Nature, LNCS 3242. Berlin: Springer, 2004: 832-842. |
24 | WANG B, SUN Y N, XUE B, et al. Evolving deep convolutional neural networks by variable-length particle swarm optimization for image classification[C]// Proceedings of the 2018 IEEE Congress on Evolutionary Computation. Piscataway: IEEE, 2018: 1-8. 10.1109/cec.2018.8477735 |
25 | SUN Y N, XUE B, ZHANG M J, et al. An experimental study on hyper-parameter optimization for stacked auto-encoders[C]// Proceedings of the 2018 IEEE Congress on Evolutionary Computation. Piscataway: IEEE, 2018: 1-8. 10.1109/cec.2018.8477921 |
26 | LU Z C, WHALEN I, BODDETI V, et al. NSGA-Net: neural architecture search using multi-objective genetic algorithm[C]// Proceedings of the 2019 Genetic and Evolutionary Computation Conference. New York: ACM, 2019: 419-427. 10.1145/3321707.3321729 |
27 | 西安理工大学. 基于非均匀分组的多尺度轻量型深度卷积神经网络模型:202010505261.4[P]. 2020-11-10. |
Xi’an University of Technology. Multi-scale lightweight deep convolutional neural network model based on non-uniform grouping: 202010505261.4[P]. 2020-11-10. | |
28 | 西安理工大学. 一种自动分组的多尺度轻量型深度卷积神经网络优化方法: 202110251256.X[P]. 2021-06-22. |
Xi’an University of Technology. An automatic grouping multi-scale lightweight deep convolutional neural network optimization method: 202110251256.X[P]. 2021-06-22. | |
29 | YANG J C, SHI R, WEI D L, et al. MedMNIST v2: a large-scale lightweight benchmark for 2D and 3D biomedical image classification[J]. Scientific Data, 2023, 10: No.41. 10.1038/s41597-022-01721-8 |
30 | HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[EB/OL]. (2017-04-17) [2022-03-22].. 10.48550/arXiv.1704.04861 |
31 | SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 4510-4520. 10.1109/cvpr.2018.00474 |
32 | MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11218. Cham: Springer, 2018: 122-138. |
33 | HUANG G, LIU Z, L van der MAATEN, et al. Densely connected convolutional networks[C]// Proceedings of the 2017 International Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2261-2269. 10.1109/cvpr.2017.243 |
[1] | 李振亮, 李波. 基于矩阵分解的卷积神经网络改进方法[J]. 《计算机应用》唯一官方网站, 2023, 43(3): 685-691. |
[2] | 申志军, 穆丽娜, 高静, 史远航, 刘志强. 细粒度图像分类综述[J]. 《计算机应用》唯一官方网站, 2023, 43(1): 51-60. |
[3] | 魏佳璇, 杜世康, 于志轩, 张瑞生. 图像分类中的白盒对抗攻击技术综述[J]. 《计算机应用》唯一官方网站, 2022, 42(9): 2732-2741. |
[4] | 陈揆能, 袁小芳. 多目标混合进化算法求解加工时间可控的开放车间调度问题[J]. 《计算机应用》唯一官方网站, 2022, 42(8): 2617-2627. |
[5] | 沈瑜, 李和成, 陈黎娟. 基于近似技术的双层规划进化算法[J]. 《计算机应用》唯一官方网站, 2022, 42(8): 2511-2518. |
[6] | 聂青青, 万定生, 朱跃龙, 李致家, 姚成. 基于时域卷积网络的水文模型[J]. 《计算机应用》唯一官方网站, 2022, 42(6): 1756-1761. |
[7] | 任炜, 白鹤翔. 基于全局与局部标签关系的多标签图像分类方法[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1383-1390. |
[8] | 季长清, 高志勇, 秦静, 汪祖民. 基于卷积神经网络的图像分类算法综述[J]. 《计算机应用》唯一官方网站, 2022, 42(4): 1044-1049. |
[9] | 史非凡, 史旭华. 基于参考向量的自适应约束多目标进化算法[J]. 《计算机应用》唯一官方网站, 2022, 42(2): 542-549. |
[10] | 代荣荣, 李宏慧, 付学良. 基于差分进化融合蚁群算法的数据中心流量调度机制[J]. 《计算机应用》唯一官方网站, 2022, 42(12): 3863-3869. |
[11] | 张谊, 万华, 涂淑琴. 基于计算机视觉的中药饮片分类技术综述与案例研究[J]. 《计算机应用》唯一官方网站, 2022, 42(10): 3224-3234. |
[12] | 贾鹤鸣, 郎春博, 姜子超. 基于轻量级卷积神经网络的植物叶片病害识别方法[J]. 计算机应用, 2021, 41(6): 1812-1819. |
[13] | 李二超, 毛玉燕. 基于空间收缩技术的约束多目标进化算法[J]. 《计算机应用》唯一官方网站, 2021, 41(12): 3419-3425. |
[14] | 许学斌, 张佳达, 刘伟, 路龙宾, 赵雨晴. 融合空间和通道特征的高精度乳腺癌分类方法[J]. 计算机应用, 2021, 41(10): 3025-3032. |
[15] | 严华健, 张国富, 苏兆品, 刘扬. 救灾物资高维多目标自适应分配问题建模与求解[J]. 计算机应用, 2020, 40(8): 2410-2419. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||