Compressed sensing image reconstruction method fusing spatial location and structure information

doi:10.11772/j.issn.1001-9081.2021030434

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (3): 930-937.DOI: 10.11772/j.issn.1001-9081.2021030434

• Multimedia computing and computer simulation • Previous Articles

Compressed sensing image reconstruction method fusing spatial location and structure information

Leping LIN¹^,², Hongmin ZHOU², Ning OUYANG¹^,²()

^1.Key Laboratory of Cognitive Radio and Information Processing，Ministry of Education（Guilin University of Electronic Technology），Guilin Guangxi 541004，China
^2.School of Information and Communication，Guilin University of Electronic Technology，Guilin Guangxi 541004，China

Received:2021-03-22 Revised:2021-06-12 Accepted:2021-06-17 Online:2022-04-09 Published:2022-03-10
Contact: Ning OUYANG
About author:LIN Leping， born in 1980， Ph. D.， associate professor. Her research interests include machine learning， intelligent information processing， image signal processing.
ZHOU Hongmin， born in 1996， M. S. candidate. Her research interests include pattern recognition， deep learning.
Supported by:
National Natural Science Foundation of China(62001133);Special Foundation for Scientific Bases and Talents of Guangxi(AD19110060);Guangxi One Thousand Young and Middle-aged College and University Backbone Teachers Cultivation Program, Fund of Guangxi Key Laboratory of Wireless Wideband Communication and Signal Processing(GXKL06200114)

融合空间位置与结构信息的压缩感知图像重建方法

林乐平¹^,², 周宏敏², 欧阳宁¹^,²()

^1.认知无线电与信息处理省部共建教育部重点实验室(桂林电子科技大学), 广西桂林 541004
^2.桂林电子科技大学信息与通信学院, 广西桂林 541004

通讯作者: 欧阳宁
作者简介:林乐平（1980—），女，广西桂平人，副教授，博士，主要研究方向：机器学习、智能信息处理、图像信号处理
周宏敏（1996—），女，湖南邵阳人，硕士研究生，主要研究方向：模式识别、深度学习；
基金资助:
国家自然科学基金资助项目(62001133);广西科技基地和人才专项(AD19110060);广西高等学校千名中青年骨干教师培育计划项目;广西无线宽带通信与信号处理重点实验室基金资助项目(GXKL06200114)

Abstract

Abstract:

Aiming at the problem of poor visual effects of block-based compressed sensing reconstructed images at low sampling rates， a compressed sensing image reconstruction method that fused Spatial Location and Structure Information （SLSI） was proposed. Firstly， observations were linearly mapped to obtain initial estimated values of image blocks. Then， based on block grouping reconstruction branch and whole image reconstruction branch， the spatial location information and structure information of the image were extracted， enhanced and fused. Finally， weighted strategy was used to fuse the outputs of the two branches to obtain final reconstructed whole image. In the block grouping reconstruction branch， reconstruction resources were allocated according to the data characteristics of the image blocks. In the whole image reconstruction branch， information exchange between adjacent image block pixels was mainly carried out through bilateral filtering and structural feature interaction module. Experimental results show that compared with compressed sensing reconstruction methods based on non-iterative Reconstruction Network （ReconNet） and Multi-scale Reconstruction neural Network with Non-Local constraint （NL-MRN）， due to the combination of the image prior with strong autocorrelation between pixels， when sampling rate is 0.05， the average Peak Signal-to-Noise Ratio （PSNR） and Structural SIMilarity index （SSIM） of the proposed method on the test image data commonly used in the compressed sensing field increase 2.617 5 dB and 0.105 3 respectively， and the visual effects of reconstructed images are better.

Key words: Convolutional Neural Network (CNN), compressed sensing, image reconstruction, feature fusion, image prior

摘要：

针对低采样率下分块压缩感知重建图像视觉效果不佳的问题，提出一种融合空间位置与结构信息的压缩感知图像重建方法（SLSI）。首先，对观测值进行线性映射得到图像块的初步估计值；然后，基于块分组重建支路和全图重建支路对图像的空间位置信息和结构信息进行提取、增强和融合；最后，通过加权策略融合双支路的输出得到最终重建全图。在块分组重建支路中，根据图像块的数据特点分配重建资源。在全图重建支路中，主要通过双边滤波和结构特征交互模块对相邻图像块像素进行信息交互。实验结果表明，与基于非迭代重建网络（ReconNet）、基于非局部约束的多尺度重建网络（NL-MRN）等压缩感知重建方法相比，由于结合了像素间强自相关性这种图像先验，在采样率为0.05的情况下，所提方法在压缩感知领域常用的测试图像数据上的峰值信噪比（PSNR）和结构相似度（SSIM）分别平均提升了2.617 5 dB和0.105 3，重建图像的视觉效果较好。

关键词: 卷积神经网络, 压缩感知, 图像重建, 特征融合, 图像先验

CLC Number:

TP391.41

Leping LIN, Hongmin ZHOU, Ning OUYANG. Compressed sensing image reconstruction method fusing spatial location and structure information[J]. Journal of Computer Applications, 2022, 42(3): 930-937.

林乐平, 周宏敏, 欧阳宁. 融合空间位置与结构信息的压缩感知图像重建方法[J]. 《计算机应用》唯一官方网站, 2022, 42(3): 930-937.

Figures/Tables 11

Fig.1 Network structure

Fig. 2 Structure feature fusion module

Fig.3 Six test images used in experiments

Fig.4 Reconstruction result comparison of different methods with sampling rate of 0.10

Fig.5 Reconstruction result comparison of different methods with sampling rate of 0.05

Fig.6 PSNR line charts of 6 test images of each method at different sampling rates

Fig. 7 SSIM line charts of 6 test images of each method at different sampling rates

Tab. 1 Comparison of average PSNR and average SSIM among different methods on 6 test images

方法	SR=0.01		SR=0.05		SR=0.10		SR=0.15
方法	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
TVAL3	13.17	0.310 4	20.02	0.509 7	22.86	0.639 6	24.91	0.724 5
D-AMP	5.11	0.007 6	17.13	0.433 9	20.98	0.571 4	23.99	0.697 8
ReconNet	17.84	0.427 6	21.43	0.578 5	24.15	0.698 4	25.65	0.754 3
NL-MRN	17.78	0.430 5	21.95	0.613 7	24.68	0.721 4	26.61	0.788 8
SLSI	19.46	0.4693	22.75	0.6415	25.60	0.7675	27.19	0.8135

Tab. 2 Comparison of average SSIM of different methods on Set11 dataset

方法	SR值
方法	0.01	0.04	0.10
ReconNet	0.408 3	0.526 6	0.641 6
DR²-Net	0.429 1	0.580 4	0.717 4
MSRNet	0.453 5	0.616 7	0.759 8
SLSI	0.494 3	0.629 7	0.773 2

Fig.8 Training results of ablation experiment at sampling rate of 0.01

Fig.9 Comparison of reconstruction images of ablation experiments

References 24

1	CANDES E J， WAKIN M B. An introduction to compressive sampling ［J］. IEEE Signal Processing Magazine， 2008， 25（2）： 21-30. 10.1109/msp.2007.914731
2	陈伟业，孙权森. 结合压缩感知与非局部信息的图像超分辨率重建［J］. 计算机应用， 2016， 36（9）： 2570-2575. 10.11772/j.issn.1001-9081.2016.09.2570
	CHEN W Y， SUN Q S. Image super-resolution reconstruction combined with compressed sensing and nonlocal information ［J］. Journal of Computer Applications， 2016， 36（9）： 2570-2575. 10.11772/j.issn.1001-9081.2016.09.2570
3	LIN L P， FANG L， JIAO L C. Geometric structure guided collaborative compressed sensing ［J］. Signal Processing： Image Communication， 2016， 40：16-25. 10.1016/j.image.2015.10.006
4	SUN Z Y， WANG H H， LIU B L， et al. CS-FCDA： a compressed sensing-based on fault-tolerant data aggregation in sensor networks ［J］. Sensors， 2018， 18（11）： 3749. 10.3390/s18113749
5	YANG J， LU X， SU W， et al. Multistatic inverse synthetic aperture radar imaging based on parametric block-sparse reconstruction ［J］. Journal of Applied Remote Sensing， 2020， 14（2）：1. 10.1117/1.jrs.14.026501
6	GAO X W， ZHANG J， CHE W， et al. Block-based compressive sensing coding of natural images by local structural measurement matrix ［C］// Proceedings of 2015 the Data Compression Conference. Piscataway： IEEE， 2015：133-142. 10.1109/dcc.2015.47
7	KULKARNI K， LOHIT S， TURAGA P， et al. ReconNet： non-iterative reconstruction of images from compressively sensed measurements ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington， DC： IEEE Computer Society， 2016： 449-458. 10.1109/cvpr.2016.55
8	YAO H， DAI F， ZHANG D， et al. DR²-Net： deep residual reconstruction network for image compressive sensing ［J］. Neurocomputing， 2019， 359： 483-493. 10.1016/j.neucom.2019.05.006
9	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington， DC： IEEE Computer Society， 2016： 770-778. 10.1109/cvpr.2016.90
10	LI W， LIU F， JIAO L， et al. Multi-Scale residual reconstruction neural network with non-local constraint ［J］. IEEE Access， 2019， 7： 70910-70918. 10.1109/access.2019.2918593
11	KAI X， ZHANG Z， REN F B. LAPRAN： a scalable laplacian pyramid reconstructive adversarial network for flexible compressive sensing reconstruction ［C］// Proceedings of the 2018 European Conference on Computer Vision. Cham： Springer， 2018： 485-500. 10.1007/978-3-030-01249-6_30
12	练秋生，富利鹏，陈书贞，等.基于多尺度残差网络的压缩感知重建方法［J］.自动化学报，2019，45（11）：2082-2091.
	LIAN Q S， FU L P， CHEN S Z， et al. A compressed sensing algorithm based on multi-scale residual reconstruction network ［J］. Acta Automatica Sinica， 2019， 45（11）： 2082-2091.
13	HUANG H， NIE G， ZHENG Y， et al. Image restoration from patch-based compressed sensing measurement ［J］. Neurocomputing， 2019， 340： 145-157. 10.1016/j.neucom.2019.02.036
14	杜秀丽，张薇，陈波.基于波浪式矩阵置换的稀疏度均衡分块压缩感知算法［J］.计算机应用，2018，38（12）：3541-3546. 10.11772/j.issn.1001-9081.2018051039
	DU X L， ZHANG W， CHEN B. Ripple matrix permutation-based sparsity balanced block compressed sensing algorithm ［J］. Journal of Computer Applications， 2018，38（12）：3541-3546. 10.11772/j.issn.1001-9081.2018051039
15	JIE H， LI S， GANG S， et al. Squeeze-and-excitation networks ［C］// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
16	KRIZHEVSKY A， SUTSKEVER I， HINTONG E. ImageNet classification with deep convolutional neural networks ［C］// Advances in Neural Information Processing Systems， 2012， 25： 1097-1105.
17	欧阳宁，梁婷，林乐平.基于自注意力网络的图像超分辨率重建［J］.计算机应用，2019，39（8）：2391-2395. 10.11772/j.issn.1001-9081.2019010158
	OUYANG N， LIANG T， LIN L P. Self-attention network based image super-resolution ［J］. Journal of Computer Applications， 2019，39（8）：2391-2395. 10.11772/j.issn.1001-9081.2019010158
18	CHEN L C， ZHU Y， PAPANDREOU G， et al. Encoder-decoder with atrous separable convolution for semantic image segmentation ［C］// Proceedings of the 2018 European Conference on Computer Vision. Cham： Springer， 2018： 801-818. 10.1007/978-3-030-01234-2_49
19	王得成，陈向宁，易辉，等.基于自适应联合双边滤波的深度图像空洞填充与优化算法［J］.中国激光，2019，46（10）：1009002. 10.3788/CJL201946.1009002
	WANG D C， CHEN X N， YI H， et al. Hole filling and optimization algorithm for depth images based on adaptive joint bilateral filtering ［J］. Chinese Journal of Lasers， 2019， 46（10）： 1009002. 10.3788/CJL201946.1009002
20	MARTIN D， FOWLKES C， TAL D， et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics ［C］// Proceedings of the 8th IEEE International Conference on Computer Vision. Piscataway： IEEE， 2002： 416-423. 10.1109/iccv.2001.937655
21	KINGMA D P， BA J L. Adam： a method for stochastic optimization ［EB/OL］. ［2021-03-20］. .
22	HE K M， ZHANG X Y， REN S Q， et al. Delving deep into rectifiers： surpassing human-level performance on ImageNet classification ［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1026-1034. 10.1109/iccv.2015.123
23	LI C， YIN W， JIANG H， et al. An efficient augmented Lagrangian method with applications to total variation minimization ［J］. Computational Optimization and Applications， 2013， 56（3）： 507-530. 10.1007/s10589-013-9576-1
24	METZLER C A， MALEKI A， BARANIUK R G. From denoising to compressed sensing ［J］. IEEE Transactions on Information Theory， 2016， 62（9）： 5117-5144. 10.1109/tit.2016.2556683

[1]	Wenqiu ZHU, Guang ZOU, Zhigao ZENG. Object tracking algorithm with hierarchical features and hybrid attention [J]. Journal of Computer Applications, 2022, 42(3): 833-843.
[2]	Dingkang YANG, Shuai HUANG, Shunli WANG, Peng ZHAI, Yidan LI, Lihua ZHANG. EE-GAN：facial expression recognition method based on generative adversarial network and network integration [J]. Journal of Computer Applications, 2022, 42(3): 750-756.
[3]	Tianmin DENG, Guotao MAO, Zhenhao ZHOU, Zhijian DUAN. Road vehicle detection and recognition algorithm based on densely connected convolutional neural network [J]. Journal of Computer Applications, 2022, 42(3): 883-889.
[4]	Lu ZHANG, Chun FANG, Ming ZHU. Indoor fall detection algorithm based on Res2Net-YOLACT and fusion feature [J]. Journal of Computer Applications, 2022, 42(3): 757-763.
[5]	Renzhi PAN, Fulan QIAN, Shu ZHAO, Yanping ZHANG. Recommendation model for user attribute preference modeling based on convolutional neural network interaction [J]. Journal of Computer Applications, 2022, 42(2): 404-411.
[6]	Pinxue WANG, Shaobing ZHANG, Miao CHENG, Lian HE, Xiaoshan QIN. Coin surface defect detection algorithm based on deformable convolution and adaptive spatial feature fusion [J]. Journal of Computer Applications, 2022, 42(2): 638-645.
[7]	Wei LI, Yaochi FAN, Qiaoyong JIANG, Lei WANG, Qingzheng XU. Variable convolutional autoencoder method based on teaching-learning-based optimization for medical image classification [J]. Journal of Computer Applications, 2022, 42(2): 592-598.
[8]	Runze WANG, Yueqin ZHANG, Qiqi QIN, Zehua ZHANG, Xumin GUO. Multi-aspect multi-attention fusion of molecular features for drug-target affinity prediction [J]. Journal of Computer Applications, 2022, 42(1): 325-332.
[9]	Hengxin LI, Kan CHANG, Yufei TAN, Mingyang LING, Tuanfa QIN. Color image demosaicking network based on inter-channel correlation and enhanced information distillation [J]. Journal of Computer Applications, 2022, 42(1): 245-251.
[10]	Yinxin BAO, Yang CAO, Quan SHI. Improved spatio-temporal residual convolutional neural network for urban road network short-term traffic flow prediction [J]. Journal of Computer Applications, 2022, 42(1): 258-264.
[11]	Huiqing XU, Bin CHEN, Jingfei WANG, Zhiyi CHEN, Jian QIN. Elongated pavement distress detection method based on convolutional neural network [J]. Journal of Computer Applications, 2022, 42(1): 265-272.
[12]	SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion [J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.
[13]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[14]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[15]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.

Compressed sensing image reconstruction method fusing spatial location and structure information

融合空间位置与结构信息的压缩感知图像重建方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 24

Related Articles 15

Recommended Articles

Metrics