Compressed sensing image reconstruction method fusing spatial location and structure information

doi:10.11772/j.issn.1001-9081.2021030434

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (3): 930-937.DOI: 10.11772/j.issn.1001-9081.2021030434

• Multimedia computing and computer simulation • Previous Articles Next Articles

Compressed sensing image reconstruction method fusing spatial location and structure information

Leping LIN¹^,², Hongmin ZHOU², Ning OUYANG¹^,²()

^1.Key Laboratory of Cognitive Radio and Information Processing，Ministry of Education（Guilin University of Electronic Technology），Guilin Guangxi 541004，China
^2.School of Information and Communication，Guilin University of Electronic Technology，Guilin Guangxi 541004，China

Received:2021-03-22 Revised:2021-06-12 Accepted:2021-06-17 Online:2022-04-09 Published:2022-03-10
Contact: Ning OUYANG
About author:LIN Leping， born in 1980， Ph. D.， associate professor. Her research interests include machine learning， intelligent information processing， image signal processing.
ZHOU Hongmin， born in 1996， M. S. candidate. Her research interests include pattern recognition， deep learning.
Supported by:
National Natural Science Foundation of China(62001133);Special Foundation for Scientific Bases and Talents of Guangxi(AD19110060);Guangxi One Thousand Young and Middle-aged College and University Backbone Teachers Cultivation Program, Fund of Guangxi Key Laboratory of Wireless Wideband Communication and Signal Processing(GXKL06200114)

融合空间位置与结构信息的压缩感知图像重建方法

林乐平¹^,², 周宏敏², 欧阳宁¹^,²()

^1.认知无线电与信息处理省部共建教育部重点实验室(桂林电子科技大学), 广西桂林 541004
^2.桂林电子科技大学信息与通信学院, 广西桂林 541004

通讯作者: 欧阳宁
作者简介:林乐平（1980—），女，广西桂平人，副教授，博士，主要研究方向：机器学习、智能信息处理、图像信号处理
周宏敏（1996—），女，湖南邵阳人，硕士研究生，主要研究方向：模式识别、深度学习；
基金资助:
国家自然科学基金资助项目(62001133);广西科技基地和人才专项(AD19110060);广西高等学校千名中青年骨干教师培育计划项目;广西无线宽带通信与信号处理重点实验室基金资助项目(GXKL06200114)

Abstract

Abstract:

Aiming at the problem of poor visual effects of block-based compressed sensing reconstructed images at low sampling rates， a compressed sensing image reconstruction method that fused Spatial Location and Structure Information （SLSI） was proposed. Firstly， observations were linearly mapped to obtain initial estimated values of image blocks. Then， based on block grouping reconstruction branch and whole image reconstruction branch， the spatial location information and structure information of the image were extracted， enhanced and fused. Finally， weighted strategy was used to fuse the outputs of the two branches to obtain final reconstructed whole image. In the block grouping reconstruction branch， reconstruction resources were allocated according to the data characteristics of the image blocks. In the whole image reconstruction branch， information exchange between adjacent image block pixels was mainly carried out through bilateral filtering and structural feature interaction module. Experimental results show that compared with compressed sensing reconstruction methods based on non-iterative Reconstruction Network （ReconNet） and Multi-scale Reconstruction neural Network with Non-Local constraint （NL-MRN）， due to the combination of the image prior with strong autocorrelation between pixels， when sampling rate is 0.05， the average Peak Signal-to-Noise Ratio （PSNR） and Structural SIMilarity index （SSIM） of the proposed method on the test image data commonly used in the compressed sensing field increase 2.617 5 dB and 0.105 3 respectively， and the visual effects of reconstructed images are better.

Key words: Convolutional Neural Network (CNN), compressed sensing, image reconstruction, feature fusion, image prior

摘要：

针对低采样率下分块压缩感知重建图像视觉效果不佳的问题，提出一种融合空间位置与结构信息的压缩感知图像重建方法（SLSI）。首先，对观测值进行线性映射得到图像块的初步估计值；然后，基于块分组重建支路和全图重建支路对图像的空间位置信息和结构信息进行提取、增强和融合；最后，通过加权策略融合双支路的输出得到最终重建全图。在块分组重建支路中，根据图像块的数据特点分配重建资源。在全图重建支路中，主要通过双边滤波和结构特征交互模块对相邻图像块像素进行信息交互。实验结果表明，与基于非迭代重建网络（ReconNet）、基于非局部约束的多尺度重建网络（NL-MRN）等压缩感知重建方法相比，由于结合了像素间强自相关性这种图像先验，在采样率为0.05的情况下，所提方法在压缩感知领域常用的测试图像数据上的峰值信噪比（PSNR）和结构相似度（SSIM）分别平均提升了2.617 5 dB和0.105 3，重建图像的视觉效果较好。

关键词: 卷积神经网络, 压缩感知, 图像重建, 特征融合, 图像先验

CLC Number:

TP391.41

Leping LIN, Hongmin ZHOU, Ning OUYANG. Compressed sensing image reconstruction method fusing spatial location and structure information[J]. Journal of Computer Applications, 2022, 42(3): 930-937.

林乐平, 周宏敏, 欧阳宁. 融合空间位置与结构信息的压缩感知图像重建方法[J]. 《计算机应用》唯一官方网站, 2022, 42(3): 930-937.

Figures/Tables 11

Fig.1 Network structure

Fig. 2 Structure feature fusion module

Fig.3 Six test images used in experiments

Fig.4 Reconstruction result comparison of different methods with sampling rate of 0.10

Fig.5 Reconstruction result comparison of different methods with sampling rate of 0.05

Fig.6 PSNR line charts of 6 test images of each method at different sampling rates

Fig. 7 SSIM line charts of 6 test images of each method at different sampling rates

Tab. 1 Comparison of average PSNR and average SSIM among different methods on 6 test images

方法	SR=0.01		SR=0.05		SR=0.10		SR=0.15
方法	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
TVAL3	13.17	0.310 4	20.02	0.509 7	22.86	0.639 6	24.91	0.724 5
D-AMP	5.11	0.007 6	17.13	0.433 9	20.98	0.571 4	23.99	0.697 8
ReconNet	17.84	0.427 6	21.43	0.578 5	24.15	0.698 4	25.65	0.754 3
NL-MRN	17.78	0.430 5	21.95	0.613 7	24.68	0.721 4	26.61	0.788 8
SLSI	19.46	0.4693	22.75	0.6415	25.60	0.7675	27.19	0.8135

Tab. 2 Comparison of average SSIM of different methods on Set11 dataset

方法	SR值
方法	0.01	0.04	0.10
ReconNet	0.408 3	0.526 6	0.641 6
DR²-Net	0.429 1	0.580 4	0.717 4
MSRNet	0.453 5	0.616 7	0.759 8
SLSI	0.494 3	0.629 7	0.773 2

Fig.8 Training results of ablation experiment at sampling rate of 0.01

Fig.9 Comparison of reconstruction images of ablation experiments

References 24

1	CANDES E J， WAKIN M B. An introduction to compressive sampling ［J］. IEEE Signal Processing Magazine， 2008， 25（2）： 21-30. 10.1109/msp.2007.914731
2	陈伟业，孙权森. 结合压缩感知与非局部信息的图像超分辨率重建［J］. 计算机应用， 2016， 36（9）： 2570-2575. 10.11772/j.issn.1001-9081.2016.09.2570
	CHEN W Y， SUN Q S. Image super-resolution reconstruction combined with compressed sensing and nonlocal information ［J］. Journal of Computer Applications， 2016， 36（9）： 2570-2575. 10.11772/j.issn.1001-9081.2016.09.2570
3	LIN L P， FANG L， JIAO L C. Geometric structure guided collaborative compressed sensing ［J］. Signal Processing： Image Communication， 2016， 40：16-25. 10.1016/j.image.2015.10.006
4	SUN Z Y， WANG H H， LIU B L， et al. CS-FCDA： a compressed sensing-based on fault-tolerant data aggregation in sensor networks ［J］. Sensors， 2018， 18（11）： 3749. 10.3390/s18113749
5	YANG J， LU X， SU W， et al. Multistatic inverse synthetic aperture radar imaging based on parametric block-sparse reconstruction ［J］. Journal of Applied Remote Sensing， 2020， 14（2）：1. 10.1117/1.jrs.14.026501
6	GAO X W， ZHANG J， CHE W， et al. Block-based compressive sensing coding of natural images by local structural measurement matrix ［C］// Proceedings of 2015 the Data Compression Conference. Piscataway： IEEE， 2015：133-142. 10.1109/dcc.2015.47
7	KULKARNI K， LOHIT S， TURAGA P， et al. ReconNet： non-iterative reconstruction of images from compressively sensed measurements ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington， DC： IEEE Computer Society， 2016： 449-458. 10.1109/cvpr.2016.55
8	YAO H， DAI F， ZHANG D， et al. DR²-Net： deep residual reconstruction network for image compressive sensing ［J］. Neurocomputing， 2019， 359： 483-493. 10.1016/j.neucom.2019.05.006
9	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington， DC： IEEE Computer Society， 2016： 770-778. 10.1109/cvpr.2016.90
10	LI W， LIU F， JIAO L， et al. Multi-Scale residual reconstruction neural network with non-local constraint ［J］. IEEE Access， 2019， 7： 70910-70918. 10.1109/access.2019.2918593
11	KAI X， ZHANG Z， REN F B. LAPRAN： a scalable laplacian pyramid reconstructive adversarial network for flexible compressive sensing reconstruction ［C］// Proceedings of the 2018 European Conference on Computer Vision. Cham： Springer， 2018： 485-500. 10.1007/978-3-030-01249-6_30
12	练秋生，富利鹏，陈书贞，等.基于多尺度残差网络的压缩感知重建方法［J］.自动化学报，2019，45（11）：2082-2091.
	LIAN Q S， FU L P， CHEN S Z， et al. A compressed sensing algorithm based on multi-scale residual reconstruction network ［J］. Acta Automatica Sinica， 2019， 45（11）： 2082-2091.
13	HUANG H， NIE G， ZHENG Y， et al. Image restoration from patch-based compressed sensing measurement ［J］. Neurocomputing， 2019， 340： 145-157. 10.1016/j.neucom.2019.02.036
14	杜秀丽，张薇，陈波.基于波浪式矩阵置换的稀疏度均衡分块压缩感知算法［J］.计算机应用，2018，38（12）：3541-3546. 10.11772/j.issn.1001-9081.2018051039
	DU X L， ZHANG W， CHEN B. Ripple matrix permutation-based sparsity balanced block compressed sensing algorithm ［J］. Journal of Computer Applications， 2018，38（12）：3541-3546. 10.11772/j.issn.1001-9081.2018051039
15	JIE H， LI S， GANG S， et al. Squeeze-and-excitation networks ［C］// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
16	KRIZHEVSKY A， SUTSKEVER I， HINTONG E. ImageNet classification with deep convolutional neural networks ［C］// Advances in Neural Information Processing Systems， 2012， 25： 1097-1105.
17	欧阳宁，梁婷，林乐平.基于自注意力网络的图像超分辨率重建［J］.计算机应用，2019，39（8）：2391-2395. 10.11772/j.issn.1001-9081.2019010158
	OUYANG N， LIANG T， LIN L P. Self-attention network based image super-resolution ［J］. Journal of Computer Applications， 2019，39（8）：2391-2395. 10.11772/j.issn.1001-9081.2019010158
18	CHEN L C， ZHU Y， PAPANDREOU G， et al. Encoder-decoder with atrous separable convolution for semantic image segmentation ［C］// Proceedings of the 2018 European Conference on Computer Vision. Cham： Springer， 2018： 801-818. 10.1007/978-3-030-01234-2_49
19	王得成，陈向宁，易辉，等.基于自适应联合双边滤波的深度图像空洞填充与优化算法［J］.中国激光，2019，46（10）：1009002. 10.3788/CJL201946.1009002
	WANG D C， CHEN X N， YI H， et al. Hole filling and optimization algorithm for depth images based on adaptive joint bilateral filtering ［J］. Chinese Journal of Lasers， 2019， 46（10）： 1009002. 10.3788/CJL201946.1009002
20	MARTIN D， FOWLKES C， TAL D， et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics ［C］// Proceedings of the 8th IEEE International Conference on Computer Vision. Piscataway： IEEE， 2002： 416-423. 10.1109/iccv.2001.937655
21	KINGMA D P， BA J L. Adam： a method for stochastic optimization ［EB/OL］. ［2021-03-20］. .
22	HE K M， ZHANG X Y， REN S Q， et al. Delving deep into rectifiers： surpassing human-level performance on ImageNet classification ［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1026-1034. 10.1109/iccv.2015.123
23	LI C， YIN W， JIANG H， et al. An efficient augmented Lagrangian method with applications to total variation minimization ［J］. Computational Optimization and Applications， 2013， 56（3）： 507-530. 10.1007/s10589-013-9576-1
24	METZLER C A， MALEKI A， BARANIUK R G. From denoising to compressed sensing ［J］. IEEE Transactions on Information Theory， 2016， 62（9）： 5117-5144. 10.1109/tit.2016.2556683

[1]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[2]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[3]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[4]	Dongwei WANG, Baichen LIU, Zhi HAN, Yanmei WANG, Yandong TANG. Deep network compression method based on low-rank decomposition and vector quantization [J]. Journal of Computer Applications, 2024, 44(7): 1987-1994.
[5]	Yangyi GAO, Tao LEI, Xiaogang DU, Suiyong LI, Yingbo WANG, Chongdan MIN. Crowd counting and locating method based on pixel distance map and four-dimensional dynamic convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2233-2242.
[6]	Ruihua LIU, Zihe HAO, Yangyang ZOU. Gait recognition algorithm based on multi-layer refined feature fusion [J]. Journal of Computer Applications, 2024, 44(7): 2250-2257.
[7]	Mengyuan HUANG, Kan CHANG, Mingyang LING, Xinjie WEI, Tuanfa QIN. Progressive enhancement algorithm for low-light images based on layer guidance [J]. Journal of Computer Applications, 2024, 44(6): 1911-1919.
[8]	Jianjing LI, Guanfeng LI, Feizhou QIN, Weijun LI. Multi-relation approximate reasoning model based on uncertain knowledge graph embedding [J]. Journal of Computer Applications, 2024, 44(6): 1751-1759.
[9]	Yue LIU, Fang LIU, Aoyun WU, Qiuyue CHAI, Tianxiao WANG. 3D object detection network based on self-attention mechanism and graph convolution [J]. Journal of Computer Applications, 2024, 44(6): 1972-1977.
[10]	Guijin HAN, Xinyuan ZHANG, Wentao ZHANG, Ya HUANG. Self-supervised image registration algorithm based on multi-feature fusion [J]. Journal of Computer Applications, 2024, 44(5): 1597-1604.
[11]	Min SUN, Qian CHENG, Xining DING. CBAM-CGRU-SVM based malware detection method for Android [J]. Journal of Computer Applications, 2024, 44(5): 1539-1545.
[12]	Hongtian LI, Xinhao SHI, Weiguo PAN, Cheng XU, Bingxin XU, Jiazheng YUAN. Few-shot object detection via fusing multi-scale and attention mechanism [J]. Journal of Computer Applications, 2024, 44(5): 1437-1444.
[13]	Wenshuo GAO, Xiaoyun CHEN. Point cloud classification network based on node structure [J]. Journal of Computer Applications, 2024, 44(5): 1471-1478.
[14]	Xin LI, Qiao MENG, Junyi HUANGFU, Lingchen MENG. YOLOv5 multi-attribute classification based on separable label collaborative learning [J]. Journal of Computer Applications, 2024, 44(5): 1619-1628.
[15]	Tianhua CHEN, Jiaxuan ZHU, Jie YIN. Bird recognition algorithm based on attention mechanism [J]. Journal of Computer Applications, 2024, 44(4): 1114-1120.

Compressed sensing image reconstruction method fusing spatial location and structure information

融合空间位置与结构信息的压缩感知图像重建方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 24

Related Articles 15

Recommended Articles

Metrics