Parallel decompression algorithm for high-speed train monitoring data

doi:10.11772/j.issn.1001-9081.2020111173

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (9): 2586-2593.DOI: 10.11772/j.issn.1001-9081.2020111173

Special Issue: 数据科学与技术

• Data science and technology • Previous Articles Next Articles

Parallel decompression algorithm for high-speed train monitoring data

WANG Zhoukai, ZHANG Jiong, MA Weigang, WANG Huaijun

Faculty of Computer Science and Engineering, Xi'an University of Technology, Xi'an Shaanxi 710048, China

Received:2020-08-04 Revised:2021-01-12 Online:2021-09-10 Published:2021-05-08
Supported by:
This work is partially supported by the National Key Research and Development Program of China (2018YFB1703000), the NSFC-High Speed Rail Joint Fund Key Project (U173410), the Special Project of Scientific Research Plan of Shaanxi Provincial Department of Education (21JK0781).

面向高速列车监测数据的并行解压缩算法

王周恺, 张炯, 马维纲, 王怀军

西安理工大学计算机科学与工程学院, 西安 710048

通讯作者: 王周恺
作者简介:王周恺(1989-),男,陕西安康人,讲师,博士,主要研究方向:并行计算;张炯(1994-),男,山西大同人,硕士研究生,主要研究方向:高性能计算;马维纲(1976-),男,甘肃兰州人,副教授,博士,主要研究方向:编译优化;王怀军(1981-),男,山东滕州人,副教授,博士,主要研究方向:边缘计算。
基金资助:
国家重点研发计划项目（2018YFB1703000）；国家自然科学基金-高速铁路联合基金重点项目（U173410）；陕西省教育厅科学研究计划专项（21JK0781）。

Abstract

Abstract: The real-time monitoring data generated by high-speed trains during running are usually processed by variable-length coding compression technology, which is convenient for transmission and storage. However, this method will complicate the internal structure of the compressed data, so that the corresponding data decompression process must follow the composition order of the compressed data, which is inefficient. In order to improve the decompression efficiency of high-speed train monitoring data, a parallel decompression algorithm for high-speed train monitoring data was proposed with the help of the speculation technology. Firstly, the structural characteristics of high-speed train monitoring data were studied, and the internal dependence that affects data division was analyzed. Secondly, the speculation technology was used to clean up internal dependence, and then, the data were divided into different parts tentatively. Thirdly, the division results were decompressed in a distributed computing environment in parallel. Finally, the parallel decompression results were combined together. Through this way, the decompression efficiency of high-speed train monitoring data was improved. Experimental results showed that on the computing cluster composed of 7 computing nodes, compared with the serial algorithm, the speedup of the proposed speculative parallel algorithm was about 3, showing a good performance of this algorithm. It can be seen that this algorithm can improve the monitoring data decompression efficiency significantly.

Key words: high-speed train, variable-length coding, decompression, speculative parallelism, distributed computing, parallelization

摘要： 高速列车在运行时产生的实时监测数据通常用变长编码压缩技术进行处理，以便于传输和存储。然而这种方法会使得压缩数据内部结构复杂，导致相应的数据解压缩过程只能遵照压缩数据的组成顺序进行，效率较低。为提升高速列车监测数据的解压缩效率，借助推测技术，提出一种面向高速列车监测数据的并行解压缩算法。首先，研究高速列车监测数据的结构特征，分析影响数据划分的内部依赖；其次，利用推测技术消解内部依赖后，对数据进行试探性划分；然后在分布式计算环境中对划分结果并行地进行解压；最后将并行解压缩结果合并起来，从而提高针对高速列车监测数据的解压缩效率。实验结果表明，在由7个计算节点组成的计算集群上，与串行算法相比，所提推测并行算法的加速比为3左右，展现了该算法良好的性能，可见该算法能够显著提高针对列车监测数据的解压缩效率。

关键词: 高速列车, 变长编码, 解压缩, 推测并行, 分布式计算, 并行化

CLC Number:

TP311

WANG Zhoukai, ZHANG Jiong, MA Weigang, WANG Huaijun. Parallel decompression algorithm for high-speed train monitoring data[J]. Journal of Computer Applications, 2021, 41(9): 2586-2593.

王周恺, 张炯, 马维纲, 王怀军. 面向高速列车监测数据的并行解压缩算法[J]. 计算机应用, 2021, 41(9): 2586-2593.

References

[1] SMITH K. Beyond GSM-R:the future of railway radio[J]. International Railway Journal,2017,57(3):30-32,34.
[2] 徐效宁, 李一楠, 李辉, 等. 融合轨道电路信息的CTCS-4级列控系统研究[J]. 铁道工程学报,2018,35(11):82-86.(XU X N, LI Y N,LI H,et al. Research on the CTCS-4 Train control system based on track circuit information fusion[J]. Journal of Railway Engineering Society,2018,35(11):82-86.)
[3] 韩哲, 张霞, 李鸥, 等. 面向有损链路的传感网压缩感知数据收集算法[J]. 软件学报,2017,28(12):3257-3273.(HAN Z, ZHANG X, LI O, et al. Data gathering algorithm based on compressive sensing under lossy WSN[J]. Journal of Software, 2017,28(12):3257-3273.)
[4] 罗瑜, 张珍珍. 一种方向插值预测变长编码的帧存有损压缩算法[J]. 电子与信息学报,2019,41(10):2495-2500.(LUO Y, ZHANG Z Z. A lossy frame memory compression algorithm using directional interpolation prediction variable length coding[J]. Journal of Electronics and Information Technology,2019,41(10):2495-2500.)
[5] WEN L L,ZHOU K L,YANG S L,et al. Compression of smart meter big data:a survey[J]. Renewable and Sustainable Energy Reviews,2018,91:59-69.
[6] 周东华, 纪洪泉, 何潇. 高速列车信息控制系统的故障诊断技术[J]. 自动化学报,2018,44(7):1153-1164.(ZHOU D H,JI H Q,HE X. Fault diagnosis techniques for the information control system of high-speed trains[J]. Acta Automatica Sinica,2018,44(7):1153-1164.)
[7] 程剑锋, 赵显琼, 刘磊. CTCS-4级列控系统关键技术研究[J]. 北京交通大学学报,2016,40(5):104-110.(CHENG J F,ZHAO X Q,LIU L. Research on key technologies of CTCS-4 level train control system[J]. Journal of Beijing Jiaotong University,2016,40(5):104-110.)
[8] 李志文. 基于全球导航卫星系统(GNSS)的列车运行状态监控系统研究[J]. 铁路通信信号工程技术,2017,14(6):53-55.(LI Z W. Research on GNSS-based train operation status monitoring system[J]. Railway Signalling and Communication Engineering, 2017,14(6):53-55.)
[9] 马新娜, 施文锐. 高速列车状态监测大数据的预警可视化分析研究[J]. 电子测量与仪器学报,2019,33(7):21-27.(MA X N, SHI W R. Research on visual analysis of condition monitoring big data for high speed train[J]. Journal of Electronic Measurement and Instrumentation,2019,33(7):21-27.)
[10] 赵雅倩, 李龙, 郭跃超, 等. 基于OpenCL的Gzip数据压缩算法[J]. 计算机应用,2018,38(S1):112-115,130.(ZHAO Y Q, LI L,GUO Y C,et al. OpenCL-based optimization for Gzip algorithm[J]. Journal of Computer Applications,2018,38(S1):112-115,130.)
[11] 杨仁忠, 张洁, 韦宏卫, 等. 基于GPU的Landsat8实时解压缩处理技术[J]. 计算机工程,2016,42(3):301-307.(YANG R Z, ZHANG J,WEI H W,et al. Real-time decompression processing technology of Landsat8 based on GPU[J]. Computer Engineering, 2016,42(3):301-307.)
[12] FUNASAKA S,NAKANO K,ITO Y. Fully parallelized LZW decompression for CUDA-enabled GPUs[J]. IEICE Transactions on Information and Systems,2016,E 99. D(12):2986-2994.
[13] 马志强, 李海生. 基于KD-tree剖分的三维动态场景快速有效压缩[J]. 计算机应用,2016,36(9):2590-2596.(MA Z Q,LI H S. Fast and effective compression for 3D dynamic scene based on KD-tree division[J]. Journal of Computer Applications,2016,36(9):2590-2596.)
[14] LAI W K,CHEN Y U,WU T Y,et al. Towards a framework for large-scale multimedia data storage and processing on Hadoop platform[J]. The Journal of Supercomputing,2014,68(1):488-507.
[15] 薛帅, 王光霞, 郭建忠, 等. 顾及最大绝对误差的频率域矢量数据压缩算法[J]. 武汉大学学报(信息科学版),2018,43(9):1438-1444.(XUE S,WANG G X,GUO J Z,et al. Vector map data compression of frequency domain with consideration of maximum absolute error[J]. Geomatics and Information Science of Wuhan University,2018,43(9):1438-1444.)
[16] 韩立敏, 田泽, 张骏, 等. 图形处理器流水线数据压缩技术研究综述[J]. 计算机应用研究,2018,35(3):648-653.(HAN L M, TIAN Z,ZHANG J,et al. Survey of data compression techniques for GPU pipeline[J]. Application Research of Computers,2018, 35(3):648-653.)
[17] THANOU D, CHOU P A, FROSSARD P. Graph-based compression of dynamic 3D point cloud sequences[J]. IEEE Transactions on Image Processing,2016,25(4):1765-1778.
[18] DI S, CAPPELLO F. Fast error-bounded lossy HPC data compression with SZ[C]//Proceedings of the 2016 IEEE International Parallel and Distributed Processing Symposium. Piscataway:IEEE,2016:730-739.
[19] 李功丽, 戴紫彬, 徐进辉, 等. 基于流体系结构的VLIW二维压缩及并行解压[J]. 电子学报,2017,45(9):2256-2262.(LI G L,DAI Z B,XU J H,et al. 2-D compression and parallel decoding of VLIW based on stream architecture[J]. Acta Electronica Sinica,2017,45(9):2256-2262.)

Parallel decompression algorithm for high-speed train monitoring data

面向高速列车监测数据的并行解压缩算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	HAN Junying, ZHANG Zhenyu, KONG Deshi. Distributed multi-task allocation method for user area in mobile crowd sensing [J]. Journal of Computer Applications, 2020, 40(2): 358-362.
[2]	SONG Xiangshuai, YANG Fuzhang, XIE Jiang, ZHANG Wu. Optimization and parallelization of Graphlet Degree Vector method [J]. Journal of Computer Applications, 2020, 40(2): 398-403.
[3]	GUO Liangmin, ZHU Ying, SUN Liping. k nearest neighbor query based on parallel ant colony algorithm in obstacle space [J]. Journal of Computer Applications, 2019, 39(3): 790-795.
[4]	LI Longyang, DONG Yihong, SHI Weijie, PAN Jianfei. SQM: subgraph matching algorithm for single large-scale graphs under Spark [J]. Journal of Computer Applications, 2019, 39(1): 46-50.
[5]	SUN Jiamin, ZHU Jiafu, YANG Fuzhang, XIE Jiang. Parallel algorithm of Markov clustering for large-scale biological networks [J]. Journal of Computer Applications, 2019, 39(1): 66-71.
[6]	YANG Fuzhang, ZHU Jiafu, SUN Jiamin, XIE Jiang. Parallel algorithm of biological complex network motifs discovery [J]. Journal of Computer Applications, 2019, 39(1): 72-77.
[7]	CUI Chen, ZHENG Linjiang, HAN Fengping, HE Mujun. Design of secondary indexes in HBase based on memory [J]. Journal of Computer Applications, 2018, 38(6): 1584-1590.
[8]	PAN Mingyu, ZHANG Lu, LONG Guobiao, LI Xianglong, MA Dongxue, XU Liang. Efficient block-based sampling algorithm for aggregation query processing on duplicate charged records [J]. Journal of Computer Applications, 2018, 38(6): 1596-1600.
[9]	ZHANG Chengchang, ZHANG Huayu, LUO Jianchang, HE Feng. Massive data analysis of power utilization based on improved K-means algorithm and cloud computing [J]. Journal of Computer Applications, 2018, 38(1): 159-164.
[10]	ZENG Qin, LI Yongsheng. Three dimensional strom tracking method based on distributed computing architecture [J]. Journal of Computer Applications, 2017, 37(4): 941-944.
[11]	LIU Youyao, YANG Pengcheng. Design and implementation of automatic C code parallelization based on JavaCC [J]. Journal of Computer Applications, 2016, 36(9): 2422-2426.
[12]	HAN Fengqing, SONG Zhijian, YU Rui. Fast deduplication for massive images [J]. Journal of Computer Applications, 2016, 36(7): 1797-1800.
[13]	ZENG Xuelin, WU Bin. Parallelized recommendation algorithm in location-based social network [J]. Journal of Computer Applications, 2016, 36(2): 316-323.
[14]	LIN Yang, JIANG Yu'e, LIN Jie. Local similarity detection algorithm for time series based on distributed architecture [J]. Journal of Computer Applications, 2016, 36(12): 3285-3291.
[15]	CHEN Zhen, XIA Jingbo, YANG Juan, WEI Zekun. Support vector machine situation assessment algorithm based on MapReduce [J]. Journal of Computer Applications, 2016, 36(1): 133-137.