关键词: 电子稳像, 变分模态分解, 长短期记忆网络, 大平移抖动, 深度学习

Abstract:

In the process of collecting images， mountain mounted photoelectric systems often encounter large translation random jitter， resulting in video blurring and poor stability. Algorithms such as mean filtering and wavelet transform usually establish mathematical models based on the physical characteristics of inter-frame motion （frequency， amplitude， etc.）. Aiming at the problem that such algorithms were usually used based on a priori filter operator， which lacks a certain degree of adaptability and are difficult to apply to complex digital image stabilization applications， an adaptive digital image stabilization algorithm based on Variational Mode Decomposition （VMD） and Long Short-Term Memory （LSTM） network was proposed. By adaptively decomposing the global motion vector sequence based on frequency information， a series of Intrinsic Mode Functions （IMFs） with narrow band characteristics were obtained； at the same time， combining the temporal and spatial domain information of IMFs， using the IMFs as training variables， an LSTM network model was established to classify the IMFs， to screen out intentional motion dominated IMF， and reconstruct intentional motion vector sequences to achieve video stability. Experimental results show that compared with algorithms such as mean filtering and wavelet transform， the proposed algorithm has the highest classification accuracy （minimum 91.4%）. Through the comprehensive evaluation given by a deep learning network on frequency， amplitude， spatiotemporal domain information， and etc.， the proposed algorithm has a significant improvement effect on large translation jitter image stabilization， and has better robustness.

Key words: digital image stabilization, variational mode decomposition, Long and Short-Term Memory (LSTM) network, large translation jitter, deep learning