Abstract:

The existing stock prediction models are mainly based on unimodal， and ignore inter-industry linkage effects and information heterogeneity. Although some studies have introduced textual modalities， they still struggle with challenges such as time lag effects and multi-granularity caused by modality inconsistency. Therefore， MATCH（Multimodal stock prediction frAmework inTegrating time-frequenCy features and Hybrid text）， a multimodal fusion framework for stock prediction that integrates heterogeneous information across modalities effectively was proposed. Specifically， a Mixture of Experts （MoE） pretraining strategy was designed to build industry-specific pretrained models of representations， so as to select matched expert networks dynamically and incorporate industry features information. At the same time， a frequency-domain decomposition and hierarchical fusion mechanism was designed to jointly model temporal patterns at multiple frequencies， and a dual-stream pretraining architecture was used to obtain representations of high-frequency future fluctuations and low-frequency future trends， which were interacted with text information across multiple time scales cross-modally， thereby capturing market dynamics more precisely and enabling effective interaction between time-series and textual data in multi-granular scenarios. Experimental results on two real-world stock datasets， S&P 500 and CMIN-US for comparing MATCH and mainstream methods such as ESTIMATE （Efficient STock Integration with teMporal generative filters and wavelet hypergraph ATtEntions） and PatchTST demonstrate that， on S&P 500 dataset， MATCH has the Sharpe Ratio （SR） improved by 50.5% over the sub-optimal baseline model Adv-ALSTM， while on the more challenging CMIN-US dataset， MATCH achieves a 2.35% SR improvement， with other metrics reaching the best. It can be seen that MATCH provides a novel and efficient solution for multimodal data fusion in finance.

Key words: financial time series, multimodal, Mixture of Experts (MoE) model, pretrained model, time-frequency analysis

摘要：

现有股票预测模型多基于单一模态，忽视了行业间的联动效应与信息异质性；部分研究虽引入了文本模态，但在处理模态异构所导致的时滞性和多粒度等问题上仍存在不足。因此，提出面向股票市场的融合时频特征与混合文本的多模态股票预测框架MATCH（Multimodal stock prediction frAmework inTegrating time-frequenCy features and Hybrid text）。一方面，设计混合专家（MoE）预训练策略为每个行业构建特定的预训练表征模型，在预测过程中动态选择匹配的专家网络，并注入行业特征信息；另一方面，设计频域分解与层次化融合机制，通过双流预训练架构获取高频未来波动和低频未来趋势的表征，把它们与不同时间尺度的文本信息进行跨模态交互，更精准地捕捉市场动态变化，并实现多粒度场景下的时序与文本有效交互。在2个真实股票数据集S&P 500和CMIN-US上，MATCH与ESTIMATE（Efficient STock Integration with teMporal generative filters and wavelet hypergraph ATtEntions）和PatchTST等主流方法进行对比的实验结果显示，在S&P 500数据集上相较次优基线模型Adv-ALSTM，MATCH的夏普比率（SR）提升了50.5%；在更具有挑战性的CMIN-US数据集上，MATCH的SR提升了2.35%，其余指标均取得了最佳成绩。MATCH预测性能提升明显可为金融多模态数据融合提供新颖且高效的解决方案。

关键词: 金融时间序列, 多模态, 混合专家模型, 预训练模型, 时频分析