《计算机应用》唯一官方网站 ›› 2026, Vol. 46 ›› Issue (4): 1034-1041.DOI: 10.11772/j.issn.1001-9081.2025040452

• 人工智能 • 上一篇    下一篇

高速数据流下无边界在线稀疏连续学习方法

韩雨晨1,2,3, 徐峰磊1, 吕凡4, 姚睿5, 胡伏原1,2,3()   

  1. 1.苏州科技大学 电子与信息工程学院,江苏 苏州 215009
    2.江苏省工业智能低碳技术工程研究中心,江苏 苏州 215009
    3.苏州市智能低碳技术应用重点实验室(苏州科技大学),江苏 苏州 215009
    4.中国科学院 自动化研究所,北京 100190
    5.中国矿业大学 计算机科学与技术学院,江苏 徐州 221116
  • 收稿日期:2025-04-23 修回日期:2025-07-29 接受日期:2025-07-31 发布日期:2025-08-11 出版日期:2026-04-10
  • 通讯作者: 胡伏原
  • 作者简介:韩雨晨(2000—),女,江苏南京人,硕士研究生,CCF会员,主要研究方向:机器学习、连续学习
    徐峰磊(1992—),男,江苏苏州人,副教授,博士,CCF会员,主要研究方向:模式识别、智能系统
    吕凡(1993—),男,江苏宿迁人,博士,CCF会员,主要研究方向:连续学习
    姚睿(1982—),男,河南南阳人,教授,博士,CCF会员,主要研究方向:计算机视觉、模式识别、深度学习、人工智能
  • 基金资助:
    国家自然科学基金资助项目(62476189)

Task-free online sparse continual learning method for high-speed data streams

Yuchen HAN1,2,3, Fenglei XU1, Fan LYU4, Rui YAO5, Fuyuan HU1,2,3()   

  1. 1.School of Electronic and Information Engineering,Suzhou University of Science and Technology,Suzhou Jiangsu 215009,China
    2.Jiangsu Industrial Intelligent and Low-carbon Technology Engineering Research Center,Suzhou Jiangsu 215009,China
    3.Suzhou Key Laboratory of Intelligent and Low-carbon Technology Application (Suzhou University of Science and Technology),Suzhou Jiangsu 215009,China
    4.Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China
    5.School of Computer Science and Technology,China University of Mining and Technology,Xuzhou Jiangsu 221116,China
  • Received:2025-04-23 Revised:2025-07-29 Accepted:2025-07-31 Online:2025-08-11 Published:2026-04-10
  • Contact: Fuyuan HU
  • About author:HAN Yuchen, born in 2000, M. S. candidate. Her research interests include machine learning, continual learning.
    XU Fenglei, born in 1992, Ph. D., associate professor. His research interests include pattern recognition, intelligent systems.
    LYU Fan, born in 1993, Ph. D. His research interests include continual learning.
    YAO Rui, born in 1982, Ph. D., professor. His research interests include computer vision, pattern recognition, deep learning, artificial intelligence.
  • Supported by:
    National Natural Science Foundation of China(62476189)

摘要:

无边界在线连续学习是一种任务无关的自主高效机器学习方法,通过连续适应新数据和抑制遗忘实现模型的动态更新。现有在线连续学习方法通常追求模型准确率而牺牲计算效率,这使得在高速数据流场景中,模型因训练速度滞后,难以及时响应数据流的变化。针对以上问题,提出一种面向效率?性能协同优化的在线稀疏连续学习框架,通过构建双向稀疏自适应调控机制突破传统方法的瓶颈。首先,设计参数重要性度量的动态稀疏拓扑优化框架,融合参数敏感性分析,实现非结构化参数剪枝;其次,建立记忆?效率双目标优化模型,基于在线类别分布估计动态调节计算预算分配,实现计算资源的最优配置;最后,构建梯度解耦优化策略,采用梯度掩码方法实现新旧知识的双向优化,在加速模型更新的同时保持知识拓扑的完整性。基准测试结果表明,所提框架展现出显著优势,当内存缓冲区大小为100时,相较于基线ER(Experience Replay),在CIFAR-10数据集上,平均在线准确率(AOA)和测试准确率(TA)分别提升了4.86%和6.25%;在CIFAR-100数据集上,AOA和TA分别提升了13.77%和3.08%;在Mini-ImageNet数据集上,AOA和TA的提升幅度达17.83%和25.00%。可视化分析表明,所提框架在保持实时响应能力的同时,成功捕获了数据流中的潜在概念漂移模式。可见,所提框架突破了传统方法在计算效率与模型性能间的权衡困境,为开放环境下的在线连续学习系统建立了新范式。

关键词: 连续学习, 稀疏化, 在线学习, 参数掩码, 计算效率

Abstract:

Task-free online continual learning is a task-agnostic and autonomous machine learning approach, where model dynamic update is achieved through continuous adaptation to new data and forgetting suppression. In the existing online continual learning methods, model accuracy is prioritized typically at the expense of computational efficiency, making it difficult for the model to respond promptly to changes in the data stream due to the lag in training speed in high-speed data stream scenarios. To address the above challenge, an efficiency-performance co-optimized online sparse continual learning framework was proposed to overcome the limitations of conventional approaches through constructing a bidirectional sparse adaptive regulation mechanism. Firstly, a dynamic sparse topology optimization framework for parameter importance measurement was designed, so that unstructured parameter pruning was achieved by incorporating parameter sensitivity analysis. Secondly, a memory-efficiency dual-objective optimization model was established, in which computational budget allocation was adjusted dynamically based on online class distribution estimation, so as to realize optimal computational resource configuration. Finally, a gradient decoupling optimization strategy was developed to employ gradient masking to enable bidirectional optimization of both old and new knowledge, thereby accelerating model updates and preserving the integrity of the knowledge topology at the same time. The benchmark tests results show that the proposed framework has significant advantages. Compared to the baseline ER (Experience Replay), with a memory buffer of 100, on CIFAR-10 dataset the framework achieves average improvements of 4.86% in Average Online Accuracy (AOA) and 6.25% in Test Accuracy (TA) ; on CIFAR-100 dataset, the framework obtains enhancements of 13.77% in AOA and 3.08% in TA; on Mini-ImageNet dataset, it shows performance gains of 17.83% and 25.00% in AOA and TA respectively. Visualization analysis shows that the proposed framework captures underlying concept drift patterns in data streams successfully while maintaining real-time response ability. It can be seen that the proposed framework breaks through the traditional methods’ dilemma of trade-off between computational efficiency and model performance, and establishes a new paradigm for online continual learning systems in open environments.

Key words: Continual Learning (CL), sparsity, online learning, parameter masking, computational efficiency

中图分类号: