Journal of Computer Applications

    Next Articles

LoRA-KANConv SAM: efficient method for small object medical image segmentation

  

  • Received:2025-04-21 Revised:2025-06-05 Accepted:2025-06-05 Online:2025-06-05 Published:2025-06-05
  • Supported by:
    Research on Key Technologies of Vital Sign Detection Based on Millimeter-Wave Radar

LoRA-KANConv SAM: 高效小目标医学图像分割方法

曹国灿1,2,黄天宇1,2*,贾敬好3,张钊1,2,孙晓川1,2,李莹琦1,2   

  1. 1.华北理工大学 人工智能学院,河北 唐山 063210

    2.河北省工业智能感知重点实验室(华北理工大学),河北 唐山 063210 3.华北理工大学附属医院,河北 唐山 063210

  • 通讯作者: 黄天宇
  • 基金资助:
    基于毫米波雷达的生命体征检测关键技术研究

Abstract: Medical image segmentation provides a basis for disease treatment and promotes the development of modern medicine by accurately identifying the key areas in medical images. Most of the traditional medical image segmentation methods are one model for one task, with high data dependence, large computing power requirements, and unstable cross-domain segmentation performance. Segmentation Anything Model (SAM) shows strong cross-modal adaptability and segmentation performance on natural images. However, there are significant differences between natural images and medical images, especially small object medical images, which makes SAM face problems of low efficiency and insufficient accuracy in the segmentation of small object medical images. To solve the above problems, an efficient fine-tuning small-object medical image segmentation method (LoRA-KANConv SAM) based on SAM was proposed. In view of the performance differences of pre-trained SAM on natural images and medical images, it was proposed to inject Low-Rank Adaptation of Large Language Models(LoRA) into the shallow network of the ViT(Vision Transformer) encoder of the SAM model to guide the initial feature extraction of the input image, reducing the number of parameters of the model while preserving the basic representation ability of the pre-trained model. The KANConv (Convolutional Kolmogorov-Arnold Networks) layer was applied to the model to extract the local detail features of the image and improve the accuracy of image segmentation. The experimental results show that the training parameters of LoRA-KANConv SAM are only 0.67% of the original parameters, and the four evaluation indicators Dice(Dice Similarity Coefficient),IoU(Intersection over Union),HD95(95th Percentile Hausdorff Distance),ASSD (Average Symmetric Surface Distance) are greatly improved, the segmentation mask prediction speed reaches 0.33 s/f, and the switching time of the fine-tuning model is only 13.2% of the full fine-tuning model.

Key words: small object medical image segmentation, Low-Rank Adaptation of Large Language Models (LoRA), KANConv (Convolutional Kolmogorov-Arnold Networks), Segmentation Anything model (SAM), Parameter-Efficient Fine-Tuning

摘要: 医学图像分割通过精准识别医学影像中的关键区域,为疾病治疗提供依据,推动现代医学发展。传统的医学图像分割方法大多是一个模型针对一个任务,数据依赖度高,算力需求大,且跨域分割性能不稳定。分割一切模型(SAM)在自然图像上表现出强大的跨模态适应能力和分割性能,但自然图像与医学图像特别是小目标医学图像有明显差异,使得SAM在小目标医学图像分割上面临着效率低和精度不足的问题。为解决上述问题,基于SAM提出一种高效微调小目标医学图像分割方法(LoRA-KANConv SAM)。针对预训练SAM在自然图像与医学图像上存在性能差异,将低秩微调(LoRA)注入SAM模型ViT编码器的浅层网络中,引导输入图像的初步特征提取,在保全预训练模型基础表征能力的同时降低模型的参数量;将KANConv层应用到模型中,提取图像的局部细节特征,提高图像分割精度。实验结果证明,LoRA-KANConv SAM算法的可训练参数仅为原参数量的0.67%,且4个评估指标Dice(Dice Similarity Coefficient),IoU(Intersection over Union),HD95(95th Percentile Hausdorff Distance),ASSD(Average Symmetric Surface Distance)大幅提升,分割掩码预测速度达到0.33 s/f,微调模型的切换时间仅为全量微调模型的13.2%。

关键词: 小目标医学图像分割;低秩微调;KANConv;分割一切模型;高效微调 

CLC Number: