Few-shot skin image classification model based on spatial transformer network and feature distribution calibration

doi:10.11772/j.issn.1001-9081.2024071039

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (8): 2720-2726.DOI: 10.11772/j.issn.1001-9081.2024071039

• Multimedia computing and computer simulation • Previous Articles

Few-shot skin image classification model based on spatial transformer network and feature distribution calibration

Jing WANG, Jiaxing LIU(), Wanying SONG, Jiaxing XUE, Wenxin DING

College of Communication and Information Engineering，Xi’an University of Science and Technology，Xi’an Shaanxi 710600，China

Received:2024-07-23 Revised:2024-10-12 Accepted:2024-10-12 Online:2024-11-19 Published:2025-08-10
Contact: Jiaxing LIU
About author:WANG Jing， born in 1986， Ph. D.， lecturer. Her research interests include computer vision， radar signal processing.
SONG Wanying， born in 1988， Ph. D.， associate professor. Her research interests include image processing， computer vision.
XUE Jiaxing， born in 2000， M. S. candidate. His research interests include image processing， computer vision.
DING Wenxin， born in 2002， M. S. candidate. Her research interests include computer vision.
Supported by:
National Natural Science Foundation of China(61901358)

基于空间变换网络和特征分布校准的小样本皮肤图像分类模型

王静, 刘嘉星(), 宋婉莹, 薛嘉兴, 丁温欣

西安科技大学通信与信息工程学院，西安 710600

通讯作者: 刘嘉星
作者简介:王静（1986—），女，河南安阳人，讲师，博士，CCF会员，主要研究方向：计算机视觉、雷达信号处理
宋婉莹（1988—），女，山东聊城人，副教授，博士，主要研究方向：图像处理、计算机视觉
薛嘉兴（2000—），男，陕西宝鸡人，硕士研究生，主要研究方向：图像处理、计算机视觉
丁温欣（2002—），女，陕西渭南人，硕士研究生，主要研究方向：计算机视觉。
基金资助:
国家自然科学基金资助项目(61901358)

Abstract

Abstract:

Deep learning-based image classification methods typically require a lot of labeled data. However， in classification task of skin lesions in the medical field， collecting a lot of image data faces numerous challenges. To classify few-shot skin diseases accurately， a few-shot classification model based on Spatial Transformer Network （STN） and feature distribution calibration was proposed. Firstly， transfer learning and meta-learning were integrated to address the overfitting issue in cross-domain few-shot transfer. Secondly， a rotation angle prediction task was inserted before the pre-training classification task to better adapt the model to the high complexity of medical image data. Thirdly， after downsampling the images， a STN was introduced to perform affine transformations on the input images explicitly， thereby enhancing feature extraction and recognition capabilities. Finally， feature distribution calibration was used to constrain new class features， and the nearest centroid algorithm was introduced for classification decisions， thereby reducing algorithm complexity while improving classification accuracy significantly. Experimental results on ISIC2018 skin lesion dataset show that compared to the current mainstream few-shot model Meta-Baseline， the proposed model has the accuracy improvements of 11.80 and 10.82 percentage points in 2-way and 3-way classification tasks， respectively； compared to the model MetaMed， the proposed model has the average accuracy improvements of 6.65 and 9.58 percentage points in 2-way 3-shot and 3-way 3-shot classification tasks， respectively. It can be seen that the proposed model improves the classification accuracy of few-shot skin diseases effectively， and can assist doctors better in enhancing clinical diagnosis accuracy.

Key words: few-shot learning, image classification, skin lesion, Spatial Transformer Network (STN), nearest centroid

摘要：

基于深度学习的图像分类模型通常需要大量标记数据，然而，在医学领域的皮肤病变分类任务中，收集大量图像数据面临着诸多挑战。为了能准确分类小样本皮肤疾病，提出一种基于空间变换网络（STN）和特征分布校准的小样本分类模型。首先，将迁移学习和元学习相结合，以解决跨域迁移小样本存在的过拟合问题；其次，在预训练分类任务前插入旋转角度预测任务，以便模型更好地适应医学图像数据的高复杂度；再次，在对图像下采样后引入STN，以通过显式地对输入图像进行仿射变换，增强特征的提取和识别能力；最后，通过特征分布校准对新类特征进行约束，并引入最邻近质心算法进行分类决策，在简化算法流程的同时显著提升分类精度。在ISIC2018皮肤病变数据集上的实验结果表明，与当前主流小样本模型Meta-Baseline相比，在2-way和3-way分类任务中，所提模型的平均精度分别提高了11.80和10.82个百分点；与模型MetaMed相比，在2-way 3-shot和3-way 3-shot分类任务中，所提模型的分类精度分别提升了6.65和9.58个百分点。可见，所提模型有效提高了小样本皮肤疾病的分类精度，能够更好地辅助医生提高临床诊断精确度。

关键词: 小样本学习, 图像分类, 皮肤病变, 空间变换网络, 最邻近质心

CLC Number:

TP391.41

Jing WANG, Jiaxing LIU, Wanying SONG, Jiaxing XUE, Wenxin DING. Few-shot skin image classification model based on spatial transformer network and feature distribution calibration[J]. Journal of Computer Applications, 2025, 45(8): 2720-2726.

王静, 刘嘉星, 宋婉莹, 薛嘉兴, 丁温欣. 基于空间变换网络和特征分布校准的小样本皮肤图像分类模型[J]. 《计算机应用》唯一官方网站, 2025, 45(8): 2720-2726.

Figures/Tables 9

References 30

[1]	HUANG S C， PAREEK A， JENSEN M， et al. Self-supervised learning for medical image classification： a systematic review and implementation guidelines［J］. npj Digital Medicine， 2023， 6： No.74.
[2]	KUMAR A， SINGH S K， SAXENA S， et al. Deep feature learning for histopathological image classification of canine mammary tumors and human breast cancer［J］. Information Sciences， 2020， 508： 405-421.
[3]	SPANHOL F A， OLIVEIRA L S， PETITJEAN C， et al. A dataset for breast cancer histopathological image classification［J］. IEEE Transactions on Biomedical Engineering， 2016， 63（7）： 1455-1462.
[4]	WANG H， WANG S， QIN Z， et al. Triple attention learning for classification of 14 thoracic diseases using chest radiography［J］. Medical Image Analysis， 2021， 67： No.101846.
[5]	BELLET A， HABRARD A， SEBBAN M. A survey on metric learning for feature vectors and structured data［R/OL］. ［2024-11-10］..
[6]	CHEN Y， LIU Z， XU H， et al. Meta-baseline： exploring simple meta-learning for few-shot learning［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2021： 9042-9051.
[7]	DAI Z， YI J， YAN L， et al. PFEMed： few-shot medical image classification using prior guided feature enhancement［J］. Pattern Recognition， 2023， 134： No.109108.
[8]	HU Y， GRIPON V， PATEUX S. Leveraging the feature distribution in transfer-based few-shot learning［C］// Proceedings of the 2021 International Conference on Artificial Neural Networks， LNCS 12892. Cham： Springer， 2021： 487-499.
[9]	谢莉，舒卫平，耿俊杰，等. 结合加权原型和自适应张量子空间的小样本宫颈细胞分类［J］. 计算机应用， 2024， 44（10）： 3200-3208.
	XIE L， SHU W P， GENG J J， et al. Few-shot cervical cell classification combining weighted prototype and adaptive tensor subspaces［J］. Journal of Computer Applications， 2024， 44（10）： 3200-3208.
[10]	LIU J， SONG L， QIN Y. Prototype rectification for few-shot learning［C］// Proceedings of the 2020 European Conference on Computer Vision， LNCS 12346. Cham： Springer， 2020： 741-756.
[11]	LEE J H， ZAHEER M Z， ASTRID M， et al. SmoothMix： a simple yet effective data augmentation to train robust classifiers［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2020： 3264-3274.
[12]	DeVRIES T， TAYLOR G W. Improved regularization of convolutional neural networks with cutout［EB/OL］. ［2024-11-10］..
[13]	ZHANG H， CISSE M， DAUPHIN Y N， et al. mixup： Beyond empirical risk minimization［EB/OL］. ［2024-11-10］..
[14]	YUN S， HAN D， CHUN S， et al. CutMix： regularization strategy to train strong classifiers with localizable features［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 6022-6031.
[15]	VERMA V， LAMB A， BECKHAM C， et al. Manifold Mixup： better representations by interpolating hidden states［C］// Proceedings of the 36th International Conference on Machine Learning. New York： JMLR.org， 2019： 6438-6447.
[16]	严一钦，罗川，李天瑞，等. 基于关系网络和vision Transformer的跨域小样本分类模型［J/OL］. 计算机应用［2024-06-21］..
	YAN Y Q， LUO C， LI T R， et al. Cross-domain few-shot classification model based on relational networks and vision Transformer［J/OL］. Journal of Computer Applications ［2024-06-21］..
[17]	GIDARIS S， SINGH P， KOMODAKIS N. Unsupervised representation learning by predicting image rotations［EB/OL］. ［2024-11-10］..
[18]	蔡安平. 基于属性的小样本分类算法及在医学图像上的应用［D］. 成都：电子科技大学， 2023.
	CAI A P. Attribute-based classification algorithm for few-shot and application to medical images［D］. Chengdu： University of Electronic Science and Technology of China， 2023.
[19]	FU W， CHEN J， ZHOU L. Boosting few-shot rare skin disease classification via self-supervision and distribution calibration［J］. Biomedical Engineering Letters， 2024， 14（4）： 877-889.
[20]	VINYALS O， BLUNDELL C， LILLICRAP T， et al. Matching networks for one shot learning［C］// Proceedings of the 30th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2016： 3637-3645.
[21]	ZAGORUYKO S， KOMODAKIS N. Wide residual networks［C］// Proceedings of the 2016 British Machine Vision Conference. Durham： BMVA Press， 2016： No.87.
[22]	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778.
[23]	ZHENG X， WANG Y， LIU Y， et al. Graph neural networks for graphs with heterophily： a survey［EB/OL］. ［2024-11-10］..
[24]	BAI J， HUANG S， XIAO Z， et al. Few-shot hyperspectral image classification based on adaptive subspaces and feature transformation［J］. IEEE Transactions on Geoscience and Remote Sensing， 2022， 60： No.5523917.
[25]	MANGLA P， SINGH M， SINHA A， et al. Charting the right manifold： Manifold Mixup for few-shot learning［C］// Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2020： 2207-2216.
[26]	张涛，王波，赵宇，等. 基于特征分布校准的小样本分类改进算法［J］.扬州大学学报（自然科学版）， 2024， 27（1）：56-61.
	ZHANG T， WANG B， ZHAO Y， et al. Improved few-shot classification algorithm based on feature distribution calibration［J］. Journal of Yangzhou University （Natural Science Edition）， 2024， 27（1）：56-61.
[27]	ZOU J， MA X， ZHONG C， et al. Dermoscopic image analysis for ISIC challenge 2018［EB/OL］. ［2024-11-10］..
[28]	SINGH R， BHARTI V， PUROHIT V， et al. MetaMed： few-shot medical image classification using gradient-based meta-learning［J］. Pattern Recognition， 2021， 120： No.108111.
[29]	CHEN W Y， LIU Y C， KIRA Z， et al. A closer look at few-shot classification［EB/OL］. ［2024-11-10］..
[30]	LIU B， CAO Y， LIN Y， et al. Negative margin matters： understanding margin in few-shot classification［C］// Proceedings of the 2020 European Conference on Computer Vision， LNCS 12349. Cham： Springer， 2020： 438-455.

类别名称	图像数	图像尺寸	数据属性
NV	6 075	600×450	元训练集
MEL	1 113	600×450
BKL	1 099	600×450
BCC	514	600×450
AKIEC	327	600×450	元测试集
VASC	142	600×450
DF	115	600×450

类别名称	图像数	图像尺寸	数据属性
NV	6 075	600×450	元训练集
MEL	1 113	600×450
BKL	1 099	600×450
BCC	514	600×450
AKIEC	327	600×450	元测试集
VASC	142	600×450
DF	115	600×450

N-way	模型	3-shot	5-shot	10-shot
2-way	Transfer^［28］	66.88	73.88	80.38
	Meta-Baseline^［29］	68.77	71.03	76.97
	Baseline+^［29］	64.77	70.27	74.67
	PT-MAP^［8］	80.63	82.96	84.53
	NegMargin^［30］	71.33	72.67	75.17
	本文模型	82.02±0.35	84.10±0.42	86.05±0.44
3-way	Transfer^［28］	55.67	59.67	65.92
	Meta-Baseline^［29］	56.80	59.20	65.22
	Baseline+^［29］	53.20	54.16	57.87
	PT-MAP^［8］	65.45	67.92	72.64
	NegMargin^［30］	60.69	57.58	63.04
	本文模型	68.08±0.65	71.34±0.35	74.25±0.75

N-way	模型	3-shot	5-shot	10-shot
2-way	Transfer^［28］	66.88	73.88	80.38
	Meta-Baseline^［29］	68.77	71.03	76.97
	Baseline+^［29］	64.77	70.27	74.67
	PT-MAP^［8］	80.63	82.96	84.53
	NegMargin^［30］	71.33	72.67	75.17
	本文模型	82.02±0.35	84.10±0.42	86.05±0.44
3-way	Transfer^［28］	55.67	59.67	65.92
	Meta-Baseline^［29］	56.80	59.20	65.22
	Baseline+^［29］	53.20	54.16	57.87
	PT-MAP^［8］	65.45	67.92	72.64
	NegMargin^［30］	60.69	57.58	63.04
	本文模型	68.08±0.65	71.34±0.35	74.25±0.75

N-way	模型	3-shot	5-shot	10-shot
2-way	MetaMed^［28］	75.37	78.25	84.25
	PFEMed^［7］	81.69	83.87	85.14
	SS-DCN^［19］	79.22	82.63	—
	本文模型	82.02±0.35	84.10±0.42	86.05±0.44
3-way	MetaMed^［28］	58.50	61.25	71.00
	PFEMed^［7］	66.94	69.78	73.81
	SS-DCN^［19］	66.34	70.69	74.79
	本文模型	68.08±0.65	71.34±0.35	74.25±0.75

Few-shot skin image classification model based on spatial transformer network and feature distribution calibration

基于空间变换网络和特征分布校准的小样本皮肤图像分类模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 30

Related Articles 15

Recommended Articles

Metrics

N-way	模型	预训练	空间转换网络	特征变换	最邻近质心	分类精度
N-way	模型	预训练	空间转换网络	特征变换	最邻近质心	3-shot	5-shot	10-shot
2-way	模型Ⅰ					63.01±0.86	65.24±0.71	68.26±0.40
	模型Ⅱ	√				75.94±0.49	76.52±0.67	78.02±0.38
	模型Ⅲ	√	√			76.59±0.80	77.97±0.37	78.73±0.47
	模型Ⅳ	√	√	√		80.67±0.30	81.80±0.65	83.84±0.34
	模型Ⅴ	√	√	√	√	82.02±0.35	84.10±0.42	86.05±0.44
3-way	模型Ⅰ					47.53±1.21	48.72±0.89	50.66±0.46
	模型Ⅱ	√				57.69±0.39	59.84±0.53	62.43±0.36
	模型Ⅲ	√	√			57.86±0.30	60.12±0.38	63.21±0.56
	模型Ⅳ	√	√	√		67.46±0.39	69.29±0.72	73.08±0.66
	模型Ⅴ	√	√	√	√	68.08±0.65	71.34±0.35	74.25±0.75

[1]	Qiaoling QI, Xiaoxiao WANG, Qianqian ZHANG, Peng WANG, Yongfeng DONG. Label noise adaptive learning algorithm based on meta-learning [J]. Journal of Computer Applications, 2025, 45(7): 2113-2122.
[2]	Zimo ZHANG, Xuezhuan ZHAO. Multi-scale sparse graph guided vision graph neural networks [J]. Journal of Computer Applications, 2025, 45(7): 2188-2194.
[3]	Ruifeng BAI, Guanglei GOU, Lang WEN, Wanyu MIAO. Granular-ball prototypical network for few-shot image classification [J]. Journal of Computer Applications, 2025, 45(7): 2269-2277.
[4]	Xiang WANG, Qianqian CUI, Xiaoming ZHANG, Jianchao WANG, Zhenzhou WANG, Jialin SONG. Wireless capsule endoscopy image classification model based on improved ConvNeXt [J]. Journal of Computer Applications, 2025, 45(6): 2016-2024.
[5]	Shuangshuang CUI, Hongzhi WANG, Jiahao ZHU, Hao WU. Two-stage data selection method for classifier with low energy consumption and high performance [J]. Journal of Computer Applications, 2025, 45(6): 1703-1711.
[6]	Sijie NIU, Yuliang LIU. Auxiliary diagnostic method for retinopathy based on dual-branch structure with knowledge distillation [J]. Journal of Computer Applications, 2025, 45(5): 1410-1414.
[7]	Biqing ZENG, Guangbin ZHONG, James Zhiqing WEN. Few-shot named entity recognition based on decomposed fuzzy span [J]. Journal of Computer Applications, 2025, 45(5): 1504-1510.
[8]	Liwei ZHANG, Quan LIANG, Yutao HU, Qiaole ZHU. Channel shuffle attention mechanism based on group convolution [J]. Journal of Computer Applications, 2025, 45(4): 1069-1076.
[9]	Meirong DING, Jinxin ZHUO, Yuwu LU, Qinglong LIU, Jicong LANG. Domain adaptation integrating environment label smoothing and nuclear norm discrepancy [J]. Journal of Computer Applications, 2025, 45(4): 1130-1138.
[10]	Yiqin YAN, Chuan LUO, Tianrui LI, Hongmei CHEN. Cross-domain few-shot classification model based on relation network and Vision Transformer [J]. Journal of Computer Applications, 2025, 45(4): 1095-1103.
[11]	Xuewen YAN, Zhangjin HUANG. Few-shot image classification method based on contrast learning [J]. Journal of Computer Applications, 2025, 45(2): 383-391.
[12]	Kun FU, Shicong YING, Tingting ZHENG, Jiajie QU, Jingyuan CUI, Jianwei LI. Graph data augmentation method for few-shot node classification [J]. Journal of Computer Applications, 2025, 45(2): 392-402.
[13]	Binhong XIE, Wanyin GAO, Wangdong LU, Yingjun ZHANG, Rui ZHANG. Dense object counting network with few-shot similarity matching feature enhancement [J]. Journal of Computer Applications, 2025, 45(2): 403-410.
[14]	Danni DING, Bo PENG, Xi WU. VPNet： fatty liver ultrasound image classification method inspired by ventral pathway [J]. Journal of Computer Applications, 2025, 45(2): 662-669.
[15]	Zongsheng ZHENG, Jia DU, Yuhe CHENG, Zecheng ZHAO, Yuewei ZHANG, Xulong WANG. Cross-modal dual-stream alternating interactive network for infrared-visible image classification [J]. Journal of Computer Applications, 2025, 45(1): 275-283.