Diabetes prediction model based on improved TabNet

doi:10.11772/j.issn.1001-9081.2023121732

Abstract

Abstract:

Addressing challenges posed by feature redundancy and unbalanced number of categories of diabetes data， an improved TabNet based diabetes prediction model — dTabNet （dual TabNet） was proposed. Firstly， the quantified global contribution indicators for features were obtained by dTabNet. At the same time， a triple-layer fully connected layer was constructed to enhance the model's representation capacity by replacing the single-layer fully connected layers in both feature-transformer and attention-transformer. Then， a feature selection module was designed to eliminate redundant and irrelevant features from the diabetes dataset， thereby enhancing learning efficiency. Finally， diabetes prediction was executed based on the feature subsets output by the feature selection module. Besides， the Focal loss function was introduced to optimize the loss function， addressing the issue of unbalanced number of categories of diabetes data. After that， the Bayesian optimization algorithm was applied to perform hyperparameter optimization for dTabNet. The proposed model was evaluated on a preprocessed real-world diabetes detection dataset. Experimental results demonstrate that dTabNet achieves a diabetes prediction accuracy of 90.9% and an Area Under the Curve （AUC） value of 94.7%. which are improved by 4.0 and 2.2 percentage points， respectively， compared to those of TabNet. On diabetes prediction data characterized by redundancy and unbalanced number of categories， the effectiveness of the proposed model is indicated.

Key words: diabetes prediction, TabNet, feature selection, unbalanced number of categories, tabular data, Bayesian optimization

摘要：

针对糖尿病数据特征冗余、类别数不平衡导致的预测困难问题，提出一种基于改进TabNet的糖尿病预测模型dTabNet（dual TabNet）。首先，通过dTabNet得到量化的特征全局贡献指标，同时构造3层全连接层替代特征转换器和注意力转换器中的单层全连接层，以增强对数据的表示能力；其次，设计特征选择模块，以去除糖尿病数据集中的冗余和无关特征，从而提高学习的效率；最后，根据特征选择模块输出的特征子集预测糖尿病。此外，引入Focal loss函数优化损失函数，以解决糖尿病数据类别数不平衡的问题；之后，利用贝叶斯优化算法对dTabNet进行超参数优化。在预处理后的真实糖尿病检测数据集上评估所提模型。实验结果表明，dTabNet针对糖尿病预测的准确率达到了90.9%，曲线下面积（AUC）值达到了94.7%，相较于TabNet的准确率和AUC值分别提高了4.0和2.2个百分点。这表明所提模型在数据冗余、类别数量不平衡的糖尿病数据上能有效地完成糖尿病预测。

关键词: 糖尿病预测, TabNet, 特征选择, 类别数量不平衡, 表格数据, 贝叶斯优化

CLC Number:

TP391.1

Shaoqin XU, Bo PENG, Wudan LONG, Danni DING. Diabetes prediction model based on improved TabNet[J]. Journal of Computer Applications, 0, (): 364-369.

徐绍钦, 彭博, 龙伍丹, 丁丹妮. 基于改进TabNet的糖尿病预测模型[J]. 《计算机应用》唯一官方网站, 0, (): 364-369.

Figures/Tables 11

符号	描述
$M ∈ R B × D$	掩码集
$P ∈ R B × D$	先验尺度矩阵，用于计量特征使用情况
$a ∈ R B × N a$	特征子集，用于下一步掩码计算
$d ∈ R B × N d$	局部预测值
$d o u t$	全局预测值
$ω b$	特征局部贡献度
$M a g g ‑ b$	特征全局贡献度
$M a g g$	特征全局贡献集
$S$	候选特征集
$D ·$	最大贡献度函数
$R ·$	最小冗余函数
$I ·$	互信息函数
$L y$	预测损失函数
$L s p a r s e$	正则化损失函数
$L$	总体损失函数

符号	描述
$M ∈ R B × D$	掩码集
$P ∈ R B × D$	先验尺度矩阵，用于计量特征使用情况
$a ∈ R B × N a$	特征子集，用于下一步掩码计算
$d ∈ R B × N d$	局部预测值
$d o u t$	全局预测值
$ω b$	特征局部贡献度
$M a g g ‑ b$	特征全局贡献度
$M a g g$	特征全局贡献集
$S$	候选特征集
$D ·$	最大贡献度函数
$R ·$	最小冗余函数
$I ·$	互信息函数
$L y$	预测损失函数
$L s p a r s e$	正则化损失函数
$L$	总体损失函数

模型	Accuracy	AUC
TabNet	86.9	92.5
dTabNet	90.9	94.7

模型	Accuracy	AUC
TabNet	86.9	92.5
+Null Deviance	88.2	93.0
+MDRMR	88.9	94.1

模型	Accuracy	AUC
TabNet	86.9	92.5
+X1	88.9	94.1
+X2	87.6	93.1
+X3	87.5	92.5
dTabNet	90.9	94.7

References 27

1	FOWLER M J. Microvascular and macrovascular complications of diabetes ［J］. Clinical Diabetes， 2008， 26（2）： 77-82.
2	RAHMAN M， ISLAM D， MUKTI R J， et al. A deep learning approach based on convolutional LSTM for detecting diabetes［J］. Computational Biology and Chemistry， 2020， 88： No.107329.
3	SUN H， SAEEDI P， KARURANGA S， et al. IDF Diabetes Atlas： global， regional and country-level diabetes prevalence estimates for 2021 and projections for 2045［J］. Diabetes Research and Clinical Practice， 2022， 183： No.109119.
4	刘月姣.《中国居民营养与慢性病状况报告（2020年）》发布［J］. 农产品市场， 2021，（2）：58-59.
5	HASAN M K， ALEEF T A， ROY S. Automatic mass classification in breast using transfer learning of deep convolutional neural network and support vector machine［C］// Proceedings of the 2020 IEEE Region 10 Symposium. Piscataway： IEEE， 2020： 110-113.
6	BIAU G， SCORNET E. A random forest guided tour［J］. TEST， 2016， 25（2）： 197-227.
7	SONG Y Y， LU Y. Decision tree methods： applications for classification and prediction［J］. Shanghai Archives of Psychiatry， 2015， 27（2）： 130-135.
8	ISLAM M M F， FERDOUSI R， RAHMAN S， et al. Likelihood prediction of diabetes at early stage using data mining techniques［C］// Proceedings of the 2019 International Symposium on Computer Vision and Machine Intelligence in Medical Image Analysis， AISC 992. Singapore： Springer， 2020：113-125.
9	HASAN M K， ALAM M A， DAS D， et al. Diabetes prediction using ensembling of different machine learning classifiers［J］. IEEE Access， 2020， 8： 76516-76531.
10	RALLAPALL S， SURYAKANTHI T. Predicting the risk of diabetes in big data electronic health records by using scalable random forest classification algorithm［C］// Proceedings of the 2016 International Conference on Advances in Computing and Communication Engineering. Piscataway： IEEE， 2016：281-284.
11	PHAM T， TRAN T， PHUNG D， et al. Predicting healthcare trajectories from medical records： a deep learning approach［J］. Journal of Biomedical Informatics， 2017， 69： 218-229.
12	ALEX S A， NAYAHI J J V， SHINE H， et al. Deep convolutional neural network for diabetes mellitus prediction［J］. Neural Computing and Applications， 2022， 34（2）： 1319-1327.
13	ARIK S O， PFISTER T. TabNet： attentive interpretable tabular learning ［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021：6679-6687.
14	ZOU Q， QU K， LUO Y， et al. Predicting diabetes mellitus with machine learning techniques［J］. Frontiers in Genetics， 2018， 9： No.515.
15	YASAR A. Data classification of early-stage diabetes risk prediction datasets and analysis of algorithm performance using feature extraction methods and machine learning techniques ［J］. International Journal of Intelligent Systems and Applications in Engineering， 2021， 9（4）： 273-281.
16	HOFFER E， HUBARA I， SOUDRY D. Train longer， generalize better： closing the generalization gap in large batch training of neural networks［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017：1729-1739.
17	MARTINS A F T， ASTUDILLO R F. From softmax to sparsemax： a sparse model of attention and multi-label classification［C］// Proceedings of the 33rd International Conference on Machine Learning. New York： JMLR.org， 2016： 1614-1623.
18	DAUPHIN Y N， FAN A， AULI M， et al. Language modeling with gated convolutional network ［C］// Proceedings of the 34th International Conference on Machine Learning. New York： JMLR.org， 2017：933-941.
19	GEHRING J， AULI M， GRANGIER D， et al. Convolutional sequence to sequence learning［C］// Proceedings of the 34th International Conference on Machine Learning. New York： JMLR.org， 2017：1243-1252.
20	BROWN G， POCOCK A， ZHAO M J， et al. Conditional likelihood maximisation： a unifying framework for information theoretic feature selection ［J］. Journal of Machine Learning Research， 2012， 13：27-66.
21	LIN T Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017：2999-3007.
22	GRANDVALET Y， BEBGIO Y. Semi-supervised learning by entropy minimization ［C］// Proceedings of the 17th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2004：529-536.
23	SAMEEN M I， PRADHAN B， LEE S. Application of convolutional neural networks featuring Bayesian optimization for landslide susceptibility assessment ［J］. CATENA， 2020， 186： No.104249.
24	ABBASIMEHR H， PAKI R. Prediction of COVID-19 confirmed cases combining deep learning methods and Bayesian optimization［J］. Chaos， Solitons and Fractals， 2021， 142： No.110511.
25	高媛媛，余振华，杜方，等. 基于贝叶斯优化的无标签网络剪枝算法［J］. 计算机应用， 2023， 43（1）： 30-36.
26	KHAN S， ALOTAIBI R M. A novel thresholding for prediction analytics with machine learning techniques ［J］. International Journal of Computer Science and Network Security， 2023， 23（1）：33-40.
27	张汇洋，刘瑞银. 交叉验证法在模型比较中的应用［J］. 应用数学进展， 2023， 12（4）： 1866-1873.

[1]	Dixin WANG, Jiahao WANG, Min LI, Hao CHEN, Guangyao HU, Yu GONG. Abnormal attack detection for underwater acoustic communication network [J]. Journal of Computer Applications, 2025, 45(2): 526-533.
[2]	Songlin WU, Guangchao ZHANG, Yuan YAO, Bo PENG. Multi-view difficult airway recognition based on discriminant region guidance [J]. Journal of Computer Applications, 2025, 45(10): 3399-3406.
[3]	Lixia XIE, Jiamin WANG, Hongyu YANG, Ze HU, Xiang CHENG. Low-latency DDoS attack detection based on hybrid feature selection [J]. Journal of Computer Applications, 2025, 45(10): 3231-3240.
[4]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[5]	Wudan LONG, Bo PENG, Jie HU, Ying SHEN, Danni DING. Road damage detection algorithm based on enhanced feature extraction [J]. Journal of Computer Applications, 2024, 44(7): 2264-2270.
[6]	Lin GAO, Yu ZHOU, Tak Wu KWONG. Evolutionary bi-level adaptive local feature selection [J]. Journal of Computer Applications, 2024, 44(5): 1408-1414.
[7]	Mingzhu LEI, Hao WANG, Rong JIA, Lin BAI, Xiaoying PAN. Oversampling algorithm based on synthesizing minority class samples using relationship between features [J]. Journal of Computer Applications, 2024, 44(5): 1428-1436.
[8]	Dapeng XU, Xinmin HOU. Feature selection method for graph neural network based on network architecture design [J]. Journal of Computer Applications, 2024, 44(3): 663-670.
[9]	Shengjie MENG, Wanjun YU, Ying CHEN. Feature selection algorithm for high-dimensional data with maximum correlation and maximum difference [J]. Journal of Computer Applications, 2024, 44(3): 767-771.
[10]	Lin SUN, Menghan LIU. K-means clustering based on adaptive cuckoo optimization feature selection [J]. Journal of Computer Applications, 2024, 44(3): 831-841.
[11]	Jingxin LIU, Wenjing HUANG, Liangsheng XU, Chong HUANG, Jiansheng WU. Unsupervised feature selection model with dictionary learning and sample correlation preservation [J]. Journal of Computer Applications, 2024, 44(12): 3766-3775.
[12]	Tian HE, Zongxin SHEN, Qianqian HUANG, Yanyong HUANG. Adaptive learning-based multi-view unsupervised feature selection method [J]. Journal of Computer Applications, 2023, 43(9): 2657-2664.
[13]	Lin SUN, Jinxu HUANG, Jiucheng XU. Feature selection for imbalanced data based on neighborhood tolerance mutual information and whale optimization algorithm [J]. Journal of Computer Applications, 2023, 43(6): 1842-1854.
[14]	Zhenhua YU, Zhengqi LIU, Ying LIU, Cheng GUO. Feature selection method based on self-adaptive hybrid particle swarm optimization for software defect prediction [J]. Journal of Computer Applications, 2023, 43(4): 1206-1213.
[15]	Lin SUN, Tianjiao MA, Zhan’ao XUE. Multilabel feature selection algorithm based on Fisher score and fuzzy neighborhood entropy [J]. Journal of Computer Applications, 2023, 43(12): 3779-3789.