Unsupervised feature selection model with dictionary learning and sample correlation preservation

doi:10.11772/j.issn.1001-9081.2023121783

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (12): 3766-3775.DOI: 10.11772/j.issn.1001-9081.2023121783

• Artificial intelligence • Previous Articles Next Articles

Unsupervised feature selection model with dictionary learning and sample correlation preservation

Jingxin LIU¹, Wenjing HUANG¹, Liangsheng XU², Chong HUANG³(), Jiansheng WU¹

^1.School of Mathematics and Computer Sciences，Nanchang University，Nanchang Jiangxi 330031，China
^2.Northern Lianchuang Communication Company Limited，Nanchang Jiangxi 330096，China
^3.Information Office，Nanchang University，Nanchang Jiangxi 330031，China

Received:2023-12-27 Revised:2024-02-06 Accepted:2024-02-23 Online:2024-03-11 Published:2024-12-10
Contact: Chong HUANG
About author:LIU Jingxin， born in 1999， M. S. candidate. Her research interests include machine learning， data mining.
HUANG Wenjing， born in 2003. Her research interests include machine learning， data mining.
XU Liangsheng， born in 1987， M. S.， engineer. His research interests include command and communication， machine learning， data analysis.
WU Jiansheng， born in 1986， Ph. D.， lecturer. His research interests include machine learning， data mining.
Supported by:
National Natural Science Foundation of China(62066027);Natural Science Foundation of Jiangxi Province(20212BAB212011);Postgraduate Innovation Foundation of Jiangxi Province(YC2022-s160)

字典学习与样本关联保持结合的无监督特征选择模型

刘晶鑫¹, 黄雯静¹, 徐亮胜², 黄冲³(), 吴建生¹

^1.南昌大学数学与计算机学院，南昌 330031
^2.北方联创通信有限公司，南昌 330096
^3.南昌大学信息化办公室，南昌 330031

通讯作者: 黄冲
作者简介:刘晶鑫（1999—），女，四川江油人，硕士研究生，主要研究方向：机器学习、数据挖掘
黄雯静（2003—），女，福建莆田人，主要研究方向：机器学习、数据挖掘
徐亮胜（1987—），男，江西上饶人，工程师，硕士，主要研究方向：指挥与通信、机器学习、数据分析
吴建生（1986—），男，江西上饶人，讲师，博士，主要研究方向：机器学习、数据挖掘。
基金资助:
国家自然科学基金资助项目(62066027);江西省自然科学基金资助项目(20212BAB212011);江西省研究生创新基金资助项目(YC2022?s160)

Abstract

Abstract:

Focusing on the issue that most unsupervised feature selection models based on dictionary learning cannot fully exploit the intrinsic correlations among data， which reduces the accuracy of feature importance judgment， an unsupervised feature selection model with Dictionary Learning and Sample Correlation Preservation （DLSCP） was proposed. Firstly， the original data were encoded by learning the dictionary atoms， and the latent representations to characterize data distribution were obtained in the dictionary space. Secondly， the intrinsic correlations among data were learned adaptively in the dictionary space to alleviate the influence of redundant and noisy features， thus obtaining accurate local structure among data. Finally， the intrinsic correlations among data were used to measure the relevance and importance of data features. Experimental results on TOX dataset show that， when selecting 50 features， DLSCP improves the Normalized Mutual Information （NMI） and clustering Accuracy （Acc） by 13.33 and 7.95 percentage points respectively compared to non negative spectral analysis model NDFS（Nonnegative Discriminative Feature Selection） and by 15.74 and 7.31 percentage points respectively compared to unsupervised feature selection model with hidden space embedding LSEUFS （Latent Space Embedding for Unsupervised Feature Selection via joint dictionary learning）， which verifies the effectiveness of DLSCP.

Key words: unsupervised feature selection, dictionary learning, adaptive graph learning, sample correlation preservation, similarity matrix

摘要：

针对大多数基于字典学习的无监督特征选择模型没有充分挖掘数据间的本质关联，进而降低了特征重要性判断的准确性这一问题，提出一种字典学习与样本关联保持结合的无监督特征选择模型（DLSCP）。首先，从数据中学习字典基以完成对原始数据的编码，并在字典空间中获得能够反映数据分布的隐表示；其次，进一步在字典空间中自适应地学习数据间的本质关联，以消除冗余特征和噪声特征的影响，从而获得准确的数据间的局部几何结构；最后，利用数据间的本质关联评估数据特征的关联性和重要性。在TOX数据集上的实验结果表明，当选择50个特征时，DLSCP在归一化互信息（NMI）和聚类准确度（Acc）这2个评价指标上，相较于非负谱分析模型NDFS（Nonnegative Discriminative Feature Selection）分别提升了13.33和7.95个百分点，相较于隐空间嵌入无监督特征选择模型LSEUFS（Latent Space Embedding for Unsupervised Feature Selection via joint dictionary learning）分别提升了15.74和7.31个百分点，验证了DLSCP的有效性。

关键词: 无监督特征选择, 字典学习, 自适应图学习, 样本关联保持, 相似度矩阵

CLC Number:

TP181

Jingxin LIU, Wenjing HUANG, Liangsheng XU, Chong HUANG, Jiansheng WU. Unsupervised feature selection model with dictionary learning and sample correlation preservation[J]. Journal of Computer Applications, 2024, 44(12): 3766-3775.

刘晶鑫, 黄雯静, 徐亮胜, 黄冲, 吴建生. 字典学习与样本关联保持结合的无监督特征选择模型[J]. 《计算机应用》唯一官方网站, 2024, 44(12): 3766-3775.

Figures/Tables 11

References 30

1	CUI L， BAI L， WANG Y， et al. Fused Lasso for feature selection using structural information ［J］. Pattern Recognition， 2021， 119： No.108058.
2	CUI L， BAI L， WANG Y， et al. Internet financing credit risk evaluation using multiple structural interacting elastic net feature selection ［J］. Pattern Recognition， 2021， 114： No.107835.
3	ZHANG Z， BAI L， LIANG Y， et al. Joint hypergraph learning and sparse regression for feature selection［J］. Pattern Recognition， 2017， 63： 291-309.
4	谢惠华，黎明，王艳，等. 基于特征优选和字典优化的组稀疏表示表情识别［J］. 模式识别与人工智能， 2021， 34（5）： 446-454.
	XIE H H， LI M， WANG Y， et al. Group sparse representation based on feature selection and dictionary optimization for expression recognition［J］. Pattern Recognition and Artificial Intelligence， 2021， 34（5）： 446-454.
5	何添，沈宗鑫，黄倩倩，等. 基于自适应学习的多视图无监督特征选择方法［J］. 计算机应用， 2023， 43（9）： 2657-2664.
	HE T， SHEN Z X， HUANG Q Q， et al. Adaptive learning-based multi-view unsupervised feature selection method［J］. Journal of Computer Applications， 2023， 43（9）： 2657-2664.
6	ZHAO J， YAN S， FENG J. Towards age-invariant face recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2022， 44（1）： 474-487.
7	ZHANG Z， LIU L， SHEN F， et al. Binary multi-view clustering［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2019， 41（7）： 1774-1782.
8	MIAO J， YANG T， SUN L， et al. Graph regularized locally linear embedding for unsupervised feature selection［J］. Pattern Recognition， 2022， 122： No.108299.
9	HAN X， LIU P， WANG L， et al. Unsupervised feature selection via graph matrix learning and the low-dimensional space learning for classification［J］. Engineering Applications of Artificial Intelligence， 2020， 87： No.103283.
10	ZHU J， CHEN J， XU B， et al. Fast orthogonal locality-preserving projections for unsupervised feature selection［J］. Neurocomputing， 2023， 531： 100-113.
11	LI X， ZHANG H， ZHANG R， et al. Generalized uncorrelated regression with adaptive graph for unsupervised feature selection［J］. IEEE Transactions on Neural Networks and Learning Systems， 2019， 30（5）： 1587-1595.
12	LI Z， NIE F， WU D， et al. Unsupervised feature selection with weighted and projected adaptive neighbors ［J］. IEEE Transactions on Cybernetics， 2023， 53（2）： 1260-1271.
13	ZHU P， HU Q， ZHANG C， et al. Coupled dictionary learning for unsupervised feature selection ［C］// Proceedings of the 30th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2016： 2422-2428.
14	LU Z， CHU Q. Feature selection using class-level regularized self-representation ［J］. Applied Intelligence， 2023， 53（11）： 13130-13144.
15	MIAO J， YANG T， FAN C， et al. Self-paced non-convex regularized analysis-synthesis dictionary learning for unsupervised feature selection［J］. Knowledge-Based Systems， 2022， 241： No.108279.
16	FAN Y， DAI J， ZHANG Q L. Latent space embedding for unsupervised feature selection via joint dictionary learning ［C］// Proceedings of the 2019 International Joint Conference on Neural Networks. Piscataway： IEEE， 2019： 1-8.
17	ZHU X F， LI X L， ZHANG S C， et al. Robust joint graph sparse coding for unsupervised spectral feature selection［J］. IEEE Transactions on Neural Networks and Learning Systems， 2017， 28（6）： 1263-1275.
18	PARSA M G， ZARE H， GHATEE M. Low-rank dictionary learning for unsupervised feature selection［J］. Expert Systems with Applications， 2022， 202： No.117149.
19	DING D， XIA F， YANG X， et al. Joint dictionary and graph learning for unsupervised feature selection［J］. Applied Intelligence， 2020， 50（9）： 1379-1397.
20	DU L， SHEN Y D. Unsupervised feature selection with adaptive structure learning［C］// Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2015： 209-218.
21	BELKIN M， NIYOGI P. Laplacian eigenmaps for dimensionality reduction and data representation ［J］. Neural Computation， 2003， 15（6）： 1373-1396.
22	HE X， NIYOGI P. Locality preserving projections［C］// Proceedings of the 16th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2003： 153-160.
23	NISHIHARA R， LESSARD L， RECHT B， et al. A general analysis of the convergence of ADMM ［C］// Proceedings of the 32nd International Conference on Machine Learning. New York： JMLR.org， 2015： 343-352.
24	MAIRAL J， BACH F， PONCE J， et al. Online learning for matrix factorization and sparse coding ［J］. Journal of Machine Learning Research， 2010， 11： 19-60.
25	HE X， CAI D， NIYOGI P. Laplacian score for feature selection［C］// Proceedings of the 18th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2005： 507-514.
26	LI Z， YANG Y， LIU J， et al. Unsupervised feature selection using nonnegative spectral analysis ［C］// Proceedings of the 26th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2012： 1026-1032.
27	SHANG R， ZHANG Z， JIAO L， et al. Self-representation based dual-graph regularized feature selection clustering［J］. Neurocomputing， 2016， 171： 1242-1253.
28	刘艳芳，叶东毅. 基于邻域保持学习的无监督特征选择算法［J］. 模式识别与人工智能， 2018， 31（12）： 1096-1102.
	LIU Y F， YE D Y. Unsupervised feature selection algorithm based on neighborhood preserving learning［J］. Pattern Recognition and Artificial Intelligence， 2018， 31（12）： 1096-1102.
29	孟田田，周水生，田昕润. 基于l_2，0范数稀疏性和模糊相似性的图优化无监督组特征选择方法［J］. 模式识别与人工智能， 2023， 36（1）： 34-48.
	MENG T T， ZHOU S S， TIAN X R. Unsupervised group feature selection method for graph optimization based on l_2，0-norm sparsity and fuzzy similarity［J］. Pattern Recognition and Artificial Intelligence， 2023， 36（1）： 34-48.
30	ZHANG Y， LU Z， WANG S. Unsupervised feature selection via transformed auto-encoder ［J］. Knowledge-Based Systems， 2021， 215： No.106748.

数据集	特征维度	样本数	类别数	数据来源
BinaryAlphadigits	320	1 404	36	https：//cs.nyu.edu/~roweis/data/
Isolet	617	1 559	26	http：//www.cad.zju.edu.cn/home/dengcai/Data/MLData.html
Event	1 000	1 579	8	http：//vision.stanford.edu/resources_links.html#datasets
ORL	1 024	400	40	http：//www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html.
TOX	5 748	171	4	http：//featureselection.asu.edu/datasets.php
CLL-SUB	11 340	111	3	http：//featureselection.asu.edu/datasets.php

数据集	特征维度	样本数	类别数	数据来源
BinaryAlphadigits	320	1 404	36	https：//cs.nyu.edu/~roweis/data/
Isolet	617	1 559	26	http：//www.cad.zju.edu.cn/home/dengcai/Data/MLData.html
Event	1 000	1 579	8	http：//vision.stanford.edu/resources_links.html#datasets
ORL	1 024	400	40	http：//www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html.
TOX	5 748	171	4	http：//featureselection.asu.edu/datasets.php
CLL-SUB	11 340	111	3	http：//featureselection.asu.edu/datasets.php

模型	不同数据集上NMI的均值和标准差
模型	BinaryAlphadigits	Isolet	Event	ORL	TOX	CLL-SUB
LapScore	56.59±3.27	67.64±1.56	9.58±0.44	68.04±1.50	10.70±0.64	18.73±0.03
NDFS	56.28±2.77	68.59±4.57	18.77±1.49	71.53±1.08	16.43±0.82	18.78±0.09
DFSC	52.56±4.90	62.05±8.93	9.59±0.24	67.76±2.37	10.95±0.37	18.73±0.03
NPLFS	46.35±8.80	48.80±7.18	12.78±2.47	60.82±4.89	8.02±2.22	12.72±2.87
F-SUGFS	49.61±7.80	62.53±6.44	9.30±0.20	71.18±1.66	20.76±4.84	19.06±3.39
LSEUFS	56.20±2.56	70.95±4.05	24.37±0.98	73.17±0.49	18.81±3.32
RJGSC	55.33±2.87	70.07±1.58	20.64±1.93	73.37±0.78
DLUFS	56.04±2.78	72.32±1.31	19.31±1.63	73.77±0.63
DGL-UFS	54.95±3.48	67.91±3.60	15.65±0.67	71.89±0.74
DLSCP	57.74±1.76	72.75±2.44	25.96±0.83	74.52±0.62	27.68±0.95	20.84±3.96

模型	不同数据集上NMI的均值和标准差
模型	BinaryAlphadigits	Isolet	Event	ORL	TOX	CLL-SUB
LapScore	56.59±3.27	67.64±1.56	9.58±0.44	68.04±1.50	10.70±0.64	18.73±0.03
NDFS	56.28±2.77	68.59±4.57	18.77±1.49	71.53±1.08	16.43±0.82	18.78±0.09
DFSC	52.56±4.90	62.05±8.93	9.59±0.24	67.76±2.37	10.95±0.37	18.73±0.03
NPLFS	46.35±8.80	48.80±7.18	12.78±2.47	60.82±4.89	8.02±2.22	12.72±2.87
F-SUGFS	49.61±7.80	62.53±6.44	9.30±0.20	71.18±1.66	20.76±4.84	19.06±3.39
LSEUFS	56.20±2.56	70.95±4.05	24.37±0.98	73.17±0.49	18.81±3.32
RJGSC	55.33±2.87	70.07±1.58	20.64±1.93	73.37±0.78
DLUFS	56.04±2.78	72.32±1.31	19.31±1.63	73.77±0.63
DGL-UFS	54.95±3.48	67.91±3.60	15.65±0.67	71.89±0.74
DLSCP	57.74±1.76	72.75±2.44	25.96±0.83	74.52±0.62	27.68±0.95	20.84±3.96

模型	不同数据集上Acc的均值和标准差
模型	BinaryAlphadigits	Isolet	Event	ORL	TOX	CLL-SUB
LapScore	40.16±3.98	44.14±1.66	20.87±0.47	43.44±1.77	39.75±0.83	53.03±0.17
NDFS	39.70±2.91	48.71±2.51	29.82±1.53	48.68±1.63	44.01±1.58	53.14±0.04
DFSC	36.42±5.16	43.53±8.52	20.77±0.35	42.65±3.23	40.00±0.71	52.99±0.36
NPLFS	30.15±7.73	31.20±5.39	24.22±2.03	36.98±5.33	38.18±1.82	50.95±1.79
F-SUGFS	33.42±7.57	43.85±4.71	20.60±0.22	47.01±2.48	44.42±6.97	53.29±2.13
LSEUFS	40.07±2.62	52.97±2.03	34.82±0.53	50.10±0.89	45.21±7.08
RJGSC	39.71±2.43	51.45±1.95	31.45±1.40	50.56±0.53
DLUFS	40.13±2.77	50.40±1.63	30.41±0.94	51.45±0.57
DGL-UFS	39.42±3.34	47.85±1.99	27.61±0.70	48.85±0.71
DLSCP	41.52±1.98	53.71±2.23	34.56±0.92	52.42±0.76	50.29±2.30	53.69±0.95

Unsupervised feature selection model with dictionary learning and sample correlation preservation

字典学习与样本关联保持结合的无监督特征选择模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 30

Related Articles 15

Recommended Articles

Metrics

[1]	Tian HE, Zongxin SHEN, Qianqian HUANG, Yanyong HUANG. Adaptive learning-based multi-view unsupervised feature selection method [J]. Journal of Computer Applications, 2023, 43(9): 2657-2664.
[2]	Jianwen GAN, Yan CHEN, Peng ZHOU, Liang DU. Clustering ensemble algorithm with high-order consistency learning [J]. Journal of Computer Applications, 2023, 43(9): 2665-2672.
[3]	Zhifeng MA, Junyang YU, Longge WANG. Diversity represented deep subspace clustering algorithm [J]. Journal of Computer Applications, 2023, 43(2): 407-412.
[4]	Yu YANG, Weiwei DUAN. Spectral clustering based dynamic community discovery algorithm in social network [J]. Journal of Computer Applications, 2023, 43(10): 3129-3135.
[5]	Lei MA, Chuan LUO, Tianrui LI, Hongmei CHEN. Fuzzy-rough set based unsupervised dynamic feature selection algorithm [J]. Journal of Computer Applications, 2023, 43(10): 3121-3128.
[6]	ZHANG Cheng, WAN Yuan, QIANG Haopeng. Deep unsupervised discrete cross-modal hashing based on knowledge distillation [J]. Journal of Computer Applications, 2021, 41(9): 2523-2531.
[7]	LIN Junchao, WAN Yuan. Self-adaptive multi-measure unsupervised feature selection method with structured graph optimization [J]. Journal of Computer Applications, 2021, 41(5): 1282-1289.
[8]	LYU Yali, MIAO Junzhong, HU Weixin. Semi-supervised learning algorithm of graph based on label metric learning [J]. Journal of Computer Applications, 2020, 40(12): 3430-3436.
[9]	SONG Yan, YIN Jun. Multi-view spectral clustering algorithm based on shared nearest neighbor [J]. Journal of Computer Applications, 2020, 40(11): 3211-3216.
[10]	YANG Yanlin, YE Zhonglin, ZHAO Haixing, MENG Lei. Link prediction algorithm based on high-order proximity approximation [J]. Journal of Computer Applications, 2019, 39(8): 2366-2373.
[11]	XU Tao, WANG Xiaoming. Generalization error bound guided discriminative dictionary learning [J]. Journal of Computer Applications, 2019, 39(4): 940-948.
[12]	TAO Yongpeng, JING Yu, XU Cong. Image denoising algorithm based on grouped dictionaries and variational model [J]. Journal of Computer Applications, 2019, 39(2): 551-555.
[13]	XIANG Wen, ZHANG Ling, CHEN Yunhua, JI Qiumin. Single image super resolution algorithm based on structural self-similarity and deformation block feature [J]. Journal of Computer Applications, 2019, 39(1): 275-280.
[14]	YANG Xiaoling, LI Zhiqing, LIU Yutong. Automatic image annotation based on multi-label discriminative dictionary learning [J]. Journal of Computer Applications, 2018, 38(5): 1294-1298.
[15]	WANG Na, WANG Xiaofeng, GENG Guohua, SONG Qiannan. Semi-supervised classification algorithm based on C-means clustering and graph transduction [J]. Journal of Computer Applications, 2017, 37(9): 2595-2599.