Semi-supervised adaptive multi-view embedding method for feature dimension reduction

doi:10.11772/j.issn.1001-9081.2018051050

Abstract

Abstract: Most of the semi-supervised multi-view feature reduction methods do not take into account of the differences in feature projections among different views, and it is not able to avoid the effects of noise and other unrelated features because of the lack of sparse constraints on the low-dimensional matrix after dimension reduction. In order to solve the two problems, a new Semi-Supervised Adaptive Multi-View Embedding method for feature dimension reduction (SS-AMVE) was proposed. Firstly, the projection was extended from the same embedded matrix in a single view to different matrices in multi-view, and the global structure maintenance term was introduced. Then, the unlabeled data was embedded and projected by the unsupervised method, and the labeled data was linearly projected in combination with the classified discrimination information. Finally, the two types of multi-projection were mapped to a unified low-dimensional space, and the combined weight matrix was used to preserve the global structure, which largely eliminated the effects of noise and unrelated factors. The experimental results show that, the clustering accuracy of the proposed method is improved by about 9% on average. The proposed method can better preserve the correlation of features between multiple views, and capture more features with discriminative information.

Key words: multi-view feature dimension reduction, semi-supervised learning, adaptive embedding, combined weight matrix, regularized sparse constraint

摘要： 半监督模式下的多视角特征降维方法，大多并未考虑到不同视角间特征投影的差异，且由于缺乏对降维后的低维矩阵的稀疏约束，无法避免噪声和其他不相关特征的影响。针对这两个问题，提出自适应嵌入的半监督多视角特征降维方法。首先，将投影从单视角下相同的嵌入矩阵扩展到多视角间不同的矩阵，引入全局结构保持项；然后，将无标签的数据利用无监督方法进行嵌入投影，对于有标签的数据，结合分类的判别信息进行线性投影；最后，再将两类多投影映射到统一的低维空间，使用组合权重矩阵来保留全局结构，很大程度上消除了噪声及不相关因素的影响。实验结果表明，所提方法的聚类准确率平均提高了约9%。该方法较好地保留了多视角间特征的相关性，捕获了更多的具有判别信息的特征。

关键词: 多视角特征降维, 半监督学习, 自适应性嵌入, 组合权重矩阵, 正则化稀疏约束

CLC Number:

TP391.4

SUN Shengzi, WAN Yuan, ZENG Cheng. Semi-supervised adaptive multi-view embedding method for feature dimension reduction[J]. Journal of Computer Applications, 2018, 38(12): 3391-3398.

孙圣姿, 万源, 曾成. 自适应嵌入的半监督多视角特征降维方法[J]. 计算机应用, 2018, 38(12): 3391-3398.

References

[1] JAMIESON K, BALAKRISHNAN H, TAY Y C. Sift:a MAC protocol for event-driven wireless sensor networks[C]//Proceedings of the 2006 European Workshop on Wireless Sensor Networks, LNCS 3868. Berlin:Springer, 2006:260-275.
[2] WANG X Y, HAN X T, YAN S C. An HOG-LBP human detector with partial occlusion handling[C]//Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Piscataway, NJ:IEEE, 2009:32-39.
[3] LIU X W, WANG L, ZHANG J, et al. Global and local structure preservation for feature selection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(6):1083-1095.
[4] ZHAO Z, WANG L, LIU H, et al. On similarity preserving feature selection[J]. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(3):619-632.
[5] HE X F, JI M, ZHANG C Y, et al. A Variance minimization criterion to feature selection using Laplacian regularization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(10):2013-2025.
[6] HOU C P, NIE F P, LI X L,et al. Joint embedding learning and sparse regression:a framework for unsupervised feature selection[J]. IEEE Transactions on Cybernetics, 2014, 44(6):793-804.
[7] HE X F, D CAI, NIYOGI P. Laplacian score for feature selection[C]//Proceedings of the 2005 International Conference on Neural Information Processing Systems. Cambridge, MA:MIT Press, 2005:507-514.
[8] YANG Y, YANG Y, SHEN H T, et al. Discriminative nonnegative spectral clustering with out-of-sample extension[J]. IEEE Transactions on Knowledge & Data Engineering, 2013. 25(8):1760-1771.
[9] XIA T, TAO D H, MEI T, et al. Multiview spectral embedding[J]. IEEE Transactions on Systems, Man and Cybernetics, Part B, 2010, 40(6):1438-1446.
[10] BELKIN M, NIYOGI P. Laplacian eigenmaps and spectral techniques for embedding and clustering[C]//Proceedings of the 200114th International Conference on Neural Information Processing Systems:Natural and Synthetic. Cambridge, MA:MIT Press, 2001:585-591.
[11] CAI D, HE X F, HAN J W. Semi-supervised discriminant analysis[C]//Proceedings of the 2007 IEEE 11th International Conference on Computer Vision. Piscataway, NJ:IEEE, 2007:1-7.
[12] XU Z L, LING I, LYU M R-T,et al. Discriminative semi-supervised feature selection via manifold regularization[J]. IEEE Transactions on Neural Networks, 2010, 21(7):1033-1047.
[13] COELHO F, BRAGA A P, VERLEYSEN M. Multi-objective semi-supervised feature selection and model selection based on pearson's correlation coefficient[C]//Proceedings of the 2010 Iberoamerican Congress Conference on Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. Berlin:Springer, 2010:509-516.
[14] DU L, SHEN Y D. Unsupervised feature selection with adaptive structure learning[C]//Proceedings of the 201521th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM, 2015:209-218.
[15] CHEN X, LIU W, SU F L, et al. Semi-supervised multi-view feature selection with label learning for VHR remote sensing images[C]//Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium. Piscataway, NJ:IEEE, 2016:2372-2375.
[16] SUN S L, JIN F, TU W T. View construction for multi-view semi-supervised learning[C]//Proceedings of the 20118th International Conference on Advances in Neural Networks. Berlin:Springer, 2011:595-601.
[17] SHI C J, AN G Y, ZHAO R Z, et al. Multiview Hessian semisupervised sparse feature selection for multimedia analysis[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(9):1947-1961.
[18] ZHU S H, SUN X, JIN D L. Multi-view semi-supervised learning for image classification[J]. Neurocomputing, 2016, 208:136-142.
[19] FENG Y F, XIAO J, ZHUANG Y T, et al. Adaptive unsupervised multi-view feature selection for visual concept recognition[C]//Proceedings of the 201211th Asian Conference on Computer Vision. Berlin:Springer, 2012:343-357.
[20] WEI X K, CAO B K, YU P S. Multi-view unsupervised feature selection by cross-diffused matrix alignment[C]//Proceedings of the 2017 International Joint Conference on Neural Networks. Piscataway, NJ:IEEE, 2017:494-501.
[21] SONG Y Q, NIE F P, ZHANG C S,et al. A unified framework for semi-supervised dimensionality reduction[J]. Pattern Recognition, 2008, 41(9):2789-2799.
[22] OLSHAUSEN B A, FIELD D J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images[J]. Nature, 1996, 381:607-609.
[23] HOU C P, ZHANG C S, WU Y, et al. Multiple view semi-supervised dimensionality reduction[J]. Pattern Recognition, 2010, 43(3):720-730.
[24] 陶红.多视角数据分析算法研究[D].长沙:国防科学技术大学,2014:15-28.(TAO H. Study on algorithms for analyzing multi-view data[D]. Changsha:National University of Defense Technology, 2014:15-28.)
[25] 汪荆琪.基于多视角的半监督特征选择算法研究[D].合肥:中国科学技术大学,2014:5-8.(WANG J Q. Semi-supervised feature selection for multi-view data[D]. Hefei:University of Science and Technology of China, 2014:5-8.)
[26] 郝伟,刘忠宝.基于Fisher准则的半监督特征提取方法[J].计算机工程与设计,2017,38(1):238-241.(HAO W, LIU Z B. Semi-supervised feature extraction method based on Fisher criterion[J]. Computer Engineering and Design, 2017, 38(1):238-241.)