关键词: 分类器, 层次聚类, 自适应采样, 数据选择, 小样本学习

Abstract:

Aiming at the problems of large training data size， long training time and high carbon emission when constructing classification models using massive data， a two-stage data selection method TSDS （Two-Stage Data Selection） was proposed for low energy consumption and high classifier performance. Firstly， the clustering center was determined by modifying the cosine similarity， and the sample data was split and hierarchically clustered on the basis of dissimilar points. Then， the clustering results were sampled adaptively according to the data distribution， so as to obtain a high-quality subset. Finally， the subset was used to train on the classification model， which accelerated the training process and improved the model accuracy at the same time. Support Vector Machine （SVM） and Multi-Layer Perceptron （MLP） classification models were constructed on six datasets， including Spambase， Bupa and Phoneme， to verify the performance of TSDS. Experimental results show that when the sample data compression ratio reaches 85.00%， TSDS can improve the classification model accuracy by 3 to 10 percentage points， and accelerates model training at the same time， with reducing the energy consumption of SVM classifiers by average 93.76%， and reducing that of MLP classifiers by average 75.41%. It can be seen that TSDS can shorten the training time and reduce the energy consumption， as well as improve the performance of classifiers in classification tasks in big data scenarios， thereby helping to achieve the “carbon peaking and carbon neutrality” goal.

Key words: classifier, hierarchical clustering, adaptive sampling, data selection, few-shot learning