Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Sentence embedding optimization based on manifold learning
Mingyue WU, Dong ZHOU, Wenyu ZHAO, Wei QU
Journal of Computer Applications    2023, 43 (10): 3062-3069.   DOI: 10.11772/j.issn.1001-9081.2022091449
Abstract278)   HTML11)    PDF (1411KB)(146)       Save

As one of the core technologies of natural language processing, sentence embedding affects the quality and performance of natural language processing system. However, the existing methods are unable to infer the global semantic relationship between sentences efficiently, which leads to the fact that the semantic similarity measurement of sentences in Euclidean space still has some problems. To address the issue, a sentence embedding optimization method based on manifold learning was proposed. In the method, Local Linear Embedding (LLE) was used to perform double weighted local linear combinations to the sentences and their semantically similar sentences, thereby preserving the local geometric information between sentences and providing helps to the inference of the global geometric information. As a result, the semantic similarity of sentences in Euclidean space was closer to the real semantics of humans. Experimental results on seven text semantic similarity tasks show that the proposed method has the average Spearman’s Rank Correlation Coefficient, (SRCC) improved by 1.21 percentage points compared with the contrastive learning-based method SimCSE (Simple Contrastive learning of Sentence Embeddings). In addition, the proposed method was applied to mainstream pre-trained models. The results show that compared to the original pre-trained models, the models optimized by the proposed method have the average SRCC improved by 3.32 to 7.70 percentage points.

Table and Figures | Reference | Related Articles | Metrics
Enhancement of topology preservation of self-organizing map
Xiang-Dong ZHOU
Journal of Computer Applications    2009, 29 (12): 3256-3258.  
Abstract1493)      PDF (486KB)(1146)       Save
In the Self-Organization Map (SOM), the weight vectors of the units in the grid are updated only according to the distance between the units and the Best Matching Unit (BMU), so the topological relationship between input data can not be preserved very well. Therefore, two improved schemes were proposed. In the first improved scheme, the weight vectors of the units were updated according to the differences of the corresponding coordinates between the units and the BMU. Experimental results show that this improved scheme can preserve topological relationship very well, but the distribution density of the input data can not be reflected quite well. In the second improved scheme, the weight vectors of the units were updated both according to the differences of the corresponding coordinates and the distance between the units and the BMU. Experimental results show that this improved scheme can not only preserve topological relationship better than SOM, but also reflect the distribution density of the input data quite well and accelerate the convergence speed of the training.
Related Articles | Metrics