Table of Content

10 October 2022, Volume 42 Issue 10

Previous Issue    Next Issue

Artificial intelligence

Survey of event extraction

Chunming MA, Xiuhong LI, Zhe LI, Huiru WANG, Dan YANG

2022, 42(10):  2975-2989.  DOI: 10.11772/j.issn.1001-9081.2021081542

Asbtract ( )   HTML ( )   PDF (3054KB) ( )  

Figures and Tables | References | Related Articles | Metrics

The event that the user is interested in is extracted from the unstructured information， and then displayed to the user in a structured way， that is event extraction. Event extraction has a wide range of applications in information collection， information retrieval， document synthesis， and information questioning and answering. From the overall perspective， event extraction algorithms can be divided into four categories： pattern matching algorithms， trigger lexical methods， ontology-based algorithms， and cutting-edge joint model methods. In the research process， different evaluation methods and datasets can be used according to the related needs， and different event representation methods are also related to event extraction research. Distinguished by task type， meta-event extraction and subject event extraction are the two basic tasks of event extraction. Among them， meta-event extraction has three methods based on pattern matching， machine learning and neural network respectively， while there are two ways to extract subjective events： based on the event framework and based on ontology respectively. Event extraction research has achieved excellent results in single languages such as Chinese and English， but cross-language event extraction still faces many problems. Finally， the related works of event extraction were summarized and the future research directions were prospected in order to provide guidelines for subsequent research.

Chinese event detection based on data augmentation and weakly supervised adversarial training

Ping LUO, Ling DING, Xue YANG, Yang XIANG

2022, 42(10):  2990-2995.  DOI: 10.11772/j.issn.1001-9081.2021081521

Asbtract ( )   HTML ( )   PDF (720KB) ( )  

Figures and Tables | References | Related Articles | Metrics

The existing event detection models rely heavily on human-annotated data， and supervised deep learning models for event detection task often suffer from over-fitting when there is only limited labeled data. Methods of replacing time-consuming human annotation data with auto-labeled data typically rely on sophisticated pre-defined rules. To address these issues， a BERT （Bidirectional Encoder Representations from Transformers） based Mix-text ADversarial training （BMAD） method for Chinese event detection was proposed. In the proposed method， a weakly supervised learning scene was set on the basis of data augmentation and adversarial learning， and a span extraction model was used to solve event detection task. Firstly， to relieve the problem of insufficient data， various data augmentation methods such as back-translation and Mix-Text were applied to augment data and create weakly supervised learning scene for event detection. And then an adversarial training mechanism was applied to learn with noise and improve the robustness of the whole model. Several experiments were conducted on commonly used real-world dataset Automatic Context Extraction （ACE） 2005. The results show that compared with algorithms such as Nugget Proposal Network （NPN）， Trigger-aware Lattice Neural Network （TLNN） and Hybrid-Character-Based Neural Network （HCBNN）， the proposed method has the F1 score improved by at least 0.84 percentage points.

Chinese Text-to-SQL model for industrial production

Jianqing LYU, Xianbing WANG, Gang CHEN, Hua ZHANG, Minggang WANG

2022, 42(10):  2996-3002.  DOI: 10.11772/j.issn.1001-9081.2021081525

Asbtract ( )   HTML ( )   PDF (1886KB) ( )  

Figures and Tables | References | Related Articles | Metrics

When the model of translating English natural language questions into Structured Query Language （SQL） statements （Text-to-SQL） is migrated to Chinese industrial Text-to-SQL task， due to the poor interpretability and strong dispersion of industrial datasets， the representation format of the information of table names and column names in database are often inconsistent with the key information in questions， and the column names in questions are often hidden in the semantics， which leads to a lower exact match accuracy. Aiming at the problems appeared in migration， the corresponding solution was proposed and a modified model was constructed. Firstly， in data use process， factory metadata information was used to solve problem of inconsistency in representation format and the problem that the column names were hidden in the semantics. Then， according to the characteristics of Chinese language expression， a self-attention model based on relative position was used to directly identify the value of where clause by questions and database mode information. Finally， according to the characteristics of the query of industrial questions， the fine-tuned Bidirectional Encoder Representation from Transformers （BERT） was used to classify questions in order to improve the accuracy of SQL statement structure prediction. An industrial dataset based on the aluminum smelting industry was constructed and experimental verification was performed on this dataset. The results show that the exact match accuracy of the proposed model on the industrial test set is 74.2%. Compared with the effect of the mainstream models on English dataset Spider， it can be seen that the proposed model can effectively deal with the Chinese industrial Text-to-SQL task.

Fine-grained entity typing method based on hierarchy awareness

Binhong XIE, Shuning LI, Yingjun ZHANG

2022, 42(10):  3003-3010.  DOI: 10.11772/j.issn.1001-9081.2021101792

Asbtract ( )   HTML ( )   PDF (1252KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Most work of Fine-Grained Entity Typing （FGET） focuses on how the semantic information of mention and context can be better coded， while ignoring the dependency between labels in the label hierarchy and their semantic information. In order to solve the above problem， a Hierarchy-Aware Fine-Grained Entity Typing （HAFGET） method was proposed. Firstly， the hierarchical encoder based on Graph Convolutional Network （GCN） was used to model the dependency between labels in different levels. Hierarchy-Aware Fine-Grained Entity Typing Multi-Label Attention （HAFGET-MLA） model and Hierarchy-Aware Fine-Grained Entity Typing Mention Feature Propagation （HAFGET-MFP） model were proposed. Then， HAFGET-MLA model and HAFGET-MFP model were carried out by using multi-label attention model and mention feature propagation model. In the former， the hierarchical perceptual label embedded was learned through the hierarchical encoder and the labels were classified after attention fusion with the mention features. In the latter， the mention features were directly input into the hierarchical encoder to update the feature representation and then classified. Experimental results on three public datasets FIGER， OntoNotes and KNET show that the accuracy and macro F1 scores of HAFGET-MLA model and HAFGET-MFP model are both improved by more than 2% compared with those of the baseline model. It is verified that the proposed method can effectively improve the typing effect.

Named entity recognition based on BERT and joint learning for judgment documents

Lanlan ZENG, Yisong WANG, Panfeng CHEN

2022, 42(10):  3011-3017.  DOI: 10.11772/j.issn.1001-9081.2021091565

Asbtract ( )   HTML ( )   PDF (1601KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Correctly identifying the entities in judgment documents is an important foundation for building legal knowledge graph and realizing smart courts. However， commonly used Named Entity Recognition （NER） models cannot solve the problem of polysemous word representation and entity boundary recognition errors in judgment document well. In order to effectively improve the recognition effect of various entities in the judgment documents， a Bidirectional Long Short-Term Memory with a sequential Conditional Random Field （BiLSTM-CRF） based on Joint Learning and BERT （Bidirectional Encoder Representation from Transformers） （JLB-BiLSTM-CRF） model was proposed. Firstly， the input character sequence was encoded by BERT to enhance the representation ability of word vectors. Then， the long text information was modeled by BiLSTM network， and the NER tasks and Chinese Word Segmentation （CWS） tasks were jointly trained to improve the boundary recognition rate of entities. Experimental results show that this model has the precision of 94.36%， the recall of 94.94%， and the F1 score of 94.65% on the test set， which are 1.05 percentage points， 0.48 percentage points and 0.77 percentage points higher than those of BERT-BiLSTM-CRF model respectively， verifying the effectiveness of JLB-BiLSTM-CRF model in NER tasks for judgment documents.

Cross-modal tensor fusion network based on semantic relation graph for image-text retrieval

Changhong LIU, Sheng ZENG, Bin ZHANG, Yong CHEN

2022, 42(10):  3018-3024.  DOI: 10.11772/j.issn.1001-9081.2021091622

Asbtract ( )   HTML ( )   PDF (2407KB) ( )  

Figures and Tables | References | Related Articles | Metrics

The key of cross-modal image-text retrieval is how to capture the semantic correlation between images and text effectively. Most of the existing methods learn the global semantic correlation between image region features and text features or local semantic correlation between inter-modality objects， and ignore the correlation between the intra-modality object relationships and inter-modality object relationships. To solve this problem， a method of Cross-Modal Tensor Fusion Network based on Semantic Relation Graph （CMTFN-SRG） for image-text retrieval was proposed. Firstly， the relationships of image regions and text words were generated by Graph Convolutional Network （GCN） and Bidirectional Gated Recurrent Unit （Bi-GRU） respectively. Then， the fine-grained semantic correlation between the data of two modals was learned by using the tensor fusion network to match the learned semantic relation graph of image regions and the graph of text words. At the same time， Gated Recurrent Unit （GRU） was used to learn global features of the image， and the global features of the image and the text were matched to capture the inter-modality global semantic correlation. The proposed method was compared with the Multi-Modality Cross Attention （MMCA） method on the benchmark datasets Flickr30K and MS-COCO. Experimental results show that the proposed method improves the Recall@1 of text-to-image retrieval task by 2.6%， 9.0% and 4.1% respectively on the test datasets Flickr30K， MS-COCO1K and MS-COCO5K.And mean Recall （mR） improves by 0.4， 1.3 and 0.1 percentage points respectively. It can be seen that the proposed method can effectively improve the precision of image-text retrieval.

Few-shot object detection based on attention mechanism and secondary reweighting of meta-features

Runchao LIN, Rong HUANG, Aihua DONG

2022, 42(10):  3025-3032.  DOI: 10.11772/j.issn.1001-9081.2021091571

Asbtract ( )   HTML ( )   PDF (2381KB) ( )  

Figures and Tables | References | Related Articles | Metrics

In the few-shot object detection task based on transfer learning， due to the lack of attention mechanism to focus on the object to be detected in the image， the ability of the existing models to suppress the surrounding background area of the object is not strong， and in the process of transfer learning， it is usually necessary to fine-tune the meta-features to achieve cross-domain sharing， which will cause meta-feature shift， and lead to the decline of the model’s ability to detect large-sample images. To solve the above problems， an improved meta-feature transfer model Up-YOLOv3 based on the attention mechanism and the meta-feature secondary reweighting mechanism was proposed. Firstly， the Convolution Block Attention Module （CBAM）-based attention mechanism was introduced in the original meta-feature transfer model Base-YOLOv2， so that the feature extraction network was able to focus on the object area in the image and pay attention to the detailed features of the image object class， thereby improving the model’s detection performance for few-shot image objects. Then， the Squeeze and Excitation-Secondary Meta-Feature Reweighting （SE-SMFR） module was introduced to reweight the meta-features of the large-sample image for the second time in order to obtain the secondary reweighted meta-features， so that the model was not only able to improve the performance of few-shot object detection， but also able to reduce the weight shift of the meta-feature information of the large-sample image. Experimental results on PASCAL VOC2007/2012 dataset show that， compared with Base-YOLOv2， Up-YOLOv3 has the detection mean Average Precision （mAP） for few-shot object images increased by 2.3 to 9.1 percentage points； compared with the original meta-feature transfer model based on YOLOv3 Base-YOLOv3， mAP for large-sample object images increased by 1.8 to 2.4 percentage points. It can be seen that the improved model has good generalization ability and robustness for both large-sample images and few-shot images of different classes.

Neural network conversion method for dynamic event stream

Yuhao ZHANG, Mengwen YUAN, Yujing LU, Rui YAN, Huajin TANG

2022, 42(10):  3033-3039.  DOI: 10.11772/j.issn.1001-9081.2021091607

Asbtract ( )   HTML ( )   PDF (2014KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Since Convolutional Neural Network （CNN） conversion method based on the weight normalization method for event stream data has a large loss of accuracy and the effective deployment of floating-point networks is difficult on hardware， a network conversion method for dynamic event stream was proposed. Firstly， the event stream data was reconstructed as the input of CNN for training. In the training process， the quantized activation function was adopted to reduce the accuracy loss， and a symmetric fixed-point quantization method was used to reduce the parameter storage. Then， instead of equivalence principle， pulse count equivalence principle was used to adapt to the sparsity of data better. Experimental results show that on three datasets N-MNIST， POKER-DVS and MNIST-DVS， compared with using the traditional activation function， Spiking Convolutional Neural Network （SCNN） using the quantized activation function has the recognition accuracy improved by 0.29 percentage points， 8.52 percentage points and 3.95 percentage points respectively， and the conversion loss reduced by 21.77%， 100.00% and 92.48% respectively. Meanwhile， the proposed quantized SCNN can effectively save 75% of storage space compared with high-precision SCNN generated on the basis of the weight normalization method， and has the conversion loss on N-MNIST and MNIST-DVS datasets reduced by 6.79% and 46.29% respectively.

Relation extraction algorithm based on residual shrinkage network

Quan YUAN, Shuxin XUE

2022, 42(10):  3040-3045.  DOI: 10.11772/j.issn.1001-9081.2021081473

Asbtract ( )   HTML ( )   PDF (1328KB) ( )  

Figures and Tables | References | Related Articles | Metrics

An improved algorithm based on residual shrinkage network with soft threshold module was proposed to solve the problem of noise caused by interference between words within a sentence in relation extraction. Firstly， the threshold was trained in each feature channel of the residual network. The threshold had two characteristics： first， its absolute value would not be too large， if it was too large， effective information would be eliminated； second， the threshold had different results for different input training. Secondly， according to the characteristics of soft threshold， the channel features lower than the threshold were deleted， and those higher than the threshold were reduced. Compared with direct deletion of negative features， soft threshold was able to save useful information of negative features. Finally， an optimization model of attention module was added to reduce the influence of mislabeling problem in distant supervision. Piecewise Convolutional Neural Network （PCNN）， Bi-directional Long Short-Term Memory （BiLSTM） network and ordinary Residual Network （ResNet） were selected as baseline models for comparison experiments. Experimental results show that the precision-recall curves of the proposed model include the curves of other models and the F1 scores of the proposed model are increased by 6.0 percentage points， 3.9 percentage points and 1.4 percentage points respectively compared to the baseline models， which verifies that addition of soft thresholding network model can improve accuracy of relation extraction by reducing in-sentence noise.

Automatic feature selection algorithm based on interaction of ReliefF with maximum information coefficient and SVM

Qian GE, Guangbin ZHANG, Xiaofeng ZHANG

2022, 42(10):  3046-3053.  DOI: 10.11772/j.issn.1001-9081.2021081486

Asbtract ( )   HTML ( )   PDF (793KB) ( )  

Figures and Tables | References | Related Articles | Metrics

In order to solve the problems of feature selection ReliefF algorithm， such as poor algorithm stability and low classification accuracy for selected feature subsets caused by using Euclidean distance to select the nearest neighbor samples， an MICReliefF （Maximum Information Coefficient-ReliefF） algorithm based on Maximum Information Coefficient （MIC） was proposed. At the same time， the classification accuracy of the Support Vector Machine （SVM） model was used as the evaluation index， and the optimal feature subset was automatically determined by multiple optimizations， thereby realizing the interactive optimization of the MICReliefF algorithm and the classification model， that is the MICReliefF-SVM automatic feature selection algorithm. The performance of the MICReliefF-SVM algorithm was verified on several UCI public datasets. Experimental results show that the MICReliefF-SVM automatic feature selection algorithm cannot only filter out more redundant features， but also select the feature subsets with good stability and generalization ability. Compared with Random Forest （RF）， max-Relevance and Min-Redundancy （mRMR）， Correlation-based Feature Selection （CFS） and other classical feature selection algorithms， MICReliefF algorithm has higher classification accuracy.

Recommendation algorithm for online learning resources based on double-end neighbor fusion knowledge graph

Haiwei FAN, Ruichi ZHANG, Yisheng AN, Jiajie QIN

2022, 42(10):  3054-3059.  DOI: 10.11772/j.issn.1001-9081.2021091629

Asbtract ( )   HTML ( )   PDF (1731KB) ( )  

Figures and Tables | References | Related Articles | Metrics

For the monotonicity of learning resources recommended by the collaborative filtering algorithms may lead to the difficulty of meeting the need of personalized resource acquisition of learners， a recommendation algorithm for online learning resources based on double-end neighbor fusion knowledge graph was proposed. Firstly， on the user end， the information of the entities and their neighbors between the learner’s existing knowledge nodes and the new knowledge nodes were aggregated to obtain the embedding representation of the learner in order to capture the learner’s personalized requirements. Secondly， on the project end， the neighborhood information of learning resources was used to expand the semantics and embedding representation of the learning resources. Finally， the user embedding representation and the project embedding representation were sent to the fully connected layer to obtain the interaction probability of them. To verify the effectiveness of the proposed algorithm， comparison experiments were performed using the public dataset MOOPer. Experimental results show that on this dataset， the proposed algorithm improves 1.12 percentage points and 1.31 percentage points on AUC （Area Under Curve） and accuracy respectively compared to the optimal baseline model， and achieves certain improvement on both of Precision@K and Recall@K.

Data science and technology

Link prediction in directed network based on high-order self-included collaborative filtering

Guangfu CHEN, Haibo WANG, Yanping LIAN

2022, 42(10):  3060-3068.  DOI: 10.11772/j.issn.1001-9081.2021081484

Asbtract ( )   HTML ( )   PDF (1649KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Aiming at the problem that most existing directed network link prediction methods only focus on the directed local and reciprocal link information and ignore the directed global structure information， a High-order Self-included Collaborative Filtering （HSCF） link prediction framework was proposed. Firstly， random walk method was used to calculate the high-order similarity matrix to preserve the high-order path information of the directed network. Secondly， an HSCF framework was constructed by combining the high-order similarity matrix with collaborative filtering method. Finally， the proposed framework was integrated with four typical directed structure similarity indices including Directed Common Neighbor （DCN）， Directed Adamic-Adar （DAA）， Directed Resource Allocation （DRA） and potential theory （Bifan）， and four directed network prediction indices HSCF-DCN， HSCF-DAA， HSCF-DRA and HSCF-Bifan were proposed on this basis. Compared with the baseline indices on ten real directed networks， the experimental results show that the AUC （Area Under Curve of Receiver Operating Characteristic （ROC）） values of HSCF-DCN， HSCF-DAA， HSCF-DRA and HSCF-Bifan are increased by an average of 8.16%， 8.85%， 9.64% and 10.33% respectively and the F-score values of them are increased by an average of 66.62%， 68.32%， 68.95% and 76.18% respectively.

Cyber security

Design and analysis of dynamic S-box based on anti-degradation chaotic system

Geng ZHAO, Senmin ZHANG, Yingjie MA, Shirui GAO

2022, 42(10):  3069-3073.  DOI: 10.11772/j.issn.1001-9081.2021081500

Asbtract ( )   HTML ( )   PDF (3094KB) ( )  

Figures and Tables | References | Related Articles | Metrics

S-box is one of the key components in block cipher algorithm， and its confounding and scrambling effect determines security strength of the whole cipher. In order to ensure better cryptographic performance of S-boxes generated based on chaotic systems， a dynamic S-box design scheme based on anti-degradation chaotic system was proposed. Firstly， Chebyshev chaotic map was disturbed by Lorenz chaotic map. Then， two initial S-boxes were generated by interception method and interval partition method. Finally， the final S-box was obtained by using index sort perturbation method. The chaotic sequences generated by anti-degenerate chaotic system do not have phenomenon of short cycle， and have characteristics of ergodic property and unpredictability. When applying these sequences to the design of S-box， the safety performance of S-box can be greatly improved， and the hidden danger of chaos generation source can be eliminated. Moreover， dynamic S-boxes can be generated in batches by adjusting system parameters. The safety performance of S-box， including nonlinearity， difference uniformity， strict avalanche criterion， output bits independence criterion and bijective characteristics， was tested and compared. Experimental results show that the S-box generated by the proposed scheme has better cryptographic performance and can be used in design of block cipher.

Blockchain-based data frame security verification mechanism in software defined network

Hexiong CHEN, Yuwei LUO, Yunkai WEI, Wei GUO, Feilu HANG, Zhengxiong MAO, Zhenhong ZHANG, Yingjun HE, Zhenyu LUO, Linjiang XIE, Ning YANG

2022, 42(10):  3074-3083.  DOI: 10.11772/j.issn.1001-9081.2021081450

Asbtract ( )   HTML ( )   PDF (2979KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Forged and tampered data frames should be identified and filtered out to ensure network security and efficiency. However， the existing schemes usually fail to work when verification devices are attacked or maliciously controlled in the Software Defined Network （SDN）. To solve the above problem， a blockchain-based data frame security verification mechanism was proposed. Firstly， a Proof of Frame Forwarding （PoFF） consensus algorithm was designed and used to build a lightweight blockchain system. Then， an efficient data frame security verifying scheme for SDN data frame was proposed on the basis of this blockchain system. Finally， a flexible semi-random verifying scheme was presented to balance the verification efficiency and the resource cost. Simulation results show that compared with the hash chain based verifying scheme， the proposed scheme decreases the missed detection rate significantly when an equal proportion of switches are maliciously controlled. Specifically， when the proportion is 40%， the decrease effect is very obvious， the missed detection rate can still be kept no more than 32% in the basic verification mode， and can be further reduced to 7% with the assistance of the semi-random verifying scheme. Both are much lower than the missed detection rate of 72% in the hash chain based verifying scheme， and the resource overhead and communication cost introduced by the proposed mechanism are within a reasonable range. Additionally， the proposed scheme can still maintain good verification performance and efficiency even when the SDN controller is completely unable to work.

Tor website traffic analysis model based on self-attention mechanism and spatiotemporal features

Rongkang XI, Manchun CAI, Tianliang LU, Yanlin LI

2022, 42(10):  3084-3090.  DOI: 10.11772/j.issn.1001-9081.2021081452

Asbtract ( )   HTML ( )   PDF (2633KB) ( )  

Figures and Tables | References | Related Articles | Metrics

The onion router （Tor） anonymous communication system is used by criminals to engage in criminal activities on the dark networks， which brings severe challenges to social security. Tor website traffic is captured and analyzed by Tor website traffic analysis technology and therefore illegal behaviors hidden on the internet are timely discovered to conduct network supervision. Based on this， a Tor website traffic analysis model based on Self-Attention and Hierarchical SpatioTemporal （SA-HST） features was proposed on the basis of self-attention mechanism and spatiotemporal features. Firstly， attention mechanism was introduced to assign different weights to the network traffic features to highlight the important features. Then， Convolutional Neural Network （CNN） with multi-channel parallel structure and Long Short-Term Memory （LSTM） network were used to extract the spatiotemporal features of input data. Finally， Softmax function was used to classify data. SA-HST can achieve 97.14% accuracy in closed world scenario， which is 8.74 percentage points and 7.84 percentage points higher compared to CUMUL（CUMULative sum fingerprinting） model and deep learning model CNN. In open world scenario， SA-HST has the evaluation indicators of confusion matrix above 96% stably. Experimental results show that self-attention mechanism can achieve efficient feature extraction under lightweight model structure. By capturing important， multi-view spatiotemporal features of anonymous traffic for classification， SA-HST has certain advantages in terms of classification accuracy， training efficiency and robustness.

Certificateless signature scheme with strong privacy protection for internet of vehicles

Dong ZHU, Xinchun YIN, Jianting NING

2022, 42(10):  3091-3101.  DOI: 10.11772/j.issn.1001-9081.2021091630

Asbtract ( )   HTML ( )   PDF (1079KB) ( )  

Figures and Tables | References | Related Articles | Metrics

To guarantee the communication security of Internet of Vehicles （IoV） and reduce the overhead caused by updating vehicles private key frequently， firstly， the existing certificateless aggregate signature schemes were proved vulnerable to public key replacement attacks and malevolent Key Generation Center （KGC） attack at the same time. Secondly， a certificateless aggregate signature scheme with strong privacy protection and suitable for IoV was proposed. In the proposed scheme， by introducing pseudonymous identities， vehicles’ identities were hidden and trusted authority was capable of tracing malicious vehicles after the events. Meanwhile， vehicles’ pseudonymous identities and public keys were able to be updated dynamically with the change of the area in the proposed scheme. In this way， it was not only able to ensure the safety of vehicles’ trajectories， but also able to avoid the communication and storage overhead brought by frequent private key update effectively. Under the assumption of the Elliptic Curve Discrete Logarithm （ECDL） problem， security proof shows that the proposed scheme satisfies authentication and integrity under the random oracle model. Moreover， anonymity， traceability and strong privacy protection are also provided by the proposed scheme. At the same time， aggregate signature technology was used to realize the aggregated verification of vehicle signatures in the scheme， which reduced the computational cost of verifying the signature. Performance analysis shows that when the number of signatures contained in the aggregate signature is 100， the communication overhead of transmitting aggregated signatures by the proposed scheme is reduced by at least approximately \mathrm{21.4}\mathrm{\%} compared with the other related schemes.

Deep robust watermarking algorithm based on multiscale knowledge learning

Bin FAN, Zhi LI, Jian GAO

2022, 42(10):  3102-3110.  DOI: 10.11772/j.issn.1001-9081.2021050737

Asbtract ( )   HTML ( )   PDF (3245KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Aiming at the problem that existing watermarking algorithms based on deep learning cannot effectively protect the copyright of high-dimensional medical images， a medical image watermarking algorithm based on multiscale knowledge learning was proposed for the copyright protection of diffusion-weighted images. First， a watermark embedding network based on multiscale knowledge learning was proposed to embed watermarks， and the semantic， texture， edge and frequency domain information of the diffusion-weighted image was extracted by a fine-tuned pre-training network as multiscale knowledge features. Then， the multiscale knowledge features were combined to reconstruct the diffusion-weighted image， and a watermark was embedded during the process redundantly to obtain a watermarked diffusion-weighted image highly similar to the original one visually. Finally， a watermark extraction network based on pyramid feature learning was proposed to improve the robustness of the algorithm by learning the distribution correlation of watermarking signals from different scales of context in the watermarked diffusion-weighted image. Experimental results show that the average Peak Signal-to-Noise Ratio （PSNR） of the reconstructed watermarked images by the proposed algorithm reaches 57.82 dB. Since diffusion-weighted images need to meet certain diffusivity features when converting to diffusion tensor images， the proposed algorithm only has 8 pixel points with the deflection angle of the principal axis direction greater than 5°， and none of these 8 pixel points is in the region of interest of the image. Besides， both of the Fraction Anisotropy （FA） and the Mean Diffusivity （MD） of the image generated by the proposed algorithm are close to 0， which fully meets the requirements of clinical diagnosis. At the same time， facing common noise attacks such as those with cropping strength less than 0.7 and rotation angle less than 15， the proposed algorithm achieves more than 95% watermarking accuracy and can effectively protect the copyright information of diffusion-weighted images.

Network and communications

Survey of named data networking

Hongqiao MA, Wenzhong YANG, Peng KANG, Jiankang YANG, Yuanshan LIU, Yue ZHOU

2022, 42(10):  3111-3123.  DOI: 10.11772/j.issn.1001-9081.2021091576

Asbtract ( )   HTML ( )   PDF (2976KB) ( )  

Figures and Tables | References | Related Articles | Metrics

The unique advantages of Named Data Networking （NDN） make it a candidate for the next generation of new internet architecture. Through the analysis of the communication principle of NDN and the comparison of it with the traditional Transmission Control Protocol/Internet Protocol （TCP/IP） architecture， the advantages of the new architecture were described. And on this basis， the key elements of this network architecture design were summarized and analyzed. In addition， in order to help researchers better understand this new network architecture， the successful applications of NDN after years of development were summed up. Following the mainstream technology， the support of NDN for cutting-edge blockchain technology was focused on. Based on this support， the research and development of the applications of NDN and blockchain technology were discussed and prospected.

Multi-input multi-output intelligent receiver model based on multi-label classification algorithm

Anyi WANG, Heng ZHANG

2022, 42(10):  3124-3129.  DOI: 10.11772/j.issn.1001-9081.2021081535

Asbtract ( )   HTML ( )   PDF (2392KB) ( )  

Figures and Tables | References | Related Articles | Metrics

The traditional wireless communication system is composed of transmitters and receivers. The information to be transmitted is transmitted through antenna after channel coding， modulation， and shaping. Due to the influence of factors such as channel fading， noise， and interference， signals arriving at the receiver will have serious distortion， and the receiver needs to recover original information from distorted signals as much as possible. To solve this problem， a Multi-Input Multi-Output （MIMO） intelligent receiver model based on multi-label classification neural network was proposed. In this model， Deep Neural Network （DNN） was used to replace the entire information recovery link of receiver from signals to information， and multi-label classification algorithm was used to replace multiple binary classifiers to achieve multi-bit information flow recovery. The training dataset has quadrature signals that contains two modulation modes including Binary Phase Shift Keying （BPSK） and Quadrature Phase Shift Keying （QPSK） as well as two coding modes of Hamming coding and cyclic coding. Experimental results show that under conditions such as noise， Rayleigh fading， and interference， when the Bit Error Rate （BER） of receiver using the traditional Alamouti decoding method is 1E-3， the intelligent receiver realizes the recovered information with the BER of 0. While maintaining the same BER performance， the proposed multi-label classification algorithm reduces the training time of each batch by about 4 min compared with the multiple binary classifier algorithms of the comparison model.

Joint optimization method of full-duplex cognitive relay network based on nonlinear energy harvesting

Lingzhao WANG, Runhe QIU

2022, 42(10):  3130-3139.  DOI: 10.11772/j.issn.1001-9081.2021081460

Asbtract ( )   HTML ( )   PDF (2891KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Aiming at the influence of the time consumption of the information transmission process and the channel estimation error on the energy efficiency of the network， a joint optimization method of full-duplex cognitive relay network based on nonlinear energy harvesting was proposed. The proposed method was based on the use of nonlinear energy harvesting in the relay and consideration the imperfect Channel State Information （CSI）. Firstly， the energy efficiency non-convex optimization problem was transformed into two convex sub-optimization problems to obtain the transmission power of the secondary user and the relay as well as the collected energy. Secondly， under the condition that the primary user interference threshold was guaranteed and the optimal transmission power was non-negative， the range of transmission channel capacity was obtained. Finally， the transmission power was substituted into the expression to obtain the time related objective function， and the Hessian matrix was used to prove that the objective function was a convex function， the optimal transmission time and power splitting factor were calculated， and the optimal solution of energy efficiency was obtained. Experimental results show that under the same conditions， the energy efficiency of the proposed joint optimization method is about 84.3% higher than that of only optimizing the transmission power. At the same time， it is verified that when the channel estimation error factor is 0.01， the energy efficiency of the network is reduced by about 1.9% by using the proposed method.

Task offloading and resource allocation based on simulated annealing algorithm in C-V2X internet of vehicles

Zhi LI, Jianbin XUE

2022, 42(10):  3140-3147.  DOI: 10.11772/j.issn.1001-9081.2021081490

Asbtract ( )   HTML ( )   PDF (2270KB) ( )  

Figures and Tables | References | Related Articles | Metrics

When big data flow calculation tasks with different attributes generated by networked vehicle nodes are transmitted and offloaded， issues such as time delay jitter， large computational energy consumption and system overhead usually happen. Therefore， according to the actual communication environment， a scheme for task offloading and resource allocation based on Simulated Annealing Algorithm （SAA） in Cellular Vehicle to Everything （C-V2X） Internet of Vehicles （IoV） was proposed. Firstly， according to the task processing priority， the tasks with high processing priority were processed by collaborative offloading and computing. Secondly， an SAA-based task offloading strategy was developed with the aid of globally searching for the optimal offloading scale factor. And the task offloading scale factor was analyzed and optimized. Finally， during the update process of task offloading scale factor， the problem of minimizing the system overhead was transformed into the convex optimization problem of power and computational resource allocation. And the Lagrange multiplier method was used to obtain the optimal solution. By comparing the proposed algorithm with the local offloading and adaptive genetic algorithm， it can be seen that： as the calculation task data size increases， the time delay， power consumption and system overhead of the adaptive genetic algorithm are decreased by 5.97%， 49.40%， and 49.36% respectively， compared with those of the local offloading. On this basis， the time delay， power consumption and system overhead of the proposed SAA-based scheme are further decreased by 6.35%， 92.27%， and 91.7% respectively， compared with those of the adaptive genetic algorithm. As the CPU cycles of the calculation task increase， the time delay， power consumption and system overhead of the adaptive genetic algorithm are decreased by 16.4%， 49.58%， and 49.23% respectively， compared with local offloading. On this basis， the time delay， power consumption and system overhead of the proposed SAA-based scheme are further decreased by 19.61%， 94.39%， and 89.88% respectively， compared with those of the adaptive genetic algorithm. Experimental results show that SAA cannot only reduce the time delay， power consumption and system overhead of communication systems but also accelerate convergence of the results.

Wireless virtual network embedding algorithm based on load balance

Qi GAO, Na LYU, Jingcheng MIAO

2022, 42(10):  3148-3153.  DOI: 10.11772/j.issn.1001-9081.2022010043

Asbtract ( )   HTML ( )   PDF (1736KB) ( )  

Figures and Tables | References | Related Articles | Metrics

To address the rigid problem of network， NV （Network Virtualization） is widely used， and the key technology of NV is VNE （Virtual Network Embedding）. To solve the problem of unbalanced power and bandwidth resource usage in the process of wireless VNE， a joint hierarchical resource wireless VNE algorithm was proposed based on the load balance principle. Firstly， a new node resource ranking method was adopted， which taking node power and average link bandwidth as the ranking basis. Secondly， the resources were ranked to dynamically adjust the power and bandwidth demanded by virtual network requests. Finally， the unit cost of power and bandwidth resources was improved， and the resource allocation scheme was selected with cost minimization as the objective function. Compared with the original wireless VNE algorithm WVNE-JBP （Wireless Virtual Network Embedding-Joint Bandwidth and Power）， the proposed algorithm has the overall acceptance rate increased by 11.7 percentage points， the average power resource utilization increased by 4.4 percentage points and the average bandwidth resource utilization increased by 1.6 percentage points. Experimental results show that the proposed algorithm can effectively improve the virtual network acceptance rate and resource utilization.

Computer software technology

Effective alignment of process model with event logs based on perceived cost

Duoqin LI, Xianwen FANG, Lili WANG, Chifeng SHAO

2022, 42(10):  3154-3161.  DOI: 10.11772/j.issn.1001-9081.2021081378

Asbtract ( )   HTML ( )   PDF (3777KB) ( )  

Figures and Tables | References | Related Articles | Metrics

The different importance of the activities in the business process in real world is not taken into account by the existing cost functions， so that in the alignment process of model and log， alignment cost may deviates from perceived cost significantly. To solve this problem， a concept of important synchronization cost function was proposed based on the typical flow characteristic of the behaviors in business processes， and an alignment method that can improve efficiency was proposed under this function. Firstly， the important synchronization cost function was defined based on the concept of perceived cost. Then， the important matching sub-sequence to segment the process model and the log trace was determined according to the log trace and the typical flow characteristic of the behaviors in the process model. Finally， based on the important synchronization cost function， the segmented sub-process and the corresponding log trace subsequence were aligned， and the segmented alignment results were combined to obtain the final alignment result. The experiments were carried out to verify the proposed method from the perspectives of accuracy and efficiency. In terms of accuracy， compared with the existing standard cost function and maximum synchronous cost function， the proposed cost function improved the alignment accuracy by up to 17.44 percentage points， and when the event log contained mixed noise， the proposed cost function had the highest average alignment accuracy of 88.67%. The efficiency of alignment was verified by comparing the time consumed by alignment. The average time of the existing two functions were 1.58 s and 2.21 s respectively， while that of the proposed method was 0.63 s， which was improved by 150.79% and 250.79% respectively. Experimental results show that the proposed method can satisfy the accuracy demand and improve the efficiency of alignment at the same time.

Large-scale Web service composition based on optimized grey wolf optimizer

Xuemin XU, Xiuguo ZHANG, Yuanyuan XIAO, Zhiying CAO

2022, 42(10):  3162-3169.  DOI: 10.11772/j.issn.1001-9081.2021091556

Asbtract ( )   HTML ( )   PDF (2213KB) ( )  

Figures and Tables | References | Related Articles | Metrics

In order to solve the problem that it is difficult to obtain a composite service with high overall performance in a large-scale Web service environment， a large-scale Web service composition method was proposed. Firstly， Document Object Model （DOM） was used to parse the user demand document in XML format to generate an abstract Web service composition sequence. Secondly， the service topic model was used for service filtering， and Top-k specific Web services were selected for each abstract Web service to reduce the composition space. Thirdly， in order to improve the quality and efficiency of service composition， an Optimized Grey Wolf Optimizer based on Logistic chaotic map and Nonlinear convergence factor （OGWO/LN） was proposed to select the optimal service composition plan. In this algorithm， chaotic map was used to generate the initial population for increasing the diversity of service composition plans and avoiding multiple local optimizations. At the same time， a nonlinear convergence factor was proposed to improve the optimization performance of the algorithm by adjusting the algorithm search ability. Finally， OGWO/LN was realized in a parallel way by MapReduce framework. Experimental results on real datasets show that compared with algorithms such as IFOA4WSC （Improved Fruit Fly Optimization Algorithm for Web Service Composition）， MR-IDPSO （MapReduce based on Improved Discrete Particle Swarm Optimization） and MR-GA （MapReduce based on Genetic Algorithm）， the proposed algorithm has the average fitness value increased by 8.69%， 7.94% and 12.25% respectively， and has better optimization performance and stability in solving the problem of large-scale Web service composition.

Static code defect detection method based on deep semantic fusion

Jingyun CHENG, Buhong WANG, Peng LUO

2022, 42(10):  3170-3176.  DOI: 10.11772/j.issn.1001-9081.2021081548

Asbtract ( )   HTML ( )   PDF (2119KB) ( )  

Figures and Tables | References | Related Articles | Metrics

With the increasing scale and complexity of computer softwares， code defect in software has become a serious threat to public safety. Aiming at the problems of poor expansibility of static analysis tools， as well as coarse detection granularity and unsatisfactory detection effect of existing methods， a static code defect detection method based on program slicing and semantic feature fusion was proposed. Firstly， key points in source code were analyzed through data flow and control flow， and the program slicing method based on Interprocedural Finite Distributive Subset （IFDS） was adopted to obtain the code snippet composed of multiple lines of statements related to code defects. Then， semantically related vector representation of code snippet was obtained by word embedding， so that the appropriate length of code snippet was selected with the accuracy guaranteed. Finally， Text Convolutional Neural Network （TextCNN） and Bi-directional Gate Recurrent Unit （BiGRU） were used to extract local key features and context sequence features of the code snippet respectively， and the proposed method was used to detect slice-level code defects. Experimental results show that the proposed method can detect different types of code defects effectively， and is significantly better than static analysis tool Flawfinder. Under the premise of fine granularity， IFDS slicing method can further improve F₁ score and accuracy，reach 89.64% and 92.08% respectively. Compared with the existing methods based on program slicing， when key points are the Application Programming Interface （API） or the variables， the proposed method has the F₁ score reached 89.69% and 89.74% respectively， and the accuracy reached 92.15% and 91.98% respectively， and all of them are higher. It can be seen that without significantly increasing time complexity， the proposed method has a better comprehensive detection performance.

Multimedia computing and computer simulation

Waterweed image segmentation method based on improved U-Net

Qiwen WU, Jianhua WANG, Xiang ZHENG, Ju FENG, Hongyan JIANG, Yubo WANG

2022, 42(10):  3177-3183.  DOI: 10.11772/j.issn.1001-9081.2021091614

Asbtract ( )   HTML ( )   PDF (2407KB) ( )  

Figures and Tables | References | Related Articles | Metrics

During the operation of the Unmanned Surface Vehicles （USVs）， the propellers are easily gotten entangled by waterweeds， which is a problem encountered by the whole industry. Concerning the global distribution， dispersivity， and complexity of the edge and texture of waterweeds in the water surface images， the U-Net was improved and used to classify all pixels in the image， in order to reduce the feature loss of the network， and enhance the extraction of both global and local features， thereby improving the overall segmentation performance. Firstly， the image data of waterweeds in multiple locations and multiple periods were collected， and a comprehensive dataset of waterweeds for semantic segmentation was built. Secondly， three scales of input images were introduced into the network to enable full extraction of the features via the network， and three loss functions for the upsampled images were introduced to balance the overall loss brought by the three different scales of input images. In addition， a hybrid attention module， including the dilated convolution branch and the channel attention enhancement branch， was proposed and introduced to the network. Finally， the proposed network was verified on the newly built waterweed dataset. Experimental results show that the accuracy， mean Intersection over Union （mIoU） and mean Pixel Accuracy （mPA） values of the proposed method can reach 96.8%， 91.22% and 95.29%， respectively， which are improved by 4.62 percentage points， 3.87 percentage points and 3.12 percentage points compared with those of U-Net （VGG16） segmentation method. The proposed method can be applied to unmanned surface vehicles for detection of waterweeds， and perform the corresponding path planning to realize waterweed avoidance.

Hardware reconstruction acceleration method of convolutional neural network-based single image defogging model

Guanjun WANG, Chunlian JIAN, Qiang XIANG

2022, 42(10):  3184-3190.  DOI: 10.11772/j.issn.1001-9081.2021081475

Asbtract ( )   HTML ( )   PDF (3412KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Single image defogging model based on Convolutional Neural Network （CNN） was difficult to deploy on mobile/embedded system and used for real-time video defogging. To solve this problem， a method of hardware reconstruction and acceleration was proposed， based on Zynq System-on-Chip （SoC）. First， a quantization-dequantization algorithm was proposed to perform quantization on two representative defogging models； second， a quantized defogging model was reconstructed and a hardware IP core with Advanced eXtensible Interface 4 （AXI4） was generated， based on video stream memory architecture， hardware/software co-design， pipeline technology and High-Level Synthesis （HLS） tool. Experimental results show that the model parameters can be quantified from float32 to int5（5 bit） under premise of defogging performance， saving about 84.4% of storage space； the highest pixel clock frequency of the generated hardware IP core is 182 Mpixel/s， which can achieve 1080P@60 frame/s video defogging； the hardware IP core processes a single hazy image with the resolution of 640 pixel × 480 pixel only in 2.4 ms， and the on-chip power consumption is only 2.25 W. This hardware IP core with AXI4 is also convenient for cross-platform migration and deployment， which can expand application scope of CNN-based single image defogging model.

Unmanned aerial vehicle image localization method based on multi-view and multi-supervision network

Jinkun ZHOU, Xianlan WANG, Nan MU, Chen WANG

2022, 42(10):  3191-3199.  DOI: 10.11772/j.issn.1001-9081.2021081518

Asbtract ( )   HTML ( )   PDF (2090KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Aiming at the problem of low accuracy of the existing cross-view image matching algorithms， an Unmanned Aerial Vehicle （UAV） image localization method based on Multi-view and Multi-supervision Network （MMNet） was proposed. Firstly， in the proposed method， satellite perspective and UAV perspective were integrated， global and local features were learnt under a unified network architecture， then classification network was trained and metric tasks were performed in multi-supervision way. Specifically， the Reweighted Regularization Triplet loss （RRT） was mainly used by MMNet to learn global features. In this loss， the reweighting and distance regularization strategies were to solve the problems of imbalance of multi-view samples and structure disorder of the feature space. Simultaneously， in order to pay attention to the context information of the central building in target location， the local features were obtained by MMNet via square ring cutting. After that， the cross entropy loss and RRT were used to perform classification and metric tasks respectively. Finally， the global and local features were aggregated by using a weighted strategy to present target location images. MMNet achieved Recall@1 （R@1） of 83.97% and Average Precision （AP） of 86.96% in UAV localization tasks on the currently popular UAV dataset University-1652. Experimental results show that MMNet significantly improves the accuracy of cross-view image matching， and then enhances the practicability of UAV image localization compared with LCM （cross-view Matching based on Location Classification）， SFPN （Salient Feature Partition Network） and other methods.

Cross-modal person re-identification model based on dynamic dual-attention mechanism

Dawei LI, Zhiyong ZENG

2022, 42(10):  3200-3208.  DOI: 10.11772/j.issn.1001-9081.2021081510

Asbtract ( )   HTML ( )   PDF (1713KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Focused on the issue that huge modal difference between cross-modal person re-identification images， pixel alignment and feature alignment are commonly utilized by most of the existing methods to realize image matching. In order to further improve the accuracy of matching two modal images， a multi-input dual-stream network model based on dynamic dual-attention mechanism was designed. Firstly， the neural network was able to learn sufficient feature information in a limited number of samples by adding images of the same person taken by different cameras in each training batch. Secondly， the gray-scale image obtained by homogeneous augmentation was used as an intermediate bridge to retain the structural information of the visible light images and eliminate the color information at the same time. The use of gray-scale images weakened the network’s dependence on color information， thereby strengthening the network model’s ability to mine structural information. Finally， a Weighted Six-Directional triple Ranking （WSDR） loss suitable for images three modalities was proposed， which made full use of cross-modal triple relationship under different angles of view， optimized relative distance between multiple modal features and improved the robustness to modal changes. Experimental results on SYSU-MM01 dataset show that the proposed model increases evaluation indexes Rank-1 and mean Average Precision （mAP） by 4.66 and 3.41 percentage points respectively compared to Dynamic Dual-attentive AGgregation （DDAG） learning model.

Neural style transfer algorithm based on Laplacian operator and color retention

Yongqian TAN, Fanju ZENG

2022, 42(10):  3209-3216.  DOI: 10.11772/j.issn.1001-9081.2021081457

Asbtract ( )   HTML ( )   PDF (3875KB) ( )  

Figures and Tables | References | Related Articles | Metrics

There are some problems in results of the neural style transfer algorithms， such as artifacts， color loss， and blurred contour， which are negative to the overall artistic effect. Therefore， a neural style transfer algorithm based on Laplacian operator and Color Retention （LCR） was proposed. Content loss term， style loss， histogram loss， and Laplacian loss are utilized in the proposed LCR algorithm to construct the total loss function. Because histogram loss and Laplacian loss are used in the LCR algorithm， the proposed algorithm has better overall artistic effect on the stylized result images than Image Style Transfer using Convolutional Neural Networks （IST-CNN） algorithm and Deep Feature Perturbation （DFP） algorithm. Firstly， the influence of image noise on latter calculation of each loss was reduced by denoising input content image and style image. Secondly， the separation of image brightness channel L and color channel a， b was achieved by converting content image and style image from RGB space to Lab space. And the brightness information of content image was transferred to style image to preserve the color of content image. Finally， in Convolutional Neural Network （CNN）， the total loss function was iteratively optimized， and then the stylized result image was output. Compared with IST-CNN and DFP algorithms， the proposed LCR algorithm has the Peak Signal-to-Noise Ratio （PSNR） improved by 12.418 dB and 8.038 dB approximately and respectively， the Structural SIMilarity （SSIM） improved by about 0.348 06 and 0.258 54 approximately and respectively， and the Mean Square Error （MSE） decreased by about 0.653 76 and 0.296 00 respectively. Experimental results show that LCR algorithm has advantage in the overall visual effect of stylized drawing.

Robust speech recognition technology based on self-supervised knowledge transfer

Caitong BAI, Xiaolong CUI, Huiji ZHENG, Ai LI

2022, 42(10):  3217-3223.  DOI: 10.11772/j.issn.1001-9081.2021050808

Asbtract ( )   HTML ( )   PDF (2421KB) ( )  

Figures and Tables | References | Related Articles | Metrics

A robust speech recognition model training algorithm based on self-supervised knowledge transfer was proposed to solve the problems of the increasingly high cost of tagging neural network training data and noise interference hindering performance improvement of speech recognition system. Firstly， three artificial features of the original speech samples were extracted in the pre-processing stage. Then， the advanced features generated by the feature extraction network were fitted to the artificial features extracted in the pre-processing stage through three shallow networks respectively in the training stage. At the same time， the feature extraction front-end and the speech recognition back-end were cross-trained， and their loss functions were integrated. Finally， the advanced features that are more conducive to denoised speech recognition were extracted by the feature extraction network after using the gradient back propagation， thereby realizing the artificial knowledge transfer and denoising as well as using training data efficiently. In the application scenario of military equipment control， the word error rate of the proposed method can be reduced to 0.12 based on the test on three open source Chinese speech recognition datasets THCHS-30 （TsingHua Continuous Chinese Speech）， Aishell-1 and ST-CMDS （Surfing Technology Commands） as well as the military equipment control command dataset. Experimental results show that the proposed method can not only train robust speech recognition models， but also improve the utilization rate of training samples through self-supervised knowledge transfer， and can complete equipment control tasks.

Technical review and case study on classification of Chinese herbal slices based on computer vision

Yi ZHANG, Hua WAN, Shuqin TU

2022, 42(10):  3224-3234.  DOI: 10.11772/j.issn.1001-9081.2021081498

Asbtract ( )   HTML ( )   PDF (4058KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Classifying similar， counterfeit and deteriorated slices in Chinese herbal slices plays a vital role in clinical application of Chinese medicine. Traditional manual identification methods are subjective and fallible. And the classification of traditional Chinese herbal slices based on computer vision is superior in speed and accuracy， which makes Chinese herbal slice screening intelligent. Firstly， general steps of Chinese medicine recognition algorithm based on computer vision were introduced， and technical development status of preprocessing， feature extraction and recognition model of Chinese medicine images were reviewed separately. Then， 12 classes of similar and easily confused Chinese herbal slices were selected as a case to study. By constructing a dataset with 9 156 pictures of Chinese herbal slices， the recognition performance differences of traditional recognition algorithms and various deep learning models were analyzed and compared. Finally， the difficulties and future development trends of computer vision in Chinese herbal slices were summarized and prospected.

Frontier and comprehensive applications

Task allocation method of spatial crowdsourcing based on user satisfaction utility

Peng PENG, Zhiwei NI, Xuhui ZHU

2022, 42(10):  3235-3243.  DOI: 10.11772/j.issn.1001-9081.2021081528

Asbtract ( )   HTML ( )   PDF (1323KB) ( )  

Figures and Tables | References | Related Articles | Metrics

In view of the actual situations such as the preference and the delay waiting of spatial crowdsourcing users of ride-hailing in life， a task allocation method of spatial crowdsourcing based on user satisfaction utility called IGSO（Improved discrete Glowworm Swarm Optimization）-SSCTA（Spatial Crowdsourcing Task Allocation based on user Satisfaction utility） was proposed. Firstly， user satisfaction utility was defined， which was composed of user preference utility， delay waiting utility and task completion expectation. Secondly， SSCTA model was constructed based on user satisfaction utility. Thirdly， IGSO algorithm was proposed by discrete coding， the initialization of reverse learning collaboration， four improved mobile strategies， adaptive selection strategy and treatment of infeasible solutions. Finally， IGSO algorithm was used to solve the above model. Experimental results on different scale datasets show that compared with the three allocation strategies of time minimization， distance minimization and random allocation， the user satisfaction utility of the proposed method is improved by 9.64%， 11.77% and 15.70% respectively， and the proposed algorithm has better stability and convergence than the greedy algorithm and other improved glowworm algorithms.

Cold chain electric vehicle routing problem based on hybrid ant colony optimization

Zhishuo LIU, Ruosi LIU, Zhe CHEN

2022, 42(10):  3244-3251.  DOI: 10.11772/j.issn.1001-9081.2021091572

Asbtract ( )   HTML ( )   PDF (1652KB) ( )  

Figures and Tables | References | Related Articles | Metrics

The trend of green logistics pushes the use of electric vehicles into cold chain logistics. Concerning the problem that maintaining a low temperature environment requires a lot of energy in electric vehicle cold chain distribution， as well as the phenomena that the limited driving range and long charging time of electric vehicles make high operation cost， the Refrigerated Electric Vehicle Routing Problem （REVRP） in electric vehicle distribution was thought deeply. Considering the characteristics of electric vehicle energy consumption and the charging requirements of social recharging stations， a linear programming model was developed with the objective of minizing total distribution cost， and the objective function was composed of fixed cost and variable cost， in the variable cost， transportation cost and cooling cost were included. The capacity constraints and power constraints were considered in the model， and a Hybrid Ant Colony Optimization （HACO） algorithm was designed to solve this model. Especially， more attention was paid to designing transfer rules suitable for social recharging stations and four local optimization operators. Based on improving the Solomon benchmark examples， the small-scale and large-scale example sets were formed， and the performance of ACO algorithm and the optimization operators were through experiments. The experiment results show that ACO algorithm and CPLEX （WebSphere ILOG CPLEX） solver can find the exact solution in the small-scale example set， and ACO algorithm can save the operation time by 99.6% . In the large-scale example set， compared with ACO algorithm， HACO algorithm combing the four optimization operators has the average optimization efficiency increased by 4.45%. The proposed algorithm can obtain a feasible solution for REVRP in a limited time.

Multi-objective routing optimization of electric power material distribution based on deep reinforcement learning

Yu XU, Yunyou ZHU, Xiao LIU, Yuting DENG, Yong LIAO

2022, 42(10):  3252-3258.  DOI: 10.11772/j.issn.1001-9081.2021091582

Asbtract ( )   HTML ( )   PDF (1802KB) ( )  

Figures and Tables | References | Related Articles | Metrics

In the existing optimization of Electric power material Vehicle Routing Problem （EVRP）， the objective function is relatively single， the constraints are not comprehensive enough， and the traditional solution algorithms are not efficient. Therefore， a multi-objective routing optimization model and solution algorithm for electric power material distribution based on Deep Reinforcement Learning （DRL） was proposed. Firstly， the electric power material distribution area constraints such as the distribution of gas stations and the fuel consumption of material transportation vehicles were fully considered to establish a multi-objective power material distribution model with the objectives of the minimum total length of the power material distribution routings， the lowest cost， and the highest material demand point satisfaction. Secondly， a power material distribution routing optimization algorithm DRL-EVRP was designed on the basis of Deep Reinforcement Learning （DRL） to solve the proposed model. In the algorithm， the improved Pointer Network （Ptr-Net） and the Q-learning algorithm were combined to form the Deep Q-Network （DQN）， which was used to take the sum of the negative value of the cumulative incremental routing length and the satisfaction as the reward function. After DRL-EVRP algorithm was trained and learned， it can be directly used for the planning of electric power material distribution routings. Simulation results show that the total length of the power material distribution routing solved by DRL-EVRP algorithm is shorter than those solved by the Extended Clarke and Wright （ECW） saving algorithm and Simulated Annealing （SA） algorithm， and the calculation time of the proposed algorithm is within an acceptable range. Therefore， the power material distribution routing can be optimized more efficiently and quickly by the proposed algorithm.

Optimization of automated stacking crane operation based on NSGA Ⅱ with dynamic rules in mixed stacking mode

Yinping GAO, Daofang CHANG, Chun‑Hsien CHEN

2022, 42(10):  3259-3267.  DOI: 10.11772/j.issn.1001-9081.2021081456

Asbtract ( )   HTML ( )   PDF (2428KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Aiming at the uncertain arrival time of external container trucks， the dynamic operation sequence of Automated Stacking Crane （ASC） was optimized to improve the operation efficiency of automated container terminal yard with objective of reducing the completion time of ASCs as well as the waiting time of ASCs and external container trucks. Firstly， combining the characteristics of container operation types and dynamic arrival of external container trucks in mixed stacking mode， a strategy of ASCs dynamically matching the operation tasks of external container trucks were proposed. Then， a multi-objective model with the shortest operating time of ASCs as well as the shortest waiting time of ASCs and external container trucks was constructed. Finally， a Non-dominated Sorting Genetic Algorithm Ⅱ based on Dynamic Rules （DRNSGA Ⅱ） was designed as the solving algorithm. In small-scale example experiments， DRNSGA Ⅱ and Genetic Algorithm （GA） were used to solve ASC operation problems under dynamic strategy and random strategy， respectively. The experimental results show that the target function value solved by DRNSGA Ⅱ under dynamic strategy is 28.2% better than that under random strategy， and the result solved by DRNSGA Ⅱ is 23.3% better than that solved by Genetic Algorithm （GA） when using dynamic strategy. The performance of DRNSGA Ⅱ and Multi-Objective Particle Swarm Optimization （MOPSO） algorithm were compared in large-scale experiments. The experimental results show that the result solved by DRNSGA Ⅱ is 6.7% better than that solved by MOPSO algorithm. It can be seen that DRNSGA Ⅱ can quickly generate a variety of non-dominated solutions to provide decision support for ASC dynamic operation in mixed stacking mode.

Simultaneous localization and mapping for mobile robots based on WiFi fingerprint sequence matching

Zhenghong QIN, Ran LIU, Yufeng XIAO, Kaixiang CHEN, Zhongyuan DENG, Tianrui DENG

2022, 42(10):  3268-3274.  DOI: 10.11772/j.issn.1001-9081.2021081522

Asbtract ( )   HTML ( )   PDF (2498KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Simultaneous Localization And Mapping （SLAM） is a research hotspot in robot localization and navigation. Reliable loop closure detection is critical for graph-based SLAM. However， loop closure detection by vision or Lidar is computationally expensive and has low reliability in large and complex environments. To solve this problem， a graph-based SLAM algorithm based on WiFi fingerprint sequence matching was proposed. In this algorithm， fingerprint sequences were used for loop closure detection. Since the fingerprint sequence contains data of multiple fingerprints， which is considered to be richer than a single fingerprint pair in the amount of information. Therefore， the traditional method based on single fingerprint pair matching was extended to fingerprint sequence matching， which greatly reduced the probability of false loop closure， thus ensuring the high accuracy of loop closure detection and satisfying high precision requirement of SLAM algorithm in large and complex environments. Two sets of experimental data （robots start from different starting points） were used to verify the proposed algorithm. The results show that the proposed algorithm is more accurate than Gaussian similarity method， and has the accuracy on the first and second set of data increased by 22.94% and 39.18% respectively. Experimental results fully verify the superiority of the proposed algorithm in improving the positioning accuracy and ensuring the reliability of loop closure detection

Flight delay prediction model based on Conv-LSTM with spatiotemporal sequence

Jingyi QU, Liu YANG, Xuyang CHEN, Qian WANG

2022, 42(10):  3275-3282.  DOI: 10.11772/j.issn.1001-9081.2021091613

Asbtract ( )   HTML ( )   PDF (3421KB) ( )  

Figures and Tables | References | Related Articles | Metrics

The accurate flight delay prediction results can provide a great reference value for the prevention of large-scale flight delays. The flight delays prediction is a time-series prediction in a specific space， however most of the existing prediction methods are the combination of two or more algorithms， and there is a problem of fusion between algorithms. In order to solve the problem above， a Convolutional Long Short-Term Memory （Conv-LSTM） network flight delay prediction model was proposed that considers the temporal and spatial sequences comprehensively. In this model， on the basis that the temporal features were extracted by Long Short-Term Memory （LSTM） network， the input of the network and the weight matrix were convolved to extract spatial features， thereby making full use of the temporal and spatial information contained in the dataset. Experimental results show that the accuracy of the Conv-LSTM model is improved by 0.65 percentage points compared with LSTM， and it is 2.36 percentage points higher than that of the Convolutional Neural Network （CNN） model that only considers spatial information. It can be seen that with considering the temporal and spatial characteristics at the same time， more accurate prediction results can be obtained in the flight delay problem. In addition， based on the proposed model， a flight delay analysis system based on Browser/Server （B/S） architecture was designed and implemented， which can be applied to the air traffic administration flow control center.

Departure flight delay prediction model based on deep fully connected neural network

Haiwen XU, Jiacai SHI, Teng WANG

2022, 42(10):  3283-3291.  DOI: 10.11772/j.issn.1001-9081.2022010002

Asbtract ( )   HTML ( )   PDF (3314KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Aiming at the problem that it is difficult to improve the accuracy of departure flight delay prediction， a departure flight delay prediction model based on Deep Fully Connected Neural Network （DFCNN） was proposed. Firstly， on the basis of considering flight information， airport weather and flight delay history， the influence of flight network structure on prediction model was considered. Secondly， experiments were carried out from three dimensions of activation function， input data item and delay time threshold to optimize and verify the model ability to suppress gradient dispersion and improve the learning performance. Finally， through adjusting the vertical expansion method of the number of neural network layers and the Dropout parameters of the random loss layers， the generalization ability of the model was improved. The results of experiments indicate that the prediction accuracy of the proposed model can be improved by 1.26 percentage points and 1.28 percentage points respectively after using tanh and Exponential Linear Unit （ELU） functions in the proposed model than using Rectified Linear Unit （ReLU）. After considering the flight network structure， the prediction accuracy calculated by the proposed model using ELU function is improved by 3.12 percentage points than without considering the flight network structure. When the Dropout parameters are adjusted， the loss value of the model is continuously reduced with 60 min time threshold. With a 5-layer hidden layer network and a Dropout parameter of 0.3， the prediction accuracy of 92.39% can be achieved by the proposed model. Therefore， the proposed model can make more accurate judgments on domestic flight delays.

Short-term trajectory prediction model of aircraft based on attention mechanism and generative adversarial network

Yuli CHEN, Qiang TONG, Tongtong CHEN, Shoulu HOU, Xiulei LIU

2022, 42(10):  3292-3299.  DOI: 10.11772/j.issn.1001-9081.2021081387

Asbtract ( )   HTML ( )   PDF (1549KB) ( )  

Figures and Tables | References | Related Articles | Metrics

Single Long Short-Term Memory （LSTM） network cannot effectively extract key information and cannot accurately fit data distribution in trajectory prediction. In order to solve the problems， a short-term trajectory prediction model of aircraft based on attention mechanism and Generative Adversarial Network （GAN） was proposed. Firstly， different weights were assigned to the trajectory by introducing attention mechanism， so that the influence of important features in the trajectory was able to be improved. Secondly， the trajectory sequence features were extracted by using LSTM， and the convergence net was used to gather all aircraft features within the time step. Finally， the characteristic of GAN optimizing continuously in adversarial game was used to optimize the model in order to improve the model accuracy. Compared with Social Generative Adversarial Network （SGAN）， the proposed model has the Average Displacement Error （ADE）， Final Displacement Error （FDE） and Maximum Displacement Error （MDE） reduced by 20.0%， 20.4% and 18.3% respectively on the dataset during climb phase. Experimental results show that the proposed model can predict future trajectories more accurately.

Non-intrusive load identification algorithm based on convolutional neural network with upsampling pyramid structure

Yu DU, Meng YAN, Xin WU

2022, 42(10):  3300-3306.  DOI: 10.11772/j.issn.1001-9081.2021081512

Asbtract ( )   HTML ( )   PDF (3366KB) ( )   PDF（mobile） (807KB) ( 10 )  

Figures and Tables | References | Related Articles | Metrics

Non-Intrusive Load Monitoring （NILM） technology provides technical support for demand side management， and non-intrusive load identification is the key link in the process of load monitoring. The long-term sampling process of load data cannot be carried out in real time and high frequency， and the time sequence of the obtained load data is lost. At the same time， the defect of insufficient representation of low-level signal features occurs in Convolution Neural Network （CNN）. In view of the above two problems， a CNN based non-intrusive load identification algorithm with upsampling pyramid structure was proposed. In the proposed algorithm， with direct orientation to the collected load current signals， the time sequence of the data was compensated by the relevant information in the time dimension of the upsampling network expanded data， and the high-level and low-level features of load signals were extracted by the bidirectional pyramid one-dimensional convolution， so that the load characteristics were fully utilized. As a result， the purpose of identifying unknown load signals can be achieved. Experimental results show that the recognition accuracy of non-intrusive load identification algorithm based on CNN with upsampling pyramid structure can reach 95.21%， indicating that the proposed algorithm has a good generalization ability， and can effectively realize load identification.