Naive Bayesian text classification algorithm in cloud computing environment

doi:10.3724/SP.J.1087.2011.02551

Journal of Computer Applications ›› 2011, Vol. 31 ›› Issue (09): 2551-2554.DOI: 10.3724/SP.J.1087.2011.02551

• Typical applications • Previous Articles Next Articles

Naive Bayesian text classification algorithm in cloud computing environment

JIANG Xiao-ping¹,LI Cheng-hua¹,XIANG Wen²,ZHANG Xin-fang²

1. College of Electronics and Information Engineering, South-central University for Nationalities, Wuhan Hubei 430074, China
2. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan Hubei 430074, China

Received:2011-03-24 Revised:2011-05-21 Online:2011-09-01 Published:2011-09-01
Contact: LI Cheng-hua
Supported by:
CZY11002;201110821229;2011ZX03002-001-01

云计算环境下朴素贝叶斯文本分类算法的实现

江小平¹,李成华¹,向文²,张新访²

1. 中南民族大学电子信息工程学院，武汉 430074
2. 华中科技大学计算机科学与技术学院，武汉 430074

通讯作者: 李成华
作者简介:江小平(1974-)，男，湖北蕲春人，副教授，博士,主要研究方向：云计算、项目管理信息化；
李成华(1972-)，男，湖北仙桃人，副教授，博士，CCF会员，主要研究方向：云计算、项目管理信息化、信息安全；
向文(1965-)，男，江西景德镇人，教授，博士，主要研究方向：云计算、信息安全；
张新访(1965-)，男，广东五华人，教授，博士生导师，博士，主要研究方向：云计算、信息安全。
基金资助:
中央高校基本科研业务费专项资金资助项目(CZY11002);武汉市科技攻关项目(201110821229);工信部国家科技重大专项(2011ZX03002-001-01)

Abstract

Abstract: The major procedures of text classification such as uniform text format expression, training, testing and classifying based on Naive Bayesian text classification algorithm were implemented using MapReduce programming mode. The experiments were given in Hadoop cloud computing environment. The experimental results indicate basically linear speedup with an increasing number of node computers. A recall rate of 86% was achieved when classifying Chinese Web pages.

Key words: cloud computing, parallel computing, MapReduce programming mode, text classification, Naive Bayes algorithm

摘要： 采用分布式编程MapReduce模型研究了文本统一格式预处理、训练、测试以及分类等基于朴素贝叶斯文本分类算法主要计算过程的MapReduce并行化方法，并在Hadoop云计算平台进行了实验。实验结果表明：朴素贝叶斯文本分类算法MapReduce并行化后在Hadoop云计算平台上部署运行，具有较好的加速比，对中文网页文本分类识别率达到了86%。

关键词: 云计算, 并行计算, MapReduce编程模型, 文本分类, 朴素贝叶斯算法

CLC Number:

JIANG Xiao-ping LI Cheng-hua XIANG Wen ZHANG Xin-fang. Naive Bayesian text classification algorithm in cloud computing environment[J]. Journal of Computer Applications, 2011, 31(09): 2551-2554.

江小平李成华向文张新访. 云计算环境下朴素贝叶斯文本分类算法的实现[J]. 计算机应用, 2011, 31(09): 2551-2554.

[1]	Runlian ZHANG, Mi ZHANG, Xiaonian WU, Rui SHU. Differential property evaluation method based on GPU for large-state cryptographic S-boxes [J]. Journal of Computer Applications, 2024, 44(9): 2785-2790.
[2]	Xinyan YU, Cheng ZENG, Qian WANG, Peng HE, Xiaoyu DING. Few-shot news topic classification method based on knowledge enhancement and prompt learning [J]. Journal of Computer Applications, 2024, 44(6): 1767-1774.
[3]	Xun YAO, Zhongzheng QIN, Jie YANG. Generative label adversarial text classification model [J]. Journal of Computer Applications, 2024, 44(6): 1781-1785.
[4]	Hang YU, Yanling ZHOU, Mengxin ZHAI, Han LIU. Text classification based on pre-training model and label fusion [J]. Journal of Computer Applications, 2024, 44(3): 709-714.
[5]	Jiawei ZHANG, Guandong GAO, Ke XIAO, Shengzun SONG. Violent crime hierarchy algorithm by joint modeling of improved hierarchical attention network and TextCNN [J]. Journal of Computer Applications, 2024, 44(2): 403-410.
[6]	Kaitian WANG, Qing YE, Chunlei CHENG. Classification method for traditional Chinese medicine electronic medical records based on heterogeneous graph representation [J]. Journal of Computer Applications, 2024, 44(2): 411-417.
[7]	Bihui YU, Xingye CAI, Jingxuan WEI. Few-shot text classification method based on prompt learning [J]. Journal of Computer Applications, 2023, 43(9): 2735-2740.
[8]	Yumeng CUI, Jingya WANG, Xiaowen LIU, Shangyi YAN, Zhizhong TAO. General text classification model combining attention and cropping mechanism [J]. Journal of Computer Applications, 2023, 43(8): 2396-2405.
[9]	Jiaxing LU, Hua DAI, Yuanlong LIU, Qian ZHOU, Geng YANG. Dictionary partition vector space model for ciphertext ranked search in cloud environment [J]. Journal of Computer Applications, 2023, 43(7): 1994-2000.
[10]	Senqi YANG, Xuliang DUAN, Zhan XIAO, Songsong LANG, Zhiyong LI. Text classification of agricultural news based on ERNIE+DPCNN+BiGRU [J]. Journal of Computer Applications, 2023, 43(5): 1461-1466.
[11]	Xu ZHANG, Long SHENG, Haifang ZHANG, Feng TIAN, Wei WANG. Pre-hospital emergency text classification model based on label confusion [J]. Journal of Computer Applications, 2023, 43(4): 1050-1055.
[12]	Chengyu LIN, Lei WANG, Cong XUE. Weakly-supervised text classification with label semantic enhancement [J]. Journal of Computer Applications, 2023, 43(2): 335-342.
[13]	Kun YOU, Qinhui WANG, Xin LI. General multi-unit false-name-proof auction mechanism for cloud computing [J]. Journal of Computer Applications, 2023, 43(11): 3351-3357.
[14]	Qian LIU, Yangming ZHANG, Dingsheng WAN. Parallel computing algorithm of grid-based distributed Xin’anjiang hydrological model [J]. Journal of Computer Applications, 2023, 43(11): 3327-3333.
[15]	JIANG Songyan, LIAO Xiaojuan, CHEN Guangzhu. Optimal task scheduling method based on satisfiability modulo theory for multiple processors with communication delay [J]. Journal of Computer Applications, 2023, 43(1): 185-191.

Naive Bayesian text classification algorithm in cloud computing environment

云计算环境下朴素贝叶斯文本分类算法的实现

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics