Visual representation model and automatic keywords extraction algorithm for hub Web pages

doi:10.3724/SP.J.1087.2012.02360

Journal of Computer Applications ›› 2012, Vol. 32 ›› Issue (08): 2360-2368.DOI: 10.3724/SP.J.1087.2012.02360

• Typical applications • Previous Articles Next Articles

Visual representation model and automatic keywords extraction algorithm for hub Web pages

1. College of Computer Science and Technology, Hunan International Economics University, Changsha Hunan 410205，China
2. School of Information Science and Engineering, Central South University, Changsha Hunan 410083, China
3.

Received:2011-11-11 Revised:2012-01-31 Online:2012-08-28 Published:2012-08-01
Contact: PENG Hao

面向导航型网页关键词自动抽取的视觉模型与算法

彭浩¹,蔡美玲¹,²,陈继锋¹,刘炽³,余炳锐¹

1. 湖南涉外经济学院计算机科学与技术学院，长沙 410205
2. 中南大学信息科学与工程学院，长沙 410083
3. 中国电力出版社用电技术出版中心，北京 100005

通讯作者: 彭浩
作者简介:彭浩(1978-)，男，湖南长沙人，讲师，硕士，主要研究方向：Web信息获取与处理、实时调度;
蔡美玲(1982-)，女，湖南长沙人，讲师，博士研究生，主要研究方向：网络计算、智能信息处理、图形图像信息处理;
陈继锋(1966-)，男，湖南浏阳人，教授，博士，主要研究方向：软件测试自动化;
刘炽(1979-),男,湖南沅陵人,工程师,硕士研究生,主要研究方向:软件工程、嵌入式系统;
余炳锐(1988-)，男，湖南怀化人，主要研究方向：Web信息获取与处理。
基金资助:
国家自然科学基金资助项目(60803024);湖南省自然科学基金资助项目(10JJ6092);湖南省大学生研究性学习和创新性实验计划项目(湘教通[2011]272号，编号：393)

Abstract

Abstract: It is very hard to exactly extract keywords from hub Web pages because of its topic noise. To resolve this problem, a new sub Web page representation model and its automatic keywords extraction algorithm were proposed in this paper. At first, the new model segmented Web page into some blocks by using the block composition algorithm. Secondly, according to the visual recognition method of humanity, the new model computed the visual measurement of these blocks. At the same time, the transmission rule of visual measurement made blocks special where keywords were contained more specially. The automatic keywords extraction algorithm could exactly find these keywords in the most special blocks. The experimental results show that the proposed algorithm has bumped up by 20.9% on average in accuracy compared with keywords extraction algorithm based on DocView model.

Key words: block composition, visual characteristic measurement, Web page representation model, automatic keywords extraction

摘要： 导航型网页中往往包含了大量的噪声信息，为自动提取网页中的关键词带来了较大的困难。为此，提出一个新的网页表示模型PIX-PAGE和导航型网页关键词自动抽取算法P-KEA。PIX-PAGE模型利用提出的区域合并算法，将一张网页分割为适当粒度的区域；然后，依据人类视觉特点，对各区域进行视觉“奇异性”量化，同时利用奇异性传递规则进一步强化关键词相关区域的视觉“奇异性”。P-KEA根据PIX-PAGE模型模型的视觉量化结果，能够较准确地找到视觉突出区域中的关键词。实验结果表明，与基于DocView模型的算法DVM相比，P-KEA的准确率平均提高了20.9%。

关键词: 区域合并, 视觉量化, 网页表示模型, 关键词自动抽取

CLC Number:

TP391.4

彭浩蔡美玲陈继锋刘炽余炳锐. 面向导航型网页关键词自动抽取的视觉模型与算法[J]. 计算机应用, 2012, 32(08): 2360-2368.

References

[1]CHAKRABARTI S, van den BERG M, DOM B. Focused crawling: a new approach to topic-specific Web resource discovery [J].Computer Networks, 1999,31(11-16): 1623-1640. [2]CHAU M, CHEN H. Incorporating Web analysis into neural networks: an example in Hopfield net searching [J].IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2007, 37(3):352-358. [3]周立柱，林玲.聚焦爬虫技术研究综述[J].JOCA，2005，25(9):1965-1969. [4]WU XIAOYUAN, BOLIVAR A. Keyword extraction for contextual advertisement [C]// WWW'08: Proceedings of the 17th International Conference on World Wide Web. New York: ACM, 2008:1195-1196. [5]刘远超，王晓龙，刘秉权，等．信息检索中的聚类分析技术[J].电子与信息学报，2006，28(4): 606-609. [6]陈竹敏.面向垂直搜索引擎的主题爬行技术研究[D]. 济南，山东大学，2008. [7]李晓明，闫宏飞，王继民.搜索引擎——原理、技术与系统[M].北京：科学出版社，2006:98-103. [8]韩客松，王永成，滕伟. Web页面中文文本主题的自动提取研究[J].情报学报，2001，20(2):217-222. [9]任克强，赵光甫，张国萍.基于带权语言网络的网页关键词抽取[J].计算机工程与应用，2008，44(8):155-157. [10]BUYUKKOKTEN O, GARCM-MOLINA H，PAEPCKE A. Seeing the whole in parts: text summarization for Web browsing on handheld devises [C]// WWW'01: Proceedings of the 10th International Conference on World Wide Web. New York: ACM, 2001：652-662. [11]王琦，唐世渭，杨冬青，等．基于DOM的网页主题信息自动提取[J].计算机研究与发展，2004，41(10)：1786-1791. [12]CAI DENG, YU SHIPENG, WEN JI-RONG, et al. VIPS: a vision-based page segmentation algorithm, MSR-TR-2003-79 [R]. Redmond: Microsoft Research Corporation, 2003.

[1]	. Pavement defect detection algorithm with enhanced morphological perception [J]. Journal of Computer Applications, 0, (): 0-0.
[2]	YANG Jianfeng, CHEN Bin, LI Yuxuan. Self-supervised point cloud anomaly detection method based on point cloud reconstruction [J]. Journal of Computer Applications, 0, (): 0-0.
[3]	. Binocular vision object location algorithm for robot arm grasping [J]. Journal of Computer Applications, 0, (): 0-0.
[4]	. Multi-attention contrastive learning for infrared small target detection [J]. Journal of Computer Applications, 0, (): 0-0.
[5]	LI Zhonghua, ZHONG Gengxin, FAN Ping, ZHU Hengliang. Camouflaged object detection by boundary mining and background guidance [J]. Journal of Computer Applications, 0, (): 0-0.
[6]	. Multi-view difficult airway recognition based on discriminant region guidance [J]. Journal of Computer Applications, 0, (): 0-0.
[7]	. Privacy preserving localization of surveillance images based on large vision models [J]. Journal of Computer Applications, 0, (): 0-0.
[8]	. Data augmentation method for abnormal passenger behavior in elevators based on dynamic graph convolutional network [J]. Journal of Computer Applications, 0, (): 0-0.
[9]	. Text-based person retrieval method based on multi-granularity shared semantic center association [J]. Journal of Computer Applications, 0, (): 0-0.
[10]	. Distribution adaptation and dynamic curriculum pseudo-labeling framework for semi-supervised fire detection [J]. Journal of Computer Applications, 0, (): 0-0.
[11]	. Image Caption Method Based on Swin-Transformer and Multi-Scale Feature Fusion [J]. Journal of Computer Applications, 0, (): 0-0.
[12]	Wei XIONG, Yibo CHEN, Lizhen ZHANG, Qian YANG, Qin ZOU. Self-supervised monocular depth estimation using multi-frame sequence images [J]. Journal of Computer Applications, 2024, 44(12): 3907-3914.
[13]	Keyi FU, Gaocai WANG, Man WU. Few-shot object detection method based on improved region proposal network and feature aggregation [J]. Journal of Computer Applications, 2024, 44(12): 3790-3797.
[14]	Chenghao YANG, Jie HU, Hongjun WANG, Bo PENG. Incomplete multi-view clustering algorithm based on attention mechanism [J]. Journal of Computer Applications, 2024, 44(12): 3784-3789.
[15]	Peng FANG, Fan ZHAO, Baoquan WANG, Yi WANG, Tonghai JIANG. Development， technologies and applications of blockchain 3.0 [J]. Journal of Computer Applications, 2024, 44(12): 3647-3657.

Visual representation model and automatic keywords extraction algorithm for hub Web pages

面向导航型网页关键词自动抽取的视觉模型与算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics