Automatic identification of Uyghur domain term in Web text

doi:10.3724/SP.J.1087.2012.00407

Journal of Computer Applications ›› 2012, Vol. 32 ›› Issue (02): 407-410.DOI: 10.3724/SP.J.1087.2012.00407

• Artificial intelligence • Previous Articles Next Articles

Automatic identification of Uyghur domain term in Web text

ZHONG Jun¹,TIAN Sheng-wei²,YU Long³

1. College of Information Science and Technology, Xinjiang University, Urumqi Xinjiang 830046,China
2. College of Software Design, Xinjiang University, Urumqi Xinjiang 830046, China
3. Network Center, Xinjiang University, Urumqi Xinjiang 830046, China

Received:2011-07-22 Revised:2011-09-22 Online:2012-02-23 Published:2012-02-01
Contact: ZHONG Jun

Web文本中维吾尔语领域术语的自动发现

钟军¹,田生伟²,禹龙³

1. 新疆大学信息科学与工程学院，乌鲁木齐 830046
2. 新疆大学软件学院，乌鲁木齐 830046
3. 新疆大学网络中心，乌鲁木齐 830046

通讯作者: 钟军
作者简介:钟军(1988-)，男，新疆乌鲁木齐人，硕士研究生，主要研究方向：人工智能；
田生伟(1973-)，男，新疆乌鲁木齐人，副教授，博士，主要研究方向：计算机智能、自然语言处理；
禹龙(1974-)，女，新疆乌鲁木齐人，副教授，硕士，主要研究方向：计算机智能、计算机网络。
基金资助:
国家自然科学基金资助项目(50774092);国家社科基金资助项目(10BTQ045，11XTQ007)

Abstract

Abstract: Since the Uyghur domain term is difficult to achieve, the workload of artificial expansion of the domain term is tremendous, and the efficiency is low, this paper used the Conditional Random Field (CRF) to identify the Uyghur domain term from the Web texts, which expanded the domain term with the conjunction word and the Mutual Information (MI) between the words based on the co-occurrence of terms. The experiments on the collected Web texts show that, for the short Uyghur domain terms, the algorithm achieves the precision as high as 97.59% and the recall 93.38%, and for the long Uyghur domain terms achieves the precision 55.72%.

Key words: Uyghur, Mutual Information (MI), Conditional Random Field (CRF), Term Frequency/Inverse Document Frequency (TF/IDF)

摘要： 针对维吾尔语领域术语获取难度大,人工扩充领域术语工作量大、效率低等特点，利用词汇共现原理，以维吾尔语连接词和互信息(MI)为工具，快速扩充原始维吾尔语领域术语;建立了以维吾尔语领域术语为特征模板，利用条件随机场(CRF)模型实现Web文本中维吾尔语领域术语的自动发现方法，并在此基础上实现长维吾尔语领域术语的自动发现。实验表明，对短维吾尔语领域术语的自动发现准确率为 97.59%，召回率为 93.38%，对长维吾尔语领域术语的自动发现正确率达到55.72%。

关键词: 维吾尔语, 互信息, 条件随机场, TF/IDF

CLC Number:

TP181

ZHONG Jun TIAN Sheng-wei YU Long. Automatic identification of Uyghur domain term in Web text[J]. Journal of Computer Applications, 2012, 32(02): 407-410.

钟军田生伟禹龙. Web文本中维吾尔语领域术语的自动发现[J]. 计算机应用, 2012, 32(02): 407-410.

References

[1]SUI ZHIFANG, CHEN YIRONG. The research on the automatic term extraction in the domain of information science and technology [C]// Proceedings of the 5th East Asia Forum of the Terminology. Haikou: China National Institute of Standardization Press, 2007: 165-169. [2]BOURIGAULT D, JACUEMIN C, L'HOMMM-C. Recent advances in computational terminology [M]. Amsterdam: John Benjamins Publishing Company, 2001: 353-370． [3]FORTUNA B, LAVRA N, VELARDI P. Advancing topic ontology learning through term extraction [C]// PRICAI 2008: Proceedings of the 10th Pacific Rim International Conference on Artificial Intelligence, LNAI 5351. Berlin: Springer-Verlag, 2008: 626-635. [4]BUITELAAR P, OLEJNIK D, SINTEK M. A protégé plug-in for ontology extraction from text based on linguistic analysis [C]// The Semantic Web Research and Applications, LNCS 3053. Berlin: Springer-Verlag, 2004: 31-44． [5]PANTEL P, LIN D.A statistical corpus-based term extractor [C]// Proceedings of 14th Biennial Conference of the Canadian Society on Computational Studies of Intelligence: Advances in Artificial Intelligence. Ottawa: [s.n.], 2001:36-44． [6]周浪，史树敏，冯冲，等.基于多策略融合的中文术语抽取方法[J].情报学报，2010,29(3):460-467. [7]杜波，田怀凤，王立，等.基于多策略的专业术语抽取器的设计[J].计算机工程，2005，31(14):159-160. [8]温春，王晓斌，石昭祥.中文领域本体学习中术语的自动抽取取[J].计算机应用研究，2009,26(7):2652-2655. [9]KAGEURA K,UMINO B. Methods of automatic term recognition: A review [J]. Terminology, 1996, 3(2): 259-289. [10]QIN LONGZHANG,QIN LU,ZHI FANGSUI.Measuring termhood in automatic terminology extraction [C]// Natural Language Processing and Knowledge Engineering.Piscataway:IEEE,2007:328-335. [11]LAFFERTY J, MCCAIINM A, PEREIRA F. Conditional random fields: Probabilistic models for segmenting and labeling sequence data [C]// ICML: International Conference on Machine Learning. San Francisco: Morgan Kaufmann Publishers, 2001: 961-965. [12]梁颖红，张文静，周德富.基于混合策略的高精度长术语自动抽取[J].中文信息学报，2009，23(6)：27-30. [13]乔亚男，齐勇，侯迪,等.一种高稳定性词汇共现模型[J].西安交通大学学报，2009, 43(6):24-27.

[1]	Yongfeng DONG, Jiaming BAI, Liqin WANG, Xu WANG. Chinese named entity recognition combining prior knowledge and glyph features [J]. Journal of Computer Applications, 2024, 44(3): 702-708.
[2]	Jin XIA, Zhengqun WANG, Shiming ZHU. Traffic flow prediction model based on time series decomposition [J]. Journal of Computer Applications, 2023, 43(4): 1129-1135.
[3]	Qingtang LIU, Xinqian MA, Jie ZHOU, Linjing WU, Pengxiao ZHOU. Understanding of math word problems integrating commonsense knowledge base and grammatical features [J]. Journal of Computer Applications, 2023, 43(2): 356-364.
[4]	Dongchang ZHAI, Hongmei CHEN. Hyperspectral band selection algorithm based on neighborhood entropy [J]. Journal of Computer Applications, 2022, 42(2): 485-492.
[5]	ZHU Xiangrong, WANG Lei, YANG Yating, DONG Rui, ZHANG Jun. Non-autoregressive method for Uyghur-Chinese neural machine translation [J]. Journal of Computer Applications, 2020, 40(7): 1891-1895.
[6]	HU Tiantian, DAN Yabo, HU Jie, LI Xiang, LI Shaobo. News named entity recognition and sentiment classification based on attention-based bi-directional long short-term memory neural network and conditional random field [J]. Journal of Computer Applications, 2020, 40(7): 1879-1883.
[7]	WEI Jiawang, WANG Xiao, YUAN Yubo. Adaptive window regression method for face feature point positioning [J]. Journal of Computer Applications, 2019, 39(5): 1459-1465.
[8]	MAO Yingchi, CAO Hai, PING Ping, LI Xiaofang. Feature selection based on maximum conditional and joint mutual information [J]. Journal of Computer Applications, 2019, 39(3): 734-741.
[9]	LIAO Bin, LI Haowen. Image depth estimation model based on atrous convolutional neural network [J]. Journal of Computer Applications, 2019, 39(1): 267-274.
[10]	TANG Xiaochuan, QIU Xiwei, LUO Liang. Interaction based algorithm for feature selection in text categorization [J]. Journal of Computer Applications, 2018, 38(7): 1857-1861.
[11]	PAN Yirong, LI Xiao, YANG Yating, MI Chenggang, DONG Rui. Reordering table reconstruction model for Chinese-Uyghur machine translation [J]. Journal of Computer Applications, 2018, 38(5): 1283-1288.
[12]	ZHOU Shuangshuang, XU Jin'an, CHEN Yufeng, ZHANG Yujie. New words detection method for microblog text based on integrating of rules and statistics [J]. Journal of Computer Applications, 2017, 37(4): 1044-1050.
[13]	WANG Ting, WANG Qi, HUANG Yueqi, YIN Yichao, GAO Ju. Automatic hyponymy extracting method based on symptom components [J]. Journal of Computer Applications, 2017, 37(10): 2999-3005.
[14]	LIU Tong, HUANG Xiutian, MA Jianshe, SU Ping. Image labeling based on fully-connected conditional random field [J]. Journal of Computer Applications, 2017, 37(10): 2841-2846.
[15]	HUANG Nian'e, HUANG He, WANG Rujing. Agriculture-related product name extraction and category labeling based on ontology and conditional random field [J]. Journal of Computer Applications, 2017, 37(1): 233-238.

Automatic identification of Uyghur domain term in Web text

Web文本中维吾尔语领域术语的自动发现

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics