本体与条件随机场结合的涉农商品名称抽取与类别标注

doi:10.11772/j.issn.1001-9081.2017.01.0233

计算机应用 ›› 2017, Vol. 37 ›› Issue (1): 233-238.DOI: 10.11772/j.issn.1001-9081.2017.01.0233

本体与条件随机场结合的涉农商品名称抽取与类别标注

黄念娥^1,2, 黄河¹, 王儒敬¹

1. 中国科学院合肥智能机械研究所, 合肥 230031;
2. 中国科学技术大学合肥物质研究院, 合肥 230027

收稿日期:2016-08-02 修回日期:2016-09-19 出版日期:2017-01-10 发布日期:2017-01-09
通讯作者: 黄河
作者简介:黄念娥(1991-),女,安徽安庆人,硕士研究生,主要研究方向:信息抽取、垂直搜索引擎;黄河(1980-),男,安徽合肥人,副研究员,博士,主要研究方向:农业大数据、农业智能系统;王儒敬(1964-),男,安徽亳州人,研究员,博士,主要研究方向:知识表示与可视化、知识获取。
基金资助:
国家科技支撑计划项目（2013BAD15B03）；中国科学院重点部署项目（Y622A21291）；安徽省科技攻关项目（1401032010）。

Agriculture-related product name extraction and category labeling based on ontology and conditional random field

HUANG Nian'e^1,2, HUANG He¹, WANG Rujing¹

1. Institute of Intelligent Machines, Chinese Academy of Sciences, Hefei Anhui 230031, China;
2. Hefei Institute of Physical Science, University of Science and Technology of China, Hefei Anhui 230027, China

Received:2016-08-02 Revised:2016-09-19 Online:2017-01-10 Published:2017-01-09
Supported by:
This work is partially supported by the National Science and Technology Support Program (2013BAD15B03), Chinese Academy of Sciences Key Deployment Project (Y622A21291), the Scientific and Technological Project of Anhui Province (1401032010).

摘要/Abstract

摘要： 传统的基于条件随机场（CRF）的信息抽取方法在进行涉农商品名称抽取与类别标注时，需要大量的训练语料，标注工作量大，且抽取精度不高。为解决该问题，提出了一种基于农业本体与CRF相结合的涉农商品名称抽取与类别标注方法，将涉农商品名称的自动抽取与分类看作序列标注的任务。首先是原始数据的分词处理和词、词性、地理属性、本体概念特征选择；然后，采用改进的拟牛顿算法训练CRF模型参数，用维特比算法实现解码，共完成4组对比实验，识别出7种类别，并将CRF和隐马尔可夫模型（HMM）、最大熵马尔可夫模型（MEMM）通过实验进行比较；最后，将CRF应用于农产品供求趋势分析。结合合适的特征模板，本体概念的加入使CRF开放测试的总体准确率提高10.20%，召回率提高59.78%，F值提高37.17%，证明了本体与CRF结合方法在涉农商品名称和类别抽取中的可行性和有效性，可以促进农产品供求对接。

关键词: 条件随机场, 农业本体, 涉农商品名称, 供求趋势, 序列标注

Abstract: Traditional information extraction method based on Conditional Random Field (CRF) requires large-scale labeled corpus, it is expensive to label corpus manually and the extraction precision is low in processing agriculture-related product name extraction and category labeling. In order to solve this problem, a method of agriculture-related product name extraction and category labeling based on agricultural ontology and CRF was proposed, automatic extraction and classification of agriculture-related product names was regarded as sequence labeling. Firstly, original data was processed, word, part of speech, geographical attributes and ontology concept features were selected. Then, parameters of the CRF model were trained by the improved quasi-Newton algorithm and decoding was implemented by Viterbi algorithm. A total of four groups of comparative experiments were completed and seven categories were identified. CRF, Hidden Markov Model (HMM) and Maximum Entropy Markov Model (MEMM) were compared through experiments. Finally, the supply and demand trend analysis of agriculture produce was accomplished. The experimental results show that the overall precision, recall and F-score of the open test were increased by 10.20%, 59.78% and 37.17% respectively by adding ontology concepts with appropriate CRF features; it also proves the feasibility, effectiveness and practical significance of the method in promoting automatic supply and demand docking of agricultural products.

Key words: Conditional Random Field (CRF), agricultural ontology, agriculture-related product name, supply and demand trend, sequence labeling

中图分类号:

TP391.1
TP18

黄念娥, 黄河, 王儒敬. 本体与条件随机场结合的涉农商品名称抽取与类别标注[J]. 计算机应用, 2017, 37(1): 233-238.

HUANG Nian'e, HUANG He, WANG Rujing. Agriculture-related product name extraction and category labeling based on ontology and conditional random field[J]. Journal of Computer Applications, 2017, 37(1): 233-238.

参考文献

[1] 于连军.基于互联网+的农业电子商务发展模式的研究[J].农业网络信息,2015(11):19-21.(YU L J. Research on the development model of agricultural E-commerce based on Internet+[J]. Agriculture Network Information, 2015(11):19-21.)
[2] LI L S, DAND Y Z, ZHANG J, et al. Domain term extraction based on conditional random fields combined with active learning strategy[J]. Journal of Information & Computational Science, 2012, 9(7):1931-1940.
[3] GUAN A Q, WANG Y B, YANG L F. Automatic term extraction for Chinese opera domain ontology[C]//Proceedings of the 12th International Conference on Fuzzy Systems and Knowledge Discovery. Piscataway, NJ:IEEE, 2015:1372-1376.
[4] 宗成庆.统计自然语言处理[M].2版.北京:清华大学出版社,2013:110-128.(ZONG C Q. Statistical Natural Language Processing[M]. 2nd ed. Beijing:Tsinghua University Press, 2013:110-128.)
[5] WALLACH H M. Conditional random fields:an introduction, technical report MS-CIS-04-21[R]. Philadelphia, PA:University of Pennsylvania, 2004:262-272.
[6] FU W J, LI L. A method and application of automatic term extraction using conditional random fields[C]//Proceedings of the 2009 International Conference on Natural Language Processing and Knowledge Engineering. Piscataway, NJ:IEEE, 2009:1-5.
[7] ZHANG C Z, WANG H L, LIU Y, et al. Automatic keyword extraction from documents using conditional random fields[J]. Journal of Computational Information System, 2008, 4(3):1169-1180.
[8] 孟洪宇.基于条件随机场的[WW)]《[WW(]伤寒论[WW)]》[WW(]中医术语自动识别[D].北京:北京中医药大学,2014:41-48.(MENG H Y. Automatic identification of TCM terminology in Shanghan Lun based on conditional random field[D]. Beijing:Beijing University of Chinese Medicine, 2014:41-48.)
[9] ZHAN Q, WANG C H. A Hybrid strategy for Chinese domain-specific terminology extraction[C]//Proceedings of the 11th International Conference on Semantics, Knowledge and Grids. Piscataway, NJ:IEEE, 2015:217-221.
[10] 王春雨.基于CRF的农业命名实体识别研究[D].保定:河北农业大学,2014:19-23.(WANG C Y. Study on recognition of Chinese agricultural named entity with CRF[D]. Baoding:Agricultural University of Hebei, 2014:19-23.)
[11] CAO Y S, WANG J, LI L. Word-level information extraction from science and technology announcements corpus based on CRF[C]//Proceedings of the 2nd IEEE International Conference on Cloud Computing and Intelligence Systems. Piscataway, NJ:IEEE, 2012:1529-1533.
[12] IZUMI M, MIURA T, SHIOYA I. Estimating the date of blog authors by CRF[C]//Proceedings of the 2007 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing. Piscataway, NJ:IEEE, 2007:249-252.
[13] GRUBER T R. A translation approach to portable ontology specifications[J]. Knowledge Acquisition, 1993, 5(2):199-220.
[14] 李传席.基于本体的自适应Web信息抽取方法研究[D].合肥:中国科学技术大学,2012:15-17.(LI C X. Adaptive Web information extraction method research based on ontology[D]. Hefei:University of Science and Technology of China, 2012:15-17.)
[15] LIU X G, DUAN X H, ZHANG H Y. Application of ontology in classification of agricultural information[C]//Proceedings of the 2012 IEEE Symposium on Robotics and Applications. Piscataway, NJ:IEEE, 2012:451-454.
[16] 周晶,吴军华,陈佳,等.基于条件随机域CRF模型的文本信息抽取[J].计算机工程与设计,2008,29(23):6094-6097.(ZHOU J, WU J H, CHEN J, et al. Using conditional random fields model for text information extraction[J]. Computer Engineering and Design, 2008, 29(23):6094-6097.)
[17] LAFFERTY J, MCCALLUM A, PEREIRA F. Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning. San Francisco, CA:Morgan Kaufmann, 2001:282-289.
[18] Sunfox66.条件随机场详解[EB/OL]. (2015-10-25)[2016-01-17] . http://wenku.baidu.com/view/bbd57f82fc4ffe473268ab59.html.(Sunfox66. Conditional random field introduction[EB/OL]. (2015-10-25)[2016-01-17] . http://wenku.baidu.com/view/bbd57f82fc4ffe473268ab59.html.)
[19] LIU D, NOCEDAL J. On the limited memory BFGS method for large scale optimization[J]. Mathematical Programming, 1989, 45(45):503-528.
[20] TAKU K. CRF++ toolkit[EB/OL]. (2014-10-15)[2016-01-15] . http://download.csdn.net/detail/linson3344/8039087.
[21] HANKCS. Han language processing[EB/OL]. (2015-03-27)[2016-01-28] . http://www.hankcs.com/nlp/hanlp.html.
[22] 施水才,王锴,韩艳铧,等.基于条件随机场的领域术语识别研究[J].计算机工程与应用,2013,49(10):147-149.(SHI S C, WANG K, HAN Y H, et al. Terminology recognition based on conditional random fields[J]. Computer Engineering and Applications, 2013, 49(10):147-149.)
[23] 贾美英,杨炳儒,郑德权,等.采用CRF技术的军事情报术语自动抽取研究[J].计算机工程与应用,2009,45(32):126-129.(JIA M Y, YANG B R, ZHENG D Q, et al. Research on automatic military intelligence term extraction using CRF model[J]. Computer Engineering and Applications, 2009, 45(32):126-129.)
[24] MCCALLUM A K. MALLET:a machine learning for language toolkit[EB/OL]. (2002-02-28)[2016-02-25] . http://mallet.cs.umass.edu.

本体与条件随机场结合的涉农商品名称抽取与类别标注

Agriculture-related product name extraction and category labeling based on ontology and conditional random field

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	李灿, 杨雅婷, 马玉鹏, 董瑞. 基于语种相似性挖掘的神经机器翻译语料库扩充方法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3145-3150.
[2]	吴赛赛, 梁晓贺, 谢能付, 周爱莲, 郝心宁. 面向领域实体关系联合抽取的标注方法[J]. 计算机应用, 2021, 41(10): 2858-2863.
[3]	胡甜甜, 但雅波, 胡杰, 李想, 李少波. 基于注意力机制的Bi-LSTM结合CRF的新闻命名实体识别及其情感分类[J]. 计算机应用, 2020, 40(7): 1879-1883.
[4]	陈佳沣, 滕冲. 基于强化学习的实体关系联合抽取模型[J]. 计算机应用, 2019, 39(7): 1918-1924.
[5]	严红, 陈兴蜀, 王文贤, 王海舟, 殷明勇. 基于深度神经网络的法语命名实体识别模型[J]. 计算机应用, 2019, 39(5): 1288-1292.
[6]	许玥, 冯梦如, 皮家甜, 陈勇. 基于深度学习模型的遥感图像分割方法[J]. 计算机应用, 2019, 39(10): 2905-2914.
[7]	廖斌, 李浩文. 基于多孔卷积神经网络的图像深度估计模型[J]. 计算机应用, 2019, 39(1): 267-274.
[8]	张晨, 钱涛, 姬东鸿. 基于神经网络的微博情绪识别与诱因抽取联合模型[J]. 计算机应用, 2018, 38(9): 2464-2468.
[9]	吴亮, 何毅, 梅雪, 刘欢. 基于时空兴趣点和概率潜动态条件随机场模型的在线行为识别方法[J]. 计算机应用, 2018, 38(6): 1760-1764.
[10]	李雅昆, 潘晴, Everett X. WANG. 基于改进的多层BLSTM的中文分词和标点预测[J]. 计算机应用, 2018, 38(5): 1278-1282.
[11]	刘一鸣, 张鹏程, 刘祎, 桂志国. 基于全卷积网络和条件随机场的宫颈癌细胞学图像的细胞核分割[J]. 计算机应用, 2018, 38(11): 3348-3354.
[12]	周霜霜, 徐金安, 陈钰枫, 张玉洁. 融合规则与统计的微博新词发现方法[J]. 计算机应用, 2017, 37(4): 1044-1050.
[13]	刘彤, 黄修添, 马建设, 苏萍. 基于完全联系的条件随机场的图像标注[J]. 计算机应用, 2017, 37(10): 2841-2846.
[14]	王婷, 王祺, 黄越圻, 殷亦超, 高炬. 基于症状构成成分的上下位关系自动抽取方法[J]. 计算机应用, 2017, 37(10): 2999-3005.
[15]	孙丽萍, 过弋, 唐文武, 徐永斌. 基于构成模式和条件随机场的企业简称预测[J]. 计算机应用, 2016, 36(2): 449-454.