Abstract: Public information in the standardization domain is characterized by dispersed sources, multiple languages, heterogeneous formats, and frequent duplicate reports, which poses challenges to resource cleaning, text mining, and knowledge organization. To address the problems of insufficient resource relevance, incomplete information, and difficult semantic parsing, this paper proposesd a preprocessing and concept extraction method for multi-source heterogeneous texts in the standardization domain. First, an Extract-Transform-Load (ETL) ETL-based workflow and a unified schema are were used to collect data from multiple sources and represent it in a structured formatmulti-source resources and represent them in a structured form. Second, a FastText-based relevance filtering model is was constructed, and machine translation, Optical Character Recognition (OCR)OCR, and attachment parsing are were integrated to standardize multilingual and multimodal resources into analyzable text. Third, event-level multi-source fusion is was performed by combining a time-window constraint, with title-level Jaccard similarity, and body-level cosine similarity. Finally, a seven-category concept system, including organization, person, location, field, document, event, and time, is was designed, and concept extraction is was implemented by combining domain lexicon expansion with a continually pretrained WoBERT model. Experimental results show that FastText achieves an F1-score of 0.957 on the resource relevance filtering task, outperforming the rule- and keyword-based method by 16.3 percentage points. Experimental results show that FastText achieves 0.957 in Precision, Recall, and F1-score on the resource filtering task. The domain-adapted WoBERT model obtains an F1-score of 0.933 on the concept extraction task, outperforming baselines such as HMM, CRF, BiLSTM-CRF, BERT-CRF, and BART. Ablation experiments and sensitivity analyses further verify the effectiveness of domain lexicon expansion, event-level fusion, and multimodal text quality control. The proposed method improves the relevance, completeness, and parsability of public information in the standardization domain, providing data support for downstream event summarization, knowledge organization, and thematic analysis.

Key words: standardization domain text, multi-source heterogeneous text, resource preprocessing, concept extraction, FastText, WoBERT

摘要： 标准化领域公开信息具有来源分散、语种多样、载体异构和重复报道频繁等特点，给资源清洗、文本挖掘和知识组织带来挑战。针对原始资源相关性不足、信息完整性欠缺和语义解析困难等问题，提出一种面向标准化领域多源异构文本的预处理与概念抽取方法。该方法首先基于抽取—转换—加载（ETL）流程和统一数据模式实现多源资源采集与结构化表示；其次采用FastText构建资源相关性过滤模型，并结合机器翻译、光学字符识别（OCR）和附件解析完成多语种、多模态资源的文本化标准化处理；随后利用时间窗口约束、标题Jaccard相似度和正文余弦相似度实现事件级多源融合；最后构建包含机构、人物、地点、领域、文件、事件和时间的7类概念体系，并结合领域词典扩充与继续预训练的WoBERT模型完成概念抽取。实验结果表明，FastText在资源相关性过滤任务中的F1值达到0.957，较规则与关键词方法提高16.3个百分点。领域适配WoBERT在概念抽取任务中的F1值达到0.933，优于对比的基线模型。消融实验与敏感性分析进一步验证了领域词典扩充、事件级融合和多模态文本质量控制的有效性，表明本文方法能够提升标准化领域公开信息的相关性、完整性和可解析性，为后续事件归纳、知识组织和专题分析提供数据支撑。

关键词: 标准化领域文本, 多源异构文本, 资源预处理, 概念抽取, FastText, WoBERT