融入领域知识的跨境民族文化生成式摘要方法

doi:10.13232/j.cnki.jnju.2023.04.009

南京大学学报(自然科学版) ›› 2023, Vol. 59 ›› Issue (4): 620–628.doi: 10.13232/j.cnki.jnju.2023.04.009

融入领域知识的跨境民族文化生成式摘要方法

赵冠博¹^,²^,³, 张勇丙¹^,²^,³(), 毛存礼¹^,²^,³, 高盛祥¹^,²^,³, 王奉孝¹^,²^,³

^1.南亚东南亚语言语音信息处理教育部工程研究中心，昆明理工大学，昆明，650500
^2.昆明理工大学信息工程与自动化学院，昆明，650500
^3.云南省人工智能重点实验室，昆明理工大学，昆明，650500

收稿日期:2023-05-29 出版日期:2023-07-31 发布日期:2023-08-18
通讯作者: 张勇丙 E-mail:zhangyongbing419@163.com
基金资助:
国家自然科学基金(62166023);云南省重大科技专项计划(202103AA080015);云南省自然科学基金重点项目(2019FA023)

A generative summary method of cross⁃border ethnic culture incorporating domain knowledge

Guanbo Zhao¹^,²^,³, Yongbing Zhang¹^,²^,³(), Cunli Mao¹^,²^,³, Shengxiang Gao¹^,²^,³, Fengxiao Wang¹^,²^,³

^1.South Asia and Southeast Asia Languages Voice Information Processing Engineering Research Center under the Ministry of Education，Kunming University of Science and Technology，Kunming，650500，China
^2.Faculty of Information Engineering and Automation，Kunming University of Science and Technology，Kunming，650500，China
^3.Yunnan Key Laboratoryof ArtificialIntelligence，Kunming University of Science and Technology，Kunming，650500，China

Received:2023-05-29 Online:2023-07-31 Published:2023-08-18
Contact: Yongbing Zhang E-mail:zhangyongbing419@163.com

摘要/Abstract

摘要：

从跨境民族文化文本中生成具有领域知识的摘要对进一步开展跨境民族文化文本检索、问答等任务具有重要的支撑作用，当前基于深度学习的生成式文本摘要取得了较好的效果，但直接用于跨境民族文化文本摘要任务会导致生成的摘要出现领域词汇丢失的问题.为此，提出一种融入领域知识的跨境民族文化生成式摘要方法（Domain Knowledge?Culture?Generative Summary，DKCGS），在编码端将跨境民族文化领域词典编码与原文本编码融合，以此增强模型对领域词汇的表征能力；在解码端，基于指针生成网络将具有同义或跨境关系的领域词汇分布与原文本分布结合，提高模型生成文化领域词汇的准确率.同时，在通用领域文本上进行预训练并进一步初始化参数，以缓解数据稀缺导致模型训练效果不佳的问题.实验结果表明，提出的方法在跨境民族文本摘要数据集上比基线模型的Rouge?1提升了0.95，有效提升了跨境民族文化文本摘要生成的质量.

关键词: 跨境民族文化, 领域知识, 指针生成网络, 预训练, 文本摘要

Abstract:

Generating summaries with domain knowledge from cross?border ethnic culture texts plays an important supporting role in further carrying out tasks such as cross?border ethnic culture text retrieval and question answering. Currently，generative text summarization based on deep learning has shown promising results. However，directly applying it to cross?border national cultural text summarization tasks may result in the omission of domain?specific words in the generated summary. Therefore，a Domain Knowledge?Culture?Generative Summary (DKCGS) method for cross?border ethnic culture summarization with Domain Knowledge is proposed. In the encoding end，the cross?border ethnic culture domain dictionary coding is integrated with the original text coding in order to enhance the representation ability of the model for domain words. At the decoder，the distribution of domain words with synonymous or cross?border relationships is combined with the distribution of the original text based on the pointer generation network to improve the accuracy of the model in generating cultural domain words. At the same time，it pre?trains and further initializes the parameters on the general domain text to alleviate the problem of poor model training effect caused by data scarcity. Experimental results show that the Rouge?1 of the proposed method is 0.95 higher than that of the baseline model on the cross?border ethnic text summarization dataset，which effectively improves the quality of cross?border ethnic culture text summarization generation.

Key words: cross?border ethnic culture, domain knowledge, pointer?generator network, pre?trained, text summary

中图分类号:

TP391

赵冠博, 张勇丙, 毛存礼, 高盛祥, 王奉孝. 融入领域知识的跨境民族文化生成式摘要方法[J]. 南京大学学报(自然科学版), 2023, 59(4): 620–628.

Guanbo Zhao, Yongbing Zhang, Cunli Mao, Shengxiang Gao, Fengxiao Wang. A generative summary method of cross⁃border ethnic culture incorporating domain knowledge[J]. Journal of Nanjing University(Natural Sciences), 2023, 59(4): 620–628.

图/表 10

图1

图2

图3

表1

表2

表3

表4

表5

表6

图4

参考文献 20

1	Hu B T， Chen Q C， Zhu F Z. LCSTS：A large scale Chinese short text summarization dataset∥Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon，Portugal：ACL，2015：1967-1972.
2	Vaswani A， Shazeer N， Parmar N，et al. Attention is all you need∥Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach，CA，USA：Curran Associates Inc.，2017：6000-6010.
3	Dong L， Yang N， Wang W H，et al. Unified language model pre?training for natural language understanding and generation∥Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver，Canada：Curran Associates Inc.，2019：13063-13075.
4	Aralikatte R， Narayan S， Maynez J，et al. Focus attention：Promoting faithfulness and diversity in summarization∥Proceedings of the 59^th Annual Meeting of the Association for Computational Linguistics，the 11th International Joint Conference on Natural Language Processing. Bangkok，Thailand：ACL，2021：6078-6095.
5	Zhu J N， Zhou Y， Zhang J J，et al. Attend，translate and summarize：An efficient method for neural cross?lingual summarization∥Proceedings of the 58^th Annual Meeting of the Association for Computational Linguistics. Washington DC，WA，USA：ACL，2020：1309-1321.
6	Nallapati R， Zhou B W， Dos Santos C，et al. Abstractive text summarization using sequence?to?sequence RNNs and beyond∥Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. Berlin，Germany：Association for Computational Linguistics，2016：280-290.
7	Dou Z Y， Liu P F， Hayashi H，et al. GSum：a general framework for guided neural abstractive summarization∥Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies. Online：Association for Computational Linguistics，2021：4830-4842.
8	Mihalcea R， Tarau P. TextRank：bringing order into text∥Proceedings of 2004 Conference on Empirical Methods in Natural Language Processing. Barcelona，Spain：ACL，2004：404-411.
9	Liu Y， Lapata M. Text summarization with pretrained encoders∥Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing，the 9th International Joint Conference on Natural Language Processing. Hong Kong，China：Association for Computational Linguistics，2019：3730-3740.
10	Zhou Q Y， Yang N， Wei F R，et al. Selective encoding for abstractive sentence summarization∥Proceedings of the 55^th Annual Meeting of the Association for Computational Linguistics. Volume 1：Long Papers. Vancouver，Canada：Association for Computational Linguistics，2017：1095-1104.
11	Rush A M， Chopra S， Weston J. A neural attention model for abstractive sentence summarization∥Proceedings of 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon，Portugal：Association for Computational Linguistics，2015：379-389.
12	Chopra S， Auli M， Rush A M. Abstractive sentence summarization with attentive recurrent neural networks∥Proceedings of 2016 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies. San Diego，CA，USA：ACL，2016：93-98.
13	See A， Liu P J， Manning C D. Get to the point：Summarization with pointer?generator networks∥Proceedings of the 55^th Annual Meeting of the Association for Computational Linguistics. Volume 1. LongPapers. Vancouver，Canada：ACL，2017：1073-1083.
14	Manakul P， Gales M. Long?span summarization via local attention and content selection∥Proceedings of the 59^th Annual Meeting of the Association for Computational Linguistics，the 11th International Joint Conference on Natural Language Processing. Bangkok，Thailand：ACL，2021：6026-6041.
15	Li C L， Xu W R， Li S，et al. Guiding generation for abstractive text summarization based on key information guide network∥Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies. Volume 2.PapersShort. New Orleans，LA，USA：Association for Computational Linguistics，2018：55-60.
16	Afzal M， Alam F， Malik K M，et al. Clinical context?aware biomedical text summarization using deep neural network：Model development and validation. Journal of Medical Internet Research，2020，22(10)：e19810.
17	蔡中祥，孙建伟. 融合指针网络的新闻文本摘要模型. 小型微型计算机系统，2021，42(3)：462-466.
	Cai Z X， Sun J W. News text summarization model integrating pointer network. Journal of Chinese Computer Systems，2021，42(3)：462-466.
18	Kingma D P， Ba J. Adam：A method for stochastic optimization∥Proceedings of the 3rd International Conference on Learning Representations. San Diego，CA，USA：ICLR，2015，arXiv：.
19	Lin C Y. ROUGE：a package for automatic evaluation of summaries∥Proceedings of the Text Summarization Branches Out. Barcelona，Spain：ACL，2004：74-81.
20	Koehn P. Statistical significance tests for machine translation evaluation∥Proceedings of 2004 Conference on Empirical Methods in Natural Language Processing. Barcelona，Spain：ACL，2004：388-395.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

领域词汇	别称
泼水节	摆爽南、佛诞节、宋干节
开门节	出洼节、出夏节
叫谷魂	招谷魂
婚礼	金欠
关门节	进洼
花街节	赶花街
……	……

项目名称	数值设置
模型隐藏大小	512
前馈隐藏大小	1024
编解码器层数	6层
Transformer头数	4
Batch_size	12
epochs	20
dropout	0.1
Beam_search	3

	模型名称	Rouge⁃1	Rouge⁃2	Rouge⁃L
抽取式方法	Lead⁃1⁃First	23.53	11.68	22.83
抽取式方法	TextRank	23.86	11.54	22.46
生成式方法	Pointer	24.64	12.71	23.36
	RNN Context	24.01	12.56	23.12
	Unlim	25.83	12.61	24.37
	FAME	25.67	12.96	24.64
	Tri⁃PCN	24.88	12.64	24.17
	DKCGS	26.62	13.63	25.41

词典编码方式	Rouge⁃1	Rouge⁃2	Rouge⁃L
随机初始化	22.57	11.43	21.11
GloVe	26.31	13.47	24.86
Word2vec	26.62	13.62	25.41

方法	Rouge⁃1	Rouge⁃2	Rouge⁃L
DKCGS	26.62	13.63	25.41
Transformer	13.56	7.02	11.22
Transformer+预训练	24.23	11.07	22.48
Transformer+预训练+领域词典	26.14	13.21	24.56

融入领域知识的跨境民族文化生成式摘要方法

A generative summary method of cross⁃border ethnic culture incorporating domain knowledge

RichHTML

PDF (PC)

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 20

相关文章 7

Metrics

本文评价

推荐阅读 0

词典规模	Rouge⁃1	Rouge⁃2	Rouge⁃L
1000	24.35	11.37	22.54
1500	25.24	12.37	22.96
2000	25.67	12.72	23.98
2500	26.43	13.53	25.11
3000	26.62	13.62	25.40
3123	26.62	13.63	25.41

[1]	刘思源, 毛存礼, 张勇丙. 基于领域知识图谱和对比学习的汉越跨境民族文本检索方法[J]. 南京大学学报(自然科学版), 2023, 59(4): 610-619.
[2]	杨京虎, 段亮, 岳昆, 李忠斌. 基于子事件的对话长文本情感分析[J]. 南京大学学报(自然科学版), 2023, 59(3): 483-493.
[3]	唐伟佳, 张华, 侯志荣. 基于空间卷积融合的中文文本匹配方法[J]. 南京大学学报(自然科学版), 2022, 58(5): 868-875.
[4]	张俊, 陈秀宏. 基于BERT模型的无监督候选词生成及排序算法[J]. 南京大学学报(自然科学版), 2022, 58(2): 286-297.
[5]	陈炳鑫, 陈黎飞. 符号序列的预训练HMM分类方法[J]. 南京大学学报(自然科学版), 2021, 57(1): 52-58.
[6]	李钝，薛昊原，李伦，郑志蕴*. 面向教学资源的实体链接算法[J]. 南京大学学报(自然科学版), 2015, 51(4): 901-908.
[7]	陈鹏1郭剑毅1,2余正涛1,2严馨1,2张志坤1,2高盛祥1,2. 融合领域知识短语树核函数的中文领域实体关系抽取[J]. 南京大学学报(自然科学版), 2015, 51(1): 181-186.