南京大学学报(自然科学版) ›› 2019, Vol. 55 ›› Issue (6): 1000–1009.doi: 10.13232/j.cnki.jnju.2019.06.012

• • 上一篇    下一篇

基于层次化表示的隐式篇章关系识别

徐扬,周文瑄,阮慧彬,孙雨,洪宇()   

  1. 苏州大学计算机科学与技术学院，苏州，215006
  • 收稿日期:2019-04-22 出版日期:2019-11-30 发布日期:2019-11-29
  • 通讯作者: 洪宇 E-mail:tianxianer@gmail.com
  • 基金资助:
    国家自然科学基金(61672368);江苏高校优势学科建设工程

Hierarchical representation for implicit discourse relation recognition

Yang Xu,Wenxuan Zhou,Huibin Ruan,Yu Sun,Yu Hong()   

  1. School of Computer Science and Technology，Soochow University，Suzhou，251006，China
  • Received:2019-04-22 Online:2019-11-30 Published:2019-11-29
  • Contact: Yu Hong E-mail:tianxianer@gmail.com

RichHTML

6144

PDF (PC)

1144

摘要：

篇章关系识别研究旨在理解篇章内部论述单元（简称“论元”，包括短语、句子及文本片段）之间的语义连接关系.现有研究通过交互式注意力机制方法，提升论元之间的信息的交互性，从而提升模型的分类能力.尽管如此，仅通过提升论元间的信息交互不能表述论元对的整体语义概念，原因在于现有方法往往将论元对视作独立的个体，忽略上下文信息对其语义上的影响.针对以上问题，提出一种基于层次化表示的隐式篇章关系识别方法，通过基于词的交互式注意力机制提取出较为重要的单词或短语，并通过论元的注意力机制赋予关键论元较高的权重，最终通过基于上下文的注意力机制融合论元对所在段落的信息，获得具有上下文语义信息的论元对表示.该方法进一步强化了论元之间信息交互性，同时强化了论元对与上下文信息间的交互.使用PDTB（Penn Discourse Treebank）语料进行实验，结果证明该方法的F 1值在四个大类关系（Comparison，Contingency，Expansion，Temporal）上相对基准系统提高了4.94%，5.43%，4.57%和7.42%.

关键词: 篇章关系识别, 注意力机制, 信息交互, 上下文信息

Abstract:

Discourse relation recognition aims at identifying semantic relations between discourse spans such as clauses，sentences or text spans which called arguments. Existing research improves information interactions between arguments through interactive attention mechanism，thus enhancing the classification ability of models. However，enhancing the interactive information between arguments can't express the whole semantic of the argument pair. Existing methods often regard argument pairs as independent individuals，ignoring the semantic impact of their context information. Therefore，we propose a method based on hierarchical representation. This method uses word?based attention mechanism to extract important words or phrases and leverages argument?based attention mechanism to give higher weights to more important arguments. Finally，we obtain the argument pair representation with context information，which further strengthens the interaction of information between arguments through the context?aware attention mechanism. This method further strengthens the interactions between arguments，and also strengthens the interactions between arguments and context information. This paper conducts experiments on PDTB （Penn Discourse Treebank）corpus，and the experimental results show that the proposed method increases the performance with 4.94%，5.43%，4.57% and 7.42% respectively on four top relations (Comparison，Contingency，Expansion and Temporal).

Key words: discourse relation recognition, attention mechanism, information interaction, context?aware information

中图分类号: 

  • TP391

图1

篇章关系识别任务框架"

图2

层次化表示方法的框架"

表1

PDTB隐式篇章关系数据统计"

关系类型 训练集 开发集 测试集
Comp. 1855 189 145
Cont. 3235 281 273
Expa. 6673 638 538
Temp. 582 48 55
总计 12345 1156 1011

表2

四个实验系统在PDTB语料上的性能"

Comp. Cont. Expa. Temp.
Bi?LSTM 33.33 50.75 67.84 29.81
Word Representation 36.67 52.25 69.22 33.69
Argument Representation 37.05 54.37 71.37 35.14
Context?Aware Representation 38.27 56.18 72.41 37.23

表3

本文所提模型与现有模型的对比"

Comp. Cont. Expa. Temp.
Ji2015 35.93 52.78 - 27.63
Chen2016 40.17 54.76 - 31.32
Qin2016 40.87 57.32 71.5 35.43
Liu2016 37.91 55.58 69.97 37.17
Bai2018 47.85 54.47 70.60 36.97
Context?Aware Representation 38.27 56.18 72.41 37.23
1 Zhou L J , Li B Y , Gao W ,et al . Unsupervised discovery of discourse relations for eliminating intra-sentence polarity ambiguities∥Proceedings of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA,USA：ACL,2011：162-171.
2 Riaz M , Girju R . Another look at causality：discovering scenario?specific contingency relation?ships with no supervision∥2010 IEEE 4th International Conference on Semantic Computing. Pittsburgh,PA,USA：IEEE,2010：361-368.
3 Do Q X , Chan Y S , Roth D . Minimally supervised event causality identification∥Proceedings of the Conference on Empirical Methods in Natural Language Processing. Edinburgh,UK：ACL,2011：294-303.
4 Litkowski K C . Question?answering using seman?tic relation triples∥Proceedings of the 8th Text Retrieval Conference. Gaithersburg,MD,USA：NIST,1999：349.
5 Yoshida Y , Suzuki J , Hirao T ,et al . Dependency?based discourse parser for single?document summarization∥Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha,Qatar：ACL,2014：1834-1839.
6 Meyer T , Popescu?Belis A . Using sense?labeled discourse connectives for statistical machine translation∥Proceedings of the Joint Workshop on Exploiting Synergies between Information Retrieval and Machine Translation (ESIRMT) and Hybrid Approaches to Machine Translation (HyTra). Stroudsburg,PA,USA：ACL,2012：129-138.
7 Xiong D Y , Ding Y , Zhang M ,et al . Lexical chain based cohesion models for document?level statis?tical machine translation∥Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle,WA,USA：ACL,2013：1563-1573.
8 Meyer T , Webber B . Implicitation of discourse connectives in (machine) translation∥Proceedings of the Workshop on Discourse in Machine Translation. Sofia,Bulgaria：ACL,2013：19-26.
9 Prasad R , Dinesh N , Lee A ,et al . The penn discourse TreeBank 2.0∥Proceedings of the 6th International Conference on Language Resources and Evaluation. Marrakech,Morocco：ELRA,2008：2961-2968.
10 Zhang B , Su J S , Xiong D Y ,et al . Shallow convolutional neural network for implicit discourse relation recognition∥Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon,Portugal：ACL,2015：2230-2235.
11 Liu Y , Li S , Zhang X ,et al . Implicit discourse relation classification via multi?task neural networks∥Proceedings of AAAI. Phoenix,AZ,USA：AAAI Press,2016：2750-2756.
12 Qin L H , Zhang Z S , Zhao H . A stacking gated neural architecture for implicit discourse relation classification∥Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin,Texas,USA：ACL,2016：2263-2270.
13 Wu C , Shi X , Chen Y ,et al . Bilingually?constrained synthetic data for implicit discourse relation recognition∥Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin，TX,USA：ACL,2016：2306-2312.
14 曾义夫,蓝天,吴祖峰 等 . 基于双记忆注意力的方面级别情感分类模型. 计算机学报,2019,42：1-14.
Zeng Y F , Lan T , Wu Z F ,et al . Bi?memory based attention model for aspect level sentiment classification. Chinese Journal of Computers,2019,42：1-14.
15 郑玉昆,李丹,范臻 等 . T?Reader：一种基于自注意力机制的多任务深度阅读理解模型. 中文信息学报,2018,32(11)：128-134.
Zheng Y K , Li D , Fan Z ,et al . T?reader：a multi?task deep reading comprehension model with self?attention mechanism. Journal of Chinese Information Processing,2018,32(11)：128-134.
16 Choi H , Cho K , Bengio Y . Fine?grained attention mechanism for neural machine translation. Neurocomputing,2018,284：171-176.
17 Ma D H , Li S J , Zhang X D ,et al . Interactive attention networks for aspect?level sentiment classification. arXiv：1709.00893,2017.
18 Pennington J , Socher R , Manning C D . Glove：Global vectors for word representation∥Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP). Doha,Qatar：ACL,2014：1532-1543.
19 Qian N . On the momentum term in gradient descent learning algorithms. Neural Networks,1999,12(1)：145-151.
20 Ji Y F , Eisenstein J . One vector is not enough：entity?augmented distributed semantics for discourse relations. Transactions of the Association for Computational Linguistics,2015,3：329-344.
21 Chen J F , Zhang Q , Liu P F ,et al . Implicit discourse relation detection via a deep architecture with gated relevance network∥Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin,Germany：ACL,2016,1：1726-1735.
22 Liu Y , Li S J . Recognizing implicit discourse relations via repeated reading：Neural networks with multi?level attention. 2016，arXiv：1609. 06380.
23 Bai H X , Zhao H . Deep enhanced representation for implicit discourse relation recognition. 2018，arXiv：1807.05154.
[1] 朱伟,张帅,辛晓燕,李文飞,王骏,张建,王炜. 结合区域检测和注意力机制的胸片自动定位与识别[J]. 南京大学学报(自然科学版), 2020, 56(4): 591-600.
[2] 曹欣怡,李鹤,王蔚. 基于语料库的语音情感识别的性别差异研究[J]. 南京大学学报(自然科学版), 2019, 55(5): 758-764.
[3] 钱付兰, 黄鑫, 赵姝, 张燕平. 基于路径相互关注的网络嵌入算法[J]. 南京大学学报(自然科学版), 2019, 55(4): 573-580.
[4] 顾健伟, 曾　诚, 邹恩岑, 陈　扬, 沈　艺, 陆　悠, 奚雪峰. 基于双向注意力流和自注意力结合的机器阅读理解[J]. 南京大学学报(自然科学版), 2019, 55(1): 125-132.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 缪长健, 施斌, 郑兴, 王湛, 魏广庆. 海上超长PHC管桩BOFDA内力测试[J]. 南京大学学报(自然科学版), 2018, 54(6): 1057 -1063 .
[2] 段新春，施　斌，孙梦雅，魏广庆，顾　凯，冯晨曦. FBG蒸发式湿度计研制及其响应特性研究[J]. 南京大学学报(自然科学版), 2018, 54(6): 1075 -1084 .
[3] 韩明鸣, 郭虎升, 王文剑. 面向非平衡多分类问题的二次合成QSMOTE方法[J]. 南京大学学报(自然科学版), 2019, 55(1): 1 -13 .
[4] 李　巍, 王　鸥, 刚毅凝, 周杨浩, 郝跃冬. 一种自动读取指针式仪表读数的方法[J]. 南京大学学报(自然科学版), 2019, 55(1): 117 -124 .
[5] 仲昭朝, 邹　婷, 唐惠炜, 庄　重, 张　臻. 铜胁迫对蚕豆根尖细胞凋亡及线粒体功能的影响[J]. 南京大学学报(自然科学版), 2019, 55(1): 154 -160 .
[6] 关立强, 祝伟光, 李义丰. Lamb波时间反转椭圆定位和层析成像混合技术研究[J]. 南京大学学报(自然科学版), 2019, 55(2): 191 -201 .
[7] 彭　宏, 刘　晨, 卢为党, 张　昱, 刘　鑫, 徐志江. 协作中继传输网络中基于无线供电的能量交易方法[J]. 南京大学学报(自然科学版), 2019, 55(2): 202 -210 .
[8] 李嘉明, 邹　勇, 郑　浩, 魏钟波, 杨柳燕, 缪爱军. 养殖塘生态系统重金属污染状况与风险评价[J]. 南京大学学报(自然科学版), 2019, 55(2): 272 -281 .
[9] 杨　蕴, 宋　健, 朱　琳, 吴剑锋, 王锦国. 基于KELM地面沉降替代模型的地下水多目标管理模型研究[J]. 南京大学学报(自然科学版), 2019, 55(3): 349 -360 .
[10] 姚宁, 苗夺谦, 张远健, 康向平. 属性的变化对于流图的影响[J]. 南京大学学报(自然科学版), 2019, 55(4): 519 -528 .

版权所有 © 2021 《南京大学学报(自然科学)》

地址：江苏省南京市鼓楼区汉口路22号，南京大学学报编辑部，210093

电话：025-83592704 , 传真：025-83592704        技术支持：北京玛格泰克科技发展有限公司