推荐系统中的准确性、新颖性和多样性的有效耦合与应用

doi:10.13232/j.cnki.jnju.2022.04.005

南京大学学报(自然科学版) ›› 2022, Vol. 58 ›› Issue (4): 604–614.doi: 10.13232/j.cnki.jnju.2022.04.005

• • 上一篇

Effective fusion and application of accuracy, novelty and diversity in recommender system

Di Han¹^,³, Yijun Chen²(), Kai Liao³, Kunling Lin³

^1.Guangdong University of Finance, Guangzhou, 510521, China
^2.Xi'an Aeronautical Institute, Xi'an, 710077, China
^3.OSChina Gitee Institute, Shenzhen, 518000, China

Received:2022-05-04 Online:2022-07-30 Published:2022-08-01
Contact: Yijun Chen E-mail:201907034@xaau.edu.cn

摘要/Abstract

摘要：

目前，基于人工智能推荐系统的研究工作大多集中在算法优化上，而关于推荐系统更重要的性能评价指标往往被忽视.具体地，独立的评价指标无法有效地反映算法之间的差异，需要进一步有效地耦合这些评价指标.为了反映推荐系统性能的差异，提出较合理的性能评估框架AND （Accuracy Novelty Diversity），可以同时反映推荐系统整体的准确性、新颖性和多样性.把AND框架融入主流的序列化推荐模型，命名为SASAND （Self?Attentive Sequential?AND）.实验结果表明，提出的AND框架在假设数据集和基准数据集的基础上，能有效反映准确性相似的不同算法之间推荐性能的差异，同时，提出的SASAND模型基于AND框架的约束，能对推荐的结果在综合考虑准确性、新颖性和多样性的前提下重新排序.与主流的推荐模型对比，SASAND能够尽最大可能达到整体最优的推荐性能输出.

关键词: 推荐系统, 指标, 准确性, 新颖性, 多样性

Abstract:

At present，most of the research work on recommender system (RS) based on artificial intelligence focuses on the algorithm optimization，but the evaluation metrics being an important process to evaluate the performance of RS are usually ignored. Specifically，independent evaluation metrics cannot effectively reflect the differences between algorithms，so how to effectively fusion these evaluation metrics needs further improvement. In order to reflect the difference in RS performance，this paper proposes a rational evaluation framework for RS performance，named AND (Accuracy Novelty Diversity)，which can simultaneously reflect metrics of accuracy，novelty and diversity. At the same time，we integrate AND framework into the mainstream recommendation model，named SASAND (Self?Attentive Sequential?AND). Experimental results show that the proposed AND framework is based on hypothetical datasets and benchmark datasets，and can effectively reflect the difference between the recommendation performance with similar seeming accuracy between different algorithms. At the same time，the proposed SASAND model is based on the constraints of the AND framework，which can re?rank the recommended results under the comprehensive consideration of accuracy，novelty and diversity. Compared with the mainstream recommendationer models，SASAND achieves the best overall recommend performance output as much as possible.

Key words: recommender system, metrics, accuracy, novelty, diversity

中图分类号:

TP181

韩迪, 陈怡君, 廖凯, 林坤玲. 推荐系统中的准确性、新颖性和多样性的有效耦合与应用[J]. 南京大学学报(自然科学版), 2022, 58(4): 604–614.

Di Han, Yijun Chen, Kai Liao, Kunling Lin. Effective fusion and application of accuracy, novelty and diversity in recommender system[J]. Journal of Nanjing University(Natural Sciences), 2022, 58(4): 604–614.

图/表 10

图1

图2

表1

表2

两个推荐列表R1和R2的差异指标中的评估性能"

Metrics	R1	R2
AND	0.3433	0.6866
NDCG	0.8614	0.8614
EPC	0.6866	0.6866
CC	1	1
$H E P C, C C$	0.8142	0.8142

表2

表3

表4

表5

SASAND模型在ML?100k数据集上由不同的γ得到的不同指标对比"

$γ$	NDCG	HR	AND	EPC	CC
0	0.4418	0.7253	0.7870	0.8590	0.9275
0.1	0.4572	0.7433	0.7903	0.8641	0.926
0.2	0.4635	0.7476	0.8083	0.8728	0.9361
0.3	0.4376	0.7158	0.8211	0.8757	0.9472
0.4	0.4192	0.6924	0.8349	0.8832	0.9564
0.5	0.3841	0.6648	0.8571	0.8859	0.9777
0.6	0.3452	0.6203	0.8918	0.8882	1.0139
0.7	0.3097	0.5949	0.9213	0.8898	1.0460
0.8	0.2027	0.4103	1.0158	0.8761	1.1706
0.9	0.0659	0.1463	1.0770	0.8968	1.2124
1	0.0468	0.1155	1.0397	0.9143	1.1476

表5

表6

统计不同γ的NDCG以及AND的变化量CR"

$γ$	NDCG	ΔNDCG	AND	ΔAND	CR
0	0.4418	-	0.7870	-	-
0.1	0.4572	0.0154	0.7903	0.0033	4.6667
0.2	0.4635	0.0217	0.8083	0.0213	1.0188
0.3	0.4376	-0.0042	0.8211	0.0341	0.1232
0.4	0.4192	-0.0226	0.8349	0.0479	0.4718
0.5	0.3841	-0.0577	0.8571	0.0701	0.8231
0.6	0.3452	-0.0966	0.8918	0.1048	0.9217
0.7	0.3097	-0.1321	0.9213	0.1343	0.9836
0.8	0.2027	-0.2391	1.0158	0.2288	1.0450
0.9	0.0659	-0.3759	1.0770	0.2900	1.2962
1.0	0.0468	-0.395	1.0397	0.2527	1.5631

表6

表7

实验数据集的统计数据（数据预处理后）"

数据集	#用户	#项目	#类别	密度	$γ$
ML⁃100k	943	1682	19	6.305%	0.3
ML⁃1m	6040	3706	18	4.468%	0.3
LastFM	1892	12523	9749	0.787%	0.1
Serendipity	104661	49151	982	0.194%	0.1

表7

表8

SASAND与原始SAS在不同数据集上的NDCG和AND指标的对比"

数据集	Metrics	SASRec	SASAND
ML⁃100k	$N D C G @ 10$	0.4418	0.4376
ML⁃100k	$A N D @ 10$	0.7870	0.8211
ML⁃1m	$N D C G @ 10$	0.5905	0.5775
ML⁃1m	$A N D @ 10$	0.7400	0.8124
LastFM	$N D C G @ 10$	0.2279	0.1993
LastFM	$A N D @ 10$	1.8917	2.4435
Serendipity	$N D C G @ 10$	0.8128	0.8106
Serendipity	$A N D @ 10$	2.0403	2.1101

表8

参考文献 27

1	Sun Z， Yu D， Fang H，et al. Are we evaluating rigorously? Benchmarking recommendation for reproducible evaluation and fair comparison∥The 14^th ACM Conference on Recommender Systems.Online：ACM，2020：23-32.
2	Cai X J， Hu Z M， Zhao P，et al. A hybrid recommendation system with many?objective evolutionary algorithm. Expert Systems with Applications，2020(159)：113648.
3	Wang S J， Hu L， Wang Y，et al. Sequential recommender systems：Challenges，progress and prospects. 2019,arXiv:.
4	Lai G K， Chang W C， Yang Y M，et al. Modeling long?and short?term temporal patterns with deep neural networks∥The 41^st International ACM SIGIR Conference on Research & Development in Information Retrieval. Ann Arbor，MI，USA：ACM，2018：95-104.
5	Adomavicius G， Tuzhilin A. Toward the next generation of recommender systems：A survey of the state?of?the?art and possible extensions. IEEE Transactions on Knowledge and Data Engineering，2005，17(6)：734-749.
6	Han D， Li J Q， Yang L，et al. A recommender system to address the Cold Start problem for App usage prediction. International Journal of Machine Learning and Cybernetics，2019，10(9)：2257-2268.
7	Silveira T， Zhang M， Lin X，et al. How good your recommender system is? A survey on evaluations in recommendation. International Journal of Machine Learning and Cybernetics，2019，10(5)：813-831.
8	Rendle S. Factorization machines∥2010 IEEE International Conference on Data Mining. Sydney，Australia：IEEE，2010：995-1000.
9	Guo H F， Tang R M， Ye Y M，et al. DeepFM：A factorization?machine based neural network for CTR prediction∥Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne，Australia：AAAI Press，2017：1725-1731.
10	Li J C， Wang Y J， McAuley J. Time interval aware self?attention for sequential recommendation∥Proceedings of the 13th International Conference on Web Search and Data Mining. Houston，TX，USA：ACM，2020：322-330.
11	Covington P， Adams J， Sargin E. Deep neural networks for YouTube recommendations∥Proceedings of the 10th ACM Conference on Recommender Systems. Boston，MA，USA：ACM，2016：191-198.
12	Adamopoulos P， Tuzhilin A. On unexpectedness in recommender systems：Or how to better expect the unexpected. ACM Transactions on Intelligent Systems and Technology，2015，5(4)：54.
13	Vargas S， Castells P. Rank and relevance in novelty and diversity metrics for recommender systems∥Proceedings of the 5th ACM Conference on Recommender Systems. Chicago，IL，USA：ACM，2011：109-116.
14	Shepherd J M， Burian S J. Detection of urban?induced rainfall anomalies in a major coastal city. Earth Interactions，2003，7(4)：1-17.
15	Krichene W， Rendle S. On sampled metrics for item recommendation∥Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York，NY，USA：Association for Computing Machinery，2020：1748-1757.
16	Zhang Y C， Séaghdha D ó， Quercia D，et al. Auralist：Introducing serendipity into music recommendation∥Proceedings of the 5th ACM International Conference on Web Search and Data Mining. Seattle，WA，USA：ACM，2012：13-22.
17	Abdollahpouri H， Burke R， Mobasher B. Managing popularity bias in recommender systems with personalized re?ranking∥Proceedings of the 32nd International Florida Artificial Intelligence Research Society Conference. Sarasota，FL，USA：AAAI Press，2019.
18	Liu W W， Burke R. Personalizing fairness?aware re?ranking. 2018，arXiv:.
19	Pei C H， Ou W， Pei D F，et al. Personalized re?ranking for recommendation∥Proceedings of the 13th ACM Conference on Recommender Systems. Copenhagen，Denmark：ACM，2019：3-11.
20	Hurley N， Zhang M. Novelty and diversity in Top?N recommendation：Analysis and evaluation. ACM Transactions on Internet Technology，2011，10(4)：14.
21	冯晨娇，宋鹏，王智强，等. 一种基于3因素概率图模型的长尾推荐方法. 计算机研究与发展，2021，58(9)：1975-1986.
	Feng C J， Song P， Wang Z Q，et al. A method on long tail recommendation based on three?factor probabilistic graphical model. Journal of Computer Research and Development，2021，58(9)：1975-1986.
22	Adomavicius G， Kwon Y. Improving aggregate recommendation diversity using ranking?based techniques. IEEE Transactions on Knowledge and Data Engineering，2012，24(5)：896-911.
23	Ziegler C N， McNee S M， Konstan J A，et al. Improving recommendation lists through topic diversification∥Proceedings of the 14th International Conference on World Wide Web. Chiba，Japan：ACM，2005：22-32.
24	Ricci F， Rokach L， Shapira B. Introduction to recommender systems handbook∥Ricci F，Rokach L，Shapira B，et al. Recommender systems handbook. Springer Berlin Heidelberg，2011：1-35.
25	Swamy M K， Reddy P K. A model of concept hierarchy?based diverse patterns with applications to recommender system. International Journal of Data Science and Analytics，2020，10(2)：177-191.
26	Kang W C， McAuley J. Self?attentive sequential recommendation∥2018 IEEE International Conference on Data Mining. Singapore，Singapore：IEEE，2018：197-206.
27	Lin T Y， Goyal P， Girshick R，et al. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42(2)：318-327.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

	NDCG	EPC	AND	CC	ANDne
Normal Predictor	0.8306	0.5760	0.6060	1.3710	1.0874
SVD	0.9327	0.6662	0.8084	1.3719	0.9944
SVD++	0.9174	0.6423	0.7588	1.3623	0.9804
KNN Basic	0.9319	0.6522	0.8000	1.3909	0.9804
KNN with Means	0.9319	0.6529	0.8030	1.3867	0.9948
KNN with Z⁃Score	0.9306	0.6529	0.8017	1.3945	0.9967
KNN Baseline	0.9309	0.6556	0.796	1.3785	0.9887
NMF	0.9097	0.6482	0.7625	1.3762	1.0131
SlopeOne	0.8968	0.6138	0.7174	1.3893	0.9921
CoClustering	0.8909	0.6099	0.6987	1.3707	0.9872
Randomchoice	0.8309	0.5737	0.6082	1.3895	1.0998
CNN	0.8526	0.6091	0.6641	1.3592	1.0843
FM	0.8378	0.5864	0.5477	1.2341	1.0738

	NDCG	EPC	AND	CC	ANDne
NormalPredictor	0.8432	0.5640	0.7161	0.9246	0.7524
SVD	0.8428	0.5622	0.7162	0.9238	0.7505
SVD++	0.8433	0.5622	0.7175	0.9237	0.7510
KNN Basic	0.8431	0.5610	0.7168	0.9231	0.7495
KNN with Means	0.8415	0.5608	0.7145	0.9238	0.7499
KNN with Z⁃Score	0.8429	0.5633	0.7164	0.9256	0.7517
KNN Baseline	0.8429	0.5644	0.7178	0.9267	0.7535
NMF	0.8438	0.5737	0.7188	0.9243	0.7513
SlopeOne	0.8436	0.5625	0.7176	0.9231	0.7507
CoClustering	0.8425	0.5610	0.7161	0.9233	0.7494
Randomchoice	0.8309	0.5608	0.6082	1.3895	1.0998
CNN	0.8429	0.5611	0.7165	0.9246	0.7516
FM	0.8426	0.5614	0.7148	0.9148	0.7489

[1]	吕亚兰, 徐媛媛, 张恒汝. 一种可解释性泛化矩阵分解推荐算法[J]. 南京大学学报(自然科学版), 2022, 58(1): 135-142.
[2]	武聪, 马文明, 王冰, 朱建豪. 融合用户标签相似度的矩阵分解算法[J]. 南京大学学报(自然科学版), 2022, 58(1): 143-152.
[3]	郝昱猛, 马文明, 王冰. 基于特定用户约束的概率矩阵分解算法[J]. 南京大学学报(自然科学版), 2021, 57(5): 818-827.
[4]	袁晓峰, 钱苏斌, 周彩根. 基于填充先验约束的矩阵分解算法[J]. 南京大学学报(自然科学版), 2021, 57(2): 197-207.
[5]	李佳佳, 丁伟, 王伯伟, 聂秀山, 崔超然. 基于随机森林的民俗体育对身体指标影响评估方法[J]. 南京大学学报(自然科学版), 2021, 57(1): 59-67.
[6]	洪佳明,黄云,刘少鹏,印鉴. 具有结果多样性的近似子图查询算法[J]. 南京大学学报(自然科学版), 2019, 55(6): 960-972.
[7]	徐媛媛,张恒汝,闵帆,黄雨婷. 三支交互推荐[J]. 南京大学学报(自然科学版), 2019, 55(6): 973-983.
[8]	何轶凡, 邹海涛, 于化龙. 基于动态加权Bagging矩阵分解的推荐系统模型[J]. 南京大学学报(自然科学版), 2019, 55(4): 644-650.
[9]	傅秀章1，2*，曹　玥1，刘　珏3. 苏南地区不同教堂声学性能比较研究[J]. 南京大学学报(自然科学版), 2017, 53(4): 686-.
[10]	李兴亮1，毛　睿2*. 基于近期最远遍历的支撑点选择[J]. 南京大学学报(自然科学版), 2017, 53(3): 483-.
[11]	万晨洁，余益军，张　莉，张晓辉，刘红玲*，于红霞. 太湖有机污染物的生态风险研究[J]. 南京大学学报(自然科学版), 2017, 53(2): 256-.
[12]	韩永和，贾梦茹，傅景威，向　萍，史孝霞，崔昕毅，罗　军，陈焱山*. 不同浓度砷酸盐胁迫对蜈蚣草根际微生物群落功能多样性特征的影响[J]. 南京大学学报(自然科学版), 2017, 53(2): 275-.
[13]	屈伟洋, 俞　扬. 多样性正则的神经网络训练方法探索[J]. 南京大学学报(自然科学版), 2017, 53(2): 340-.
[14]	孟　娜1，梁吉业1，2*，庞天杰1. 一种基于抽样的谱聚类集成算法 [J]. 南京大学学报(自然科学版), 2016, 52(6): 1090-.
[15]	周治平，张道文*，王杰锋，孙子文. 基于流形结构邻域选择的局部投影近邻传播算法[J]. 南京大学学报(自然科学版), 2015, 51(4): 741-748.