南京大学学报(自然科学版) ›› 2021, Vol. 57 ›› Issue (5): 724–734.doi: 10.13232/j.cnki.jnju.2021.05.002

• • 上一篇    

基于联合轨迹特征的徘徊行为识别方法

卢锦亮1, 吴广潮1, 冯夫健2,3(), 王林3   

  1. 1.华南理工大学数学学院，广州，510640
    2.华南理工大学软件学院，广州，510640
    3.贵州民族大学，贵州省模式识别与智能系统重点实验室，贵阳，550025
  • 收稿日期:2021-06-23 出版日期:2021-09-29 发布日期:2021-09-29
  • 通讯作者: 冯夫健 E-mail:fujian_feng@gzmu.edu.cn
  • 作者简介:E⁃mail：fujian_feng@gzmu.edu.cn
  • 基金资助:
    贵州省科技计划(黔科合基础［2019］1164号);贵州省教育厅青年项目(黔教合KY字［2021］104);贵州省教育厅创新群体重大研究项目(黔教合KY字［2018］018);广东省普通高校青年创新人才类项目(2019KQNCX186)

Wandering recognition method based on joint trajectory features

Jinliang Lu1, Guangchao Wu1, Fujian Feng2,3(), Lin Wang3   

  1. 1.School of Mathematics，South China University of Technology，Guangzhou，510640，China
    2.School of Software Engineering，South China University of Technology，Guangzho, 510640，China
    3.Key Laboratory of Pattern Recognition and Intelligent System，Guizhou Minzu University，Guiyang，550025，China
  • Received:2021-06-23 Online:2021-09-29 Published:2021-09-29
  • Contact: Fujian Feng E-mail:fujian_feng@gzmu.edu.cn

RichHTML

585

PDF (PC)

769

摘要：

徘徊行为判定作为异常行为识别的热点难题，目前存在难以构建具有场景特定性的识别模型问题.特定场景下的行人轨迹数据，或是有标签，或是无标签，针对如何利用特定场景下有标签轨迹数据信息问题，结合行人多种轨迹特征，提出基于分类函数的徘徊行为识别模型，能够通过分类函数自动学习该场景下有标签轨迹数据中的徘徊行为模式；针对如何利用特定场景下无标签轨迹数据信息问题，提出基于异常检测的徘徊行为识别模型，能够在大量无标签轨迹数据中自动学习潜在的徘徊行为模式.基于两种徘徊行为识别模型，提出徘徊行为识别框架，能够利用目标跟踪算法获取特定场景视频中行人的轨迹数据，并根据数据是否带有标签合适地构建对应的具有场景特定的徘徊行为识别模型.为了验证所提模型的有效性，选用CASIA?AR公开数据集作为测试集，和三种基准模型一起，在行走、奔跑和徘徊行为的识别上进行了对比实验.实验结果表明，所提模型在测试集上的准确率和召回率都优于基准模型，F1指标也有提升，验证了所提模型的有效性、场景特定性和迁移性.

关键词: 徘徊行为识别模型, 异常行为, 分类函数, 异常值检测, 轨迹拟合, 联合轨迹特征

Abstract:

Wandering behavior determination is hot in abnormal behaviors recognition problem. At present，it is difficult to construct a scene?specific recognition model. Pedestrians' trajectory data from specific scene may be labeled or unlabeled. In view of the problem of how to use the information of labeled trajectory data from a specific scene，this paper proposes a novel wandering behavior recognition model based on classification functions by combining multiple trajectory features of pedestrians，which can automatically learn the wandering behavioral patterns from labeled trajectory data from the specific scene through the classification functions. In view of the problem of how to use the information of unlabeled trajectory data from a specific scene，this paper proposes a novel wandering behavior recognition model based on outlier detection，which can automatically learn potential wandering behavioral patterns from a large amount of unlabeled trajectory data from the specific scene. Based on these two novel wandering behavior recognition models，this paper proposes a wandering behavior recognition framework which uses object tracking algorithm to obtain pedestrians' trajectory data from videos of specific scene，and constructs scene?specific wandering behavior recognition model according to whether the data has labels or not. We use CASIA?AR public dataset as the test set，and verify the effectiveness，scene specificity and transfer ability of the proposed model by comparing the recognition result on walking，running and wandering behavior with three benchmark models. Experimental results show that the proposed models have high precision and recall on the test set, yielding an improvement F1 score compared with three benchmark models.

Key words: wandering recognition model, abnormal behavior, classification functions, outlier detection, trajectory fitting, joint trajectory features

中图分类号: 

  • TP391

图1

徘徊行为识别框架"

表1

实验使用的数据集"

DatasetPublicNumber of DataAnnotation
CASIA?ARPublic48Labeled
MOT?PrivatePrivate3Unlabeled
VOT?PrivatePrivate13Labeled

表2

MOT?Private数据集"

NameDescription
IMG_9023multi?object scene，2∶00，30 fps，unlabel
IMG_9024multi?object scene，2∶00，30 fps，unlabel
IMG_9027multi?object scene，2∶00，30 fps，unlabel

表3

VOT?Private数据集"

Video NumberAnnotationDescription
01walksingle?object scene，30 fps
02wandersingle?object scene，30 fps
03walksingle?object scene，30 fps
04walksingle?object scene，30 fps
05walksingle?object scene，30 fps
06wandersingle?object scene，30 fps
07wandersingle?object scene，30 fps
08wandersingle?object scene，30 fps
09wandersingle?object scene，30 fps
10runsingle?object scene，30 fps
11runsingle?object scene，30 fps
12wandersingle?object scene，30 fps
13wandersingle?object scene，30 fps

图2

不同轨迹对应的原始轨迹曲线和拟合轨迹曲线"

图3

4号轨迹（左）和12号轨迹（右）的曲率频数分布直方图"

表4

13条轨迹的Hκ及其类别"

Track NumberValue of HκIs Wandering
010.053no
021.277yes
030.478no
040.366no
050.203no
061.300yes
073.221yes
082.190yes
092.124yes
100.227no
110.000no
123.056yes
130.619yes

图4

改变区间划分后4号轨迹（左）和12号轨迹（右）的曲率频数分布直方图"

表5

改变区间划分后13条轨迹的Hκ及其类别"

Track NumberValue of HκIs Wandering
010.000no
023.881yes
032.752no
040.811no
050.000no
064.281yes
074.385yes
085.107yes
094.778yes
100.000no
110.000no
124.583yes
133.190yes

表6

五种徘徊行为识别模型的识别结果"

ModelVideosNumber of VideosNumber of WanderFPFNPrecisionRecallF1 Score
Stay Timerun1600087.50%100%0.933
walk16020
wander161600
Curvature Information Entropyrun16000100%87.50%0.933
walk16000
wander161602
Differential Displacement Differencerun16000100%87.50%0.933
walk16000
wander161602
Classification Functionrun16000100%100%1.000
walk16000
wander161600
Outlier Detectionrun16000100%93.75%0.968
walk16000
wander161601

表7

不同轨迹特征对SVM识别结果的影响"

EκSκHκRTPrecisionRecallF1 Score
√√√√√100%100%1.000
√√√100%93.75%0.968
√100%100%1.000
√100%100%1.000

表8

三种分类函数的识别结果"

ModelVideosNumber of VideosNumber of WanderFPFNPrecisionRecallF1 Score
SVMrun16000100%100%1.000
walk16000
wander161600
CARTrun16000100%100%1.000
walk16000
wander161600
GBDTrun16000100%100%1.000
walk16000
wander161600
1 Sultani W，Chen C，Shah M. Real?world anomaly detection in surveillance videos∥Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City，UT，USA：IEEE，2018：6479-6488.
2 王新文，谢林柏，彭力. 跌倒异常行为的双重残差网络识别方法. 计算机科学与探索，2020，14(9)：1580-1589. （Wang X W，Xie L B，Peng L. Double residual network recognition method for falling abnormal behavior. Journal of Frontiers of Computer
Science & Technology，2020，14(9)：1580-1589.
3 罗凡波，王平，梁思源等. 基于深度学习与稀疏光流的人群异常行为识别. 计算机工程，2020，46(4)：287-293，300.
Luo F B，Wang P，Liang S Y，et al. Crowd abnormal behavior recognition based on deep learning and sparse optical flow. Computer Engineering，2020，46(4)：287-293，300.
4 王昆仑，刘文璨，何小海等. 一种用于异常行为检测的运动特征描述子. 计算机科学，2020，47(4)：119-124.
Wang K L，Liu W C，He X H，et al. Motion feature descriptor for abnormal behavior detection. Computer Science，2020，47(4)：119-124.
5 Hu X，Dai J，Huang Y P，et al. A weakly supervised framework for abnormal behavior detection and localization in crowded scenes. Neurocomputing，2020(383)：270-281.
6 高志辉. 监狱犯人越界检测算法研究. 硕士学位论文. 长沙：国防科学技术大学，2016. （Gao Z H. Research on dectecting the prisoners' crossing?border in Jail. Master Dissertation. Changsha：National
University of Defense Technology，2016.
7 石教坤. HOG+SVM在火车站站台乘客越界预警软件中的应用. 硕士学位论文. 重庆：重庆师范大学，2019.
Shi J K. Application of HOG+SVM in passenger crossing warning software for railway station platform. Master Dissertation. Chongqing：Chongqing Normal University，2019.
8 陈岗. 治安监控中基于计算机视觉的异常行为检测技术研究. 硕士学位论文. 上海：上海交通大学，2015.
Chen G. The research of abnormal behavior detection based on computer vision in the security monitoring. Master Dissertation. Shanghai：Shanghai Jiao Tong University，2015.
9 Feng F，Liu S，Pan Y，et al. Crowd anomaly scattering detection based on information entropy∥The 15th Chinese Conference on Image and Graphics
Technologies and Applications. Springer Berlin
Heidelberg，2020：193-205.
10 Feng F，Huang H，Zhu H，et al. Abnormal crowd behavior detection based on movement trajectory∥The 15th Chinese Conference on Image and Graphics Technologies and Applications. Springer Berlin Heidelberg，2020：103-113.
11 Bird N D，Masoud O，Papanikolopoulos N P，et al. Detection of loitering individuals in public transportation areas. IEEE Transactions on Intelligent Transportation Systems，2005，6(2)：167-177.
12 Ard? H，?str?m K. Multi sensor loitering detection using online Viterbi∥10th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance. Rio de Janeiro,Brazil：IEEE，2007：87-94.
13 Arroyo R，Yebes J J，Bergasa L M，et al. Expert video?surveillance system for real?time detection of suspicious behaviors in shopping malls. Expert Systems with Applications，2015，42(21)：7991-8005.
14 钟俊洪. 基于深度学习的目标跟踪及异常徘徊检测. 硕士学位论文. 广州：华南理工大学，2015. （Zhong J H. Object tracking and abnormal loitering detection based on deep learning. Master Disser?
tation. Guangzhou：South China University of
Technology，2015.
15 刘强，罗斌，翟素兰等. 基于离散曲率熵的徘徊行为检测. 计算机工程与应用，2013，49(18)：164-166，216.
Liu Q，Luo B，Zhai S L，et al. Loitering detection based on discrete curvature entropy. Computer Engineering and Applications，2013，49(18)：164-166，216.
16 朱梦哲，冯瑞. 基于三维模型的行人徘徊行为检测算法. 计算机应用与软件，2017，34(4)：149-156.
Zhu M Z，Feng R. Detection algorithm of pedestrian wandering based on 3D model. Computer Applications and Software，2017，34(4)：149-156.
17 董坤. 视频监控中运动人体检测与异常行为分析研究. 硕士学位论文. 南京：南京邮电大学，2013. （Dong K. Moving human detection and abnormal
behavior analysis in video surveillance. Master
Dissertation. Nanjing：Nanjing University of Posts and Telecommunications，2013.
18 Da Silva C L，Petry L M，Bogorny V. A survey and comparison of trajectory classification methods∥2019 8th Brazilian Conference on Intelligent Systems (BRACIS). Salvador，Brazil：IEEE，2019：788-793.
19 Wang W G，Chu X M，Jiang Z L，et al. Classification of ship trajectories by using naive bayesian algorithm∥2019 5th International Conference on Transportation Information and Safety. Liverpool，UK：IEEE，2019：466-470.
20 Mli'ch J，Chmelar P. Trajectory classification based on hidden Markov models∥The 18th International Conference on Computer Graphics and Vision. Moscow，Rissia：GraphiCon，2008：101-105.
21 Huang P，Lu J L. Learning trajectory patterns via canonical correlation analysis. International Journal of Cognitive Informatics and Natural Intelligence，2021，15(2)：1-17.
22 Liu F T，Ting K M，Zhou Z H. Isolation forest∥2008 8th IEEE International Conference on Data Mining. Pisa，Italy：IEEE，2008：413-422.
23 Li B，Wu W，Wang Q，et al. SiamRPN++：Evolution of Siamese visual tracking with very deep networks∥Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach，CA，USA：IEEE，2019：4277-4286.
24 Cortes C，Vapnik V. Support?vector networks. Machine Learning，1995，20(3)：273-297.
25 Breiman L，Friedman J H，Stone C J，et al. Classification and regression trees. New York：CRC Press，1984.
26 Friedman J H. Greedy function approximation：A gradient boosting machine. The Annals of Statistics，2001，29(5)：1189-1232.
27 Zhan Y F，Wang C Y，Wang X G，et al. FairMOT：On the fairness of detection and Re?identification in multiple object tracking. 2020,arXiv:.
[1] 薛峰, 李凡, 李爽, 李华锋. 基于域分离和对抗学习的跨域行人重识别[J]. 南京大学学报(自然科学版), 2021, 57(5): 715-723.
[2] 贾霄, 郭顺心, 赵红. 基于图像属性的零样本分类方法综述[J]. 南京大学学报(自然科学版), 2021, 57(4): 531-543.
[3] 吴礼福, 徐行. 融合韵律与动态倒谱特征的语音疲劳度检测[J]. 南京大学学报(自然科学版), 2021, 57(4): 709-714.
[4] 颜志良, 丰智鹏, 刘丹, 王会青. 一种混合深度神经网络的赖氨酸乙酰化位点预测方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 627-640.
[5] 王发旺, 陈睿, 伏云发. 基于DBN和RF的跨被试情绪识别研究[J]. 南京大学学报(自然科学版), 2021, 57(4): 617-626.
[6] 段建设, 崔超然, 宋广乐, 马乐乐, 马玉玲, 尹义龙. 基于多尺度注意力融合的知识追踪方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 591-598.
[7] 杜淑颖, 侯海薇, 丁世飞. 基于多层次特征的深度集成聚类算法[J]. 南京大学学报(自然科学版), 2021, 57(4): 575-581.
[8] 黄鹤, 吴琨, 宋京, 王会峰, 茹锋, 郭璐. 融合全局与区域大气光值图的暗通道图像去雾方法[J]. 南京大学学报(自然科学版), 2021, 57(4): 551-565.
[9] 乔宇鑫, 葛洪伟. 自适应样本加权的多视图聚类算法[J]. 南京大学学报(自然科学版), 2021, 57(4): 544-550.
[10] 王晓琳, 赵磊, 张维, 伏云发. 基于HHT和核函数选择的情绪特征提取与识别[J]. 南京大学学报(自然科学版), 2021, 57(3): 502-511.
[11] 史再峰, 王仲琦, 王宁, 岳玉含, 王子菊. 基于对抗式多任务学习的医学CT图像金属伪影去除方法[J]. 南京大学学报(自然科学版), 2021, 57(3): 482-492.
[12] 范习健, 杨绪兵, 张礼, 业巧林, 业宁. 一种融合视觉和听觉信息的双模态情感识别算法[J]. 南京大学学报(自然科学版), 2021, 57(2): 309-317.
[13] 罗金屯, 滕飞, 周亚波, 池茂儒, 张海波. 数据驱动的高速铁路轮轨作用力反演模型[J]. 南京大学学报(自然科学版), 2021, 57(2): 299-308.
[14] 邓涛, 陈红梅, 王丽珍. 并行环境下的空间模式匹配[J]. 南京大学学报(自然科学版), 2021, 57(2): 279-288.
[15] 梁丽娜, 姚睿, 迟文浩, 周勇, 赵佳琦. 基于空间感知与细化残差的视频运动目标分割[J]. 南京大学学报(自然科学版), 2021, 57(2): 245-254.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!

版权所有 © 2021 《南京大学学报(自然科学)》

地址：江苏省南京市鼓楼区汉口路22号，南京大学学报编辑部，210093

电话：025-83592704 , 传真：025-83592704        技术支持：北京玛格泰克科技发展有限公司