南京大学学报(自然科学版) ›› 2022, Vol. 58 ›› Issue (6): 1050–1058.doi: 10.13232/j.cnki.jnju.2022.06.013

• • 上一篇    

双重结构的最小二乘回归子空间聚类算法

卢桂馥(), 汤荣, 姚亮   

  1. 安徽工程大学计算机与信息学院,芜湖,241000
  • 收稿日期:2022-07-04 出版日期:2022-11-30 发布日期:2022-12-14
  • 通讯作者: 卢桂馥 E-mail:lu-guifu@ahpu.edu.cn
  • 基金资助:
    国家自然科学基金(61976005);安徽省自然科学基金(1908085MF183)

Double structure least squares regression subspace clustering

Guifu Lu(), Rong Tang, Liang Yao   

  1. School of Computer and Information,AnHui Polytechnic University, Wuhu,241000,China
  • Received:2022-07-04 Online:2022-11-30 Published:2022-12-14
  • Contact: Guifu Lu E-mail:lu-guifu@ahpu.edu.cn

摘要:

最小二乘回归(Least Square Regression,LSR)算法是一种流行的子空间聚类方法,在处理计算机视觉和机器学习的相关问题中的应用十分普遍.然而,当数据含有噪声时,其求得的亲和矩阵不是块对角化的,还存在一定的噪声,这使亲和矩阵不够鲁棒可靠,因而降低了算法的聚类性能.为了解决以上不足,提出一种双重结构的最小二乘回归子空间聚类算法(Double Structure Least Squares Regression Subspace Clustering,DSLSR).首先对原始数据实施LSR算法,由于其生成的亲和矩阵往往不是块对角矩阵且含有噪声,需要对求得的亲和矩阵再次实施LSR算法来去除亲和矩阵中的噪声,使亲和矩阵更干净可靠,从而提升算法的聚类性能.最后,把两次LSR过程纳入一个统一的算法框架,设计一个统一的目标函数.此外,还采取了增广拉格朗日乘子方法对目标函数进行优化求解.在一些数据集上的实验证实,DSLSR算法比现有算法的性能更卓越.

关键词: 子空间聚类, 目标函数, 最小二乘回归, 亲和矩阵

Abstract:

The Least Square Regression (LSR) algorithm is a popular subspace clustering approach,which is widely used in dealing with problems related to computer vision and machine learning. However,when the data is noisy,the affinity matrix is not block diagonal with some noise,which makes the affinity matrix not robust enough. To resolve the above issues,this paper researches a double structure least squares regression subspace clustering (DSLSR). In DSLSR,the LSR algorithm is first conducted on the original data. Because the original data contains noise,the generated affinity matrix is often not block diagonal matrix and contains noise. Therefore,it is necessary to implement LSR again on the generated affinity matrix to remove the noise and make the affinity matrix cleaner and more reliable,which can improve the clustering performance of the algorithm. Finally,the two LSR processes are incorporated into a unified algorithm framework,and a unified objective function is designed. In addition,a useful approach based on augmented Lagrange multiplier is employed to optimize the objective function. Experiments on some datasets exhibit that the DSLSR acquires better property than existing algorithms.

Key words: subspace clustering, objective function, least squares regression, affinity matrix

中图分类号: 

  • TP391

图1

ORL的代表性图片"

图2

Yale的代表性图片"

图3

Ceu的代表性图片"

图4

不同算法在ORL数据集上的亲和矩阵可视化"

表1

不同数据集上的算法性能比较"

数据集方法ACCNMIF⁃score
ORL_32×32K⁃means0.44420.64720.3294
SSC0.67840.81730.5433
LSR0.66310.81440.5348
LRR0.60470.77930.4781
CAN0.62500.79260.4881
SSR0.70260.83720.5819
DSLSR0.71330.84090.6012
Yale_32×32K⁃means0.44970.53230.3160
SSC0.53190.56390.3706
LSR0.52030.54540.3434
LRR0.51270.52800.3204
CAN0.47640.51770.3034
SSR0.49170.53300.3261
DSLSR0.58180.59710.4179
CeuK⁃means0.27020.56660.1693
SSC0.94400.98690.9453
LSR0.96110.99260.9627
LRR0.98380.99680.9842
CAN0.65010.87870.6086
SSR0.94680.99010.9523
DSLSR0.97850.99570.9782

图5

各种算法的t?SNE算法特征图比较"

图6

在ORL数据库上参数λ1和λ2对DSLSR算法性能的影响"

图7

DSLSR在三种数据集上的收敛曲线"

1 Cai T T, Shen X T. High?dimensional data analysis. Singapore:World Scientific,2011:320.
2 Zhou T, Zhang C Q, Peng X,et al. Dual shared?specific multiview subspace clustering. IEEE Transactions on Cybernetics202050(8):3517-3530.
3 Lauer F, Schn?rr C. Spectral clustering of linear subspaces for motion segmentation∥2009 IEEE 12th International Conference on Computer Vision. Kyoto,Japan:IEEE,2009:678-685.
4 Zheng Q H, Zhu J H, Li Z Y,et al. Feature concatenation multi?view subspace clustering. Neurocomputing2020(379):89-102.
5 Yang J F, Liang J, Wang K,et al. Subspace clustering via good neighbors. IEEE Transactions on Pattern Analysis and Machine Intelligence202042(6):1537-1544.
6 Zhang C Q, Fu H Z, Hu Q H,et al. Generalized latent multi?view subspace clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence202042(1):86-99.
7 Bradley P S, Mangasarian O L. K?plane clustering. Journal of Global Optimization200016(1):23-32.
8 Zhang T, Szlam A, Lerman G. Median K?flats for hybrid linear modeling with many outliers∥2009 IEEE 12th International Conference on Computer Vision Workshops. Kyoto,Japan:IEEE,2009:234-241.
9 Costeira J P, Kanade T. A multibody factorization method for independently moving objects. International Journal of Computer Vision199829(3):159-179.
10 Vidal R, Ma Y, Sastry S. Generalized principal component analysis (GPCA). IEEE Transactions on Pattern Analysis and Machine Intelligence200527(12):1945-1959.
11 Elhamifar E, Vidal R. Sparse subspace clustering∥IEEE Conference on Computer Vision and Pattern Recognition. Miami,FL,USA:IEEE,2009:2790-2797.
12 Chen G L, Lerman G. Spectral Curvature Clustering (SCC). International Journal of Computer Vision200981(3):317-330.
13 Tipping M E, Bishop C M. Mixtures of probabilistic principal component analyzers. Neural Computation199911(2):443-482.
14 Ma Y, Derksen H, Hong W,et al. Segmentation of multivariate mixed data via lossy data coding and compression. IEEE Transactions on Pattern Analysis and Machine Intelligence200729(9):1546-1562.
15 Liu G C, Lin Z C, Yan S C,et al. Robust recovery of subspace structures by low?rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence201335(1):171-184.
16 Elhamifar E, Vidal R. Sparse subspace clustering:Algorithm,theory,and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence201335(11):2765-2781.
17 Lu C Y, Min H, Zhao Z Q,et al. Robust and efficient subspace segmentation via least squares regression∥The 12th European Conference on Computer Vision. Florence,Italy:Springer,2012:347-360.
18 李海洋,王恒远. 基于TL1范数约束的子空间聚类方法. 电子与信息学报201739(10):2428-2436.
Li H Y, Wang H Y. Subspace clustering method based on TL1 norm constraints. Journal of Electronics & Information Technology201739(10):2428-2436.
19 李波,卢春园,冷成财,等. 基于局部图拉普拉斯约束的鲁棒低秩表示聚类方法. 自动化学报201541(11):1971-1980.
Li B, Lu C Y, Leng C C,et al. Robust low rank subspace clustering based on local graph laplace constraint. Acta Automatica Sinica201541(11):1971-1980.
20 Zheng M, Bu J J, Chen C,et al. Graph regularized sparse coding for image representation. IEEE Transactions on Image Processing201120(5):1327-1336.
21 Long M S, Ding G G, Wang J M,et al. Transfer sparse coding for robust image representation∥2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland,OR,USA:IEEE,2013:407-414.
22 Nie F P, Wang X Q, Huang H. Clustering and projected clustering with adaptive neighbors∥Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York,NY,USA:Association for Computing Machinery,2014:977-986.
23 Xu J, Yu M Y, Shao L,et al. Scaled simplex representation for subspace clustering. IEEE Transactions on Cybernetics202151(3):1493-1505.
24 Lin Z C, Liu R S, Su Z X. Linearized alternating direction method with adaptive penalty for low?rank representation∥Proceedings of the 24th International Conference on Neural Information Processing Systems. Granada,Spain:Curran Associates Inc.,2011:612-620.
25 Yin M, Gao J B, Lin Z C,et al. Dual graph regularized latent low?rank representation for subspace clustering. IEEE Transactions on Image Processing201524(12):4918-4933.
26 Bartels R H, Stewart G W. Solution of the matrix equation A X + X B = C F 4 . Communications of the ACM197215(9):820-826.
27 Kuybeda O, Frank G A, Bartesaghi A,et al. A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo?electron tomography. Journal of Structural Biology2013181(2):116-127.
28 Wilkin G A, Huang X Z. K?means clustering algorithms:Implementation and comparison∥Second International Multi?Symposiums on Computer and Computational Sciences. Iowa City,IA,USA,IEEE,2007:133-136.
[1] 夏菁, 丁世飞. 基于低秩稀疏约束的自权重多视角子空间聚类[J]. 南京大学学报(自然科学版), 2020, 56(6): 862-869.
[2] 王丽娟,丁世飞,丁玲. 基于迁移学习的软子空间聚类算法[J]. 南京大学学报(自然科学版), 2020, 56(4): 515-523.
[3] 帅 惠, 袁晓彤, 刘青山. 基于L0约束的稀疏子空间聚类[J]. 南京大学学报(自然科学版), 2018, 54(1): 23-.
[4]  严丽宇1,魏 巍1,2*,郭鑫垚1,崔军彪1.  一种基于带核随机子空间的聚类集成算法[J]. 南京大学学报(自然科学版), 2017, 53(6): 1033-.
[5] 刘 波1, 王红军1*,成 聪2,杨 燕1. 基于属性最大间隔的子空间聚类[J]. 南京大学学报(自然科学版), 2014, 50(4): 482-.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!