Abstract: Incremental support vector machine model (ISVM) joins a sample or a batch of samples to learn in each cycle, and then the problem can be reduced from large-scale to a series of sub issues. Therefore, ISVM can improve the efficiency of support vector machine (SVM) to deal with large scale data. However, using traditional support vector machine (TISVM), the convergence speed, efficiency and the eventual generalization ability may be decreased due to the incorrect selection of the incremental samples. To solve the problem, an ISVM approach ( incremental support vector machine based on the probability density distribution, namely PISVM) is proposed through choosing those incremental training samples including much important classification information based on probability density distribution. Using the approach can make the classifier get to the optimal hyper lane at the fastest speed. In order to verify the validity of the proposed approach, some experiments are done using the three approaches: the PISVM approach, the TISVM method and the minimum distance classifier approach. The experiment results on UCI data set demonstrate that the proposed PISVM can obtain high learning efficiency with good generalization performance simultaneously.