Abstract: In the case of linear value function approximated reinforcement learning，it is meaningful to do research on off－policy algorithms to get a better balance on explore－exploit trade－off.In recent years，Sutton et al proposed off－policy gradient temporal difference learning algorithms，which possess good properties in speed and convergence.The main contribution of this paper is proposing a generalized oblique projection framework，which utilizes the weighted sum of two projections，so as to derive off－policy temporal difference learning algorithms with a generalized oblique projection.To derive a good algorithm，methods of Temporal Difference fixed－point and Bellman residual are widely used.However，they can be viewed as oblique projections of the true value function，therefore these two projections are not optimal.This paper starts from understanding different algorithms from the view of projection，and proposes a kind of method to obtain a better projection，by generalizing a kind of oblique projection as the weighted sum of projections of these two projections.Further，to obtain convergent algorithms for off－policy setting，we extend the generalized projection based on the norm of expected TD update，and generates two kinds of objective functions.Employing the approach of stochastic gradient method，this paper derives two convergent off－policy linear residual Temporal Difference algorithms.To theoretically prove the convergence of our algorithms，we use the method of ordinary－differential－equation approach，which views iterations as ordinary differential equations，and tries to guarantee the stability of them.Experimental results on Baird’s off－policy counterexample demonstrate the effectiveness of the proposed algorithms.Discussions on performance for different weight value parameters are given at last.