In this paper, a numerical method for solving stochastic optimal control problem by using Markov chain approximation method has presented. The basic idea of the Markov chain approximation method is to approximate the original controlled process by an appropriate controlled Markov chain on a finite state space. Also, we need to approximate the original cost function by one which is appropriate for the approximating chain. These approximations should be chosen such that a good numerical approximation to the associated optimal control problem can be obtained, which means the conditional mean and covariance of the changes in state of the chain are proportional to the local mean drift and covariance for the original process. The finite difference approximations are used to the construction of locally consistent approximating Markov chain, the coefficients of the resulting discrete equation can serve as the desired transition probabilities and interpolation interval. The convergence is analogous to the convergence of a sequence of finite difference or finite element approximations to an original problem as the approximation interval goes to zero. Finally, we propose an iterative algorithm for solving stochastic optimal control and efficiency of the proposed algorithm is illustrated by an example.