An efficient vision-based vehicle detection and tracking system is presented in this paper. One of the most vital stages of the driver assistance system is the detection and tracking. This will help the drivers to fully understand the road conditions. Vehicle detection from a moving camera is difficult compared to a stationary camera. In this method, we used the practical implementation of a vehicle detection based on Viola-Jones algorithm. To track the detected vehicle, we are using a nonlinear filter such as Unscented Kalman Filter. The simulation and experimental results indicate that real-time nonlinear tracking problem can be resolved by using this nonlinear Unscented Kalman Filter.