A real-time road-direction point detection model is developed based on convolutional neural network architecture which can adapt to complex environment. Firstly, the concept of road-direction point is defined for either single road or crossroad. For single road, the predicted road-direction point can serve as a guiding point for a self-driving vehicle to go ahead. In the situation of crossroad, multiple road-direction points can also be detected which will help this vehicle to make a choice from possible directions. Meanwhile, different types of road surface can be classified by this model for both paved roads and unpaved roads. This information will be beneficial for a self-driving vehicle to speed up or slow down according to various road conditions. Finally, the performance of this model is evaluated on different platforms including Jetson TX1. The processing speed can reach 12 FPS on this portable embedded system so that it provides an effective and economic solution of road-direction estimation in the applications of autonomous navigation.