In this paper, we present a novel method for vehicle detection based on the Faster R-CNN frame. We integrate MobileNet into Faster R-CNN structure. First, the MobileNet is used as the base network to generate the feature map. In order to retain the more information of vehicle objects, a fusion strategy is applied to multi-layer features to generate a fused feature map. The fused feature map is then shared by region proposal network (RPN) and Fast R-CNN. In the RPN system, we employ a novel dimension cluster method to predict the anchor sizes, instead of choosing the properties of anchors manually. Our detection method improves the detection accuracy and saves computation resources. The results show that our proposed method respectively achieves 85.21% and 91.16% on the mean average precision (mAP) for DIOR dataset and UA-DETRAC dataset, which are respectively 1.32% and 1.49% improvement than Faster R-CNN (ResNet152). Also, since less operations and parameters are required in the base network, our method costs the storage size of 42.52MB, which is far less than 214.89MB of Faster R-CNN(ResNet50).