The support vector machine has received wide acceptance for its high generalization ability in real world classification applications. But a drawback is that it uniquely classifies each pattern to one class or none. This is not appropriate to be applied in classification problem involves overlapping patterns. In this paper, a novel multi-model classifier (DR-SVM) which combines SVM classifier with kNN algorithm under rough set technique is proposed. Instead of classifying the patterns directly, patterns lying in the overlapped region are extracted firstly. Then, upper and lower approximations of each class are defined on the basis of rough set technique. The classification operation is carried out on these new sets. Simulation results on synthetic data set and benchmark data sets indicate that, compared with conventional classifiers, more reasonable and accurate information about the pattern's category could be obtained by use of DR-SVM.