This paper proposes a novel method to segment the object of a specific class based on a rough detection window (such as Deformable Part Model (DPM) in this paper), which is robust to the positions of the bounding boxes. In our method, the DPM is first used to generate the root and part windows of the object. Then a set of object part candidates are generated by randomly sampling windows around the root window. Furthermore, an undirected graph (the minimum spanning tree) is constructed to describe the spatial relationships between the part windows. Finally, the object is segmented by grouping the part proposals on the undirected graph, which is formulated as an energy function minimization problem. A novel energy function consisting of the data term and the smoothness term is designed to characterize the combination of the part proposals, which is globally minimized by the dynamic programming on a tree. Our experimental results on challenging dataset demonstrate the effectiveness of the proposed method.