Tea sprouts segmentation via machine vision is the core technology of tea automatic picking. A novel method for Tea Sprouts Segmentation based on improved deep convolutional encoder-decoder Network (TS-SegNet) is proposed in this paper. In order to increase the segmentation accuracy and stability, the improvement is carried out by a contrastive-center loss function and skip connections. Therefore, the intra-class compactness and inter-class separability are comprehensively utilized, and the TS-SegNet can obtain more discriminative tea sprouts features. The experimental results indicate that the proposed method leads to good segmentation results, and the segmented tea sprouts are almost coincident with the ground truth.