In this paper, we propose a Pyramid Predictive Attention Network (PPAN) for medical image segmentation. In the medical field, the size of dataset generally restricts the performance of deep CNN and deploying the trained network with gross parameters into the terminal device with limited memory is an expectation. Our team aims to the future home medical diagnosis and search for lightweight medical image segmentation network. Therefore, we designed PPAN mainly made of Xception blocks which are modified from DeepLab v3+ and consist of separable depthwise convolutions to speed up the computation and reduce the parameters. Meanwhile, by utilizing pyramid predictions from each dimension stage will guide the network more accessible to optimize the training process towards the final segmentation target without degrading the performance. IoU metric is used for the evaluation on the test dataset. We compared our designed network performance with the current state of the art segmentation networks on our RGB tongue dataset which was captured by the developed TIAS system for tongue diagnosis. Our designed network reduced 80 percentage parameters compared to the most widely used U-Net in medical image segmentation and achieved similar or better performance. Any terminal with limited storage which is needed a segment of RGB image can refer to our designed PPAN.