A method based on clustering is presented for restoring and segmenting gray scale images. An optimum clustering obtained by a gradient method gives an image with gray scale values which vary smoothly in each segmented region. The method is also applied to restoration from sparsely sampled data.