We describe an approach to describe moving pictures in terms of their structural properties for video editing, video indexing, and video coding. The description contains 2D shape, motion, spatial relation, and relative depth of each region. To obtain the description, we develop the incremental segmentation scheme which includes dynamic occlusion analysis to determine relative depths of several objects. The scheme has been designed along the analysis-by-synthesis" approach, and uses a sequence of images to estimate object boundaries and motion information successively/incrementally. The scheme consists of three components: motion estimation, prediction with dynamic occlusion analysis, and update of the segmentation results. By combining the information from extended (longer) image sequences, and also by treating the segmentation and dynamic occlusion analysis simultaneously, the scheme attempts to improve successively over time the accuracy of the object boundary and motion estimation.