Since the detection of optical flow (two-dimensional motion field on an image) from image sequences is essentially an ill-posed problem, most of the conventional methods use a smoothness constraint for optical flow heuristically and detect reasonable optical flow. However, little discussion exists regarding the degree of smoothness. Furthermore, to recover the relative three-dimensional motion and depth between a camera and a rigid object, in general at first, the optical flow is detected without a rigid motion constraint, and next, the motion and depth are estimated using the detected optical flow. Rigorously speaking, the optical flow should be detected with such a constraint, and consequently three-dimensional motion and depth should be determined. To solve these problems, in this paper, we apply a parametric model to an optical flow, and construct an estimation algorithm based on this model.