Research into closed-form Gaussian sum smoother has provided an attractive approach for tracking in clutter, joint detection and tracking (in clutter), and multiple target tracking (in clutter) via the probability hypothesis density (PHD). However, Gaussian sum smoother with nonlinear target model has particular nonlinear expressions in the backward smoothed density that are different from the other filters and smoothers. In order to extend the closed-form solution of linear Gaussian sum smoother to nonlinear model, two closed-form approximations for nonlinear Gaussian sum smoother are proposed, which use Gaussian particle approximation and unscented transformation approximation, separately. Since the estimated target number of PHD smoother is not stable, a heuristic approximation method is added. At last, the Bernoulli smoother and PHD smoother are simulated using Gaussian particle approximation and unscented transformation approximation, and simulation results show that the two proposed algorithms can obtain smoothed tracks with nonlinear models, and have better performance than filter.