Most existing outlier detection algorithms only utilized location of trajectory points and neglected some important factors such as speed, acceleration, and corner. To address this problem, we present a Trajectory Outlier Detection algorithm based on Multi-Factors (TODMF). TODMF is improved in terms of distance-based outlier detection algorithms. It combines multi-factors into outlier detection to find more meaningful trajectory outliers. We resort to Canonical Correlation Analysis (CCA) to optimize the number of factors when determining what factors will be considered. Finally, the experiments with real trajectory data sets show that TODMF performs efficiently and effectively when applied to the problem of trajectory outlier detection.