Robust yet efficient techniques for detecting and tracking targets in infrared (IR) images are a significant component of automatic target recognition (ATR) systems. In our previous works, we have proposed infrared target detection and tracking systems based on sparse representation method. The proposed infrared target detection and tracking algorithms are based on sparse representation and Bayesian probabilistic techniques, respectively. In this paper, we adopt Naïve Bayes Nearest Neighbor (NBNN) that is an extremely simple, efficient algorithm that requires no training phase. State-of-the-art image classification techniques need a comprehensive learning and training step (e.g., using Boosting, SVM, etc.) In contrast, non-parametric Nearest Neighbor based image classifiers need no training time and they also have other more advantageous properties. Results of tracking in infrared sequences demonstrated that our algorithm is robust to illumination changes, and the tracking algorithm is found to be suitable for real-time tracking of a moving target in infrared sequences and its performance was quite good.