We propose a sea surveillance system that automatically detects intruding objects in the sea. The difficulty with an automatic system is detecting objects such as moving boats while reducing false positives caused by some waves and reflections in the sea. A false positive is reporting an object which doesn't actually exist, while a false negative is a failure in detecting an intruding object. Firstly, we identify factors of false positives. Secondly, we propose a new surveillance system considering these factors. Our proposed system combines three detecting methods. The first method is detection of Differences between Surveillance images and Flapping Reference images (DSFR). The second method is detection of Contours from Averaging images (CA). The third method is Silhouette object Detection (SD). The combination of DSFR and CA detects various moving objects under normal light conditions, while SD detects objects under backlight conditions. Finally we apply our proposed method to actual situations. Our proposed method detected boats while reducing false positives effectively.