The number of threats on the Internet is rapidly increasing, and anomaly detection has become of increasing importance. High-speed backbone traffic is particularly degraded, but their analysis is a complicated task due to the amount of data, the lack of payload data, the asymmetric routing and the use of sampling techniques. Most anomaly detection schemes focus on the statistical properties of network traffic and highlight anomalous traffic through their singularities. In this paper, we concentrate on unusual traffic distributions, which are easily identifiable in temporal-spatial space (e.g., time/address or port). We present an anomaly detection method that uses a pattern recognition technique to identify anomalies in pictures representing traffic. The main advantage of this method is its ability to detect attacks involving mice flows. We evaluate the parameter set and the effectiveness of this approach by analyzing six years of Internet traffic collected from a trans-Pacific link. We show several examples of detected anomalies and compare our results with those of two other methods. The comparison indicates that the only anomalies detected by the pattern-recognition-based method are mainly malicious traffic with a few packets.

Intrusion detection system (IDS) has played an important role as a device to defend our networks from cyber attacks. However, since it is unable to detect unknown attacks, i.e., 0-day attacks, the ultimate challenge in intrusion detection field is how we can exactly identify such an attack by an automated manner. Over the past few years, several studies on solving these problems have been made on anomaly detection using unsupervised learning techniques such as clustering, one-class support vector machine (SVM), etc. Although they enable one to construct intrusion detection models at low cost and effort, and have capability to detect unforeseen attacks, they still have mainly two problems in intrusion detection: a low detection rate and a high false positive rate. In this paper, we propose a new anomaly detection method based on clustering and multiple one-class SVM in order to improve the detection rate while maintaining a low false positive rate. We evaluated our method using KDD Cup 1999 data set. Evaluation results show that our approach outperforms the existing algorithms reported in the literature; especially in detection of unknown attacks.

Traffic volume anomalies refer to apparently abrupt changes in the time series of traffic volume, which can propagate through the network. Detecting and tracing these anomalies is a critical and difficult task for network operators. In this paper, we first propose a traffic decomposition method, which decomposes the traffic into three components: the trend component, the autoregressive (AR) component, and the noise component. A traffic volume anomaly is detected when the AR component is outside the prediction band for multiple links simultaneously. Then, the anomaly is traced using the projection of the detection result matrices for the observed links which are selected by a shortest-path-first algorithm. Finally, we validate our detection and tracing method by using the real traffic data from the third-generation Science Information Network (SINET3) and show the detected and traced results.

Traditional traffic analysis is can be performed online only when detection targets are well specified and are fairly primitive. Local processing at measurement point is discouraged as it would considerably affect major functionality of a network device. When traffic is analyzed at flow level, the notion of flow timeout generates differences in flow lifespan and impedes unbiased monitoring, where only n-top flows ordered by a certain metric are considered. This paper proposes an alternative manner of traffic analysis based on source IP aggregation. The method uses flows as basic building blocks but ignores timeouts, using short monitoring intervals instead. Multidimensional space of metrics obtained through IP aggregation, however, enhances capabilities of traffic analysis by facilitating detection of various anomalous conditions in traffic simultaneously.

We propose an algorithm for finding heavy hitters in terms of cardinality (the number of distinct items in a set) in massive traffic data using a small amount of memory. Examples of such cardinality heavy-hitters are hosts that send large numbers of flows, or hosts that communicate with large numbers of other hosts. Finding these hosts is crucial to the provision of good communication quality because they significantly affect the communications of other hosts via either malicious activities such as worm scans, spam distribution, or botnet control or normal activities such as being a member of a flash crowd or performing peer-to-peer (P2P) communication. To precisely determine the cardinality of a host we need tables of previously seen items for each host (e.g., flow tables for every host) and this may infeasible for a high-speed environment with a massive amount of traffic. In this paper, we use a cardinality estimation algorithm that does not require these tables but needs only a little information called the cardinality summary. This is made possible by relaxing the goal from exact counting to estimation of cardinality. In addition, we propose an algorithm that does not need to maintain the cardinality summary for each host, but only for partitioned addresses of a host. As a result, the required number of tables can be significantly decreased. We evaluated our algorithm using actual backbone traffic data to find the heavy-hitters in the number of flows and estimate the number of these flows. We found that while the accuracy degraded when estimating for hosts with few flows, the algorithm could accurately find the top-100 hosts in terms of the number of flows using a limited-sized memory. In addition, we found that the number of tables required to achieve a pre-defined accuracy increased logarithmically with respect to the total number of hosts, which indicates that our method is applicable for large traffic data for a very large number of hosts. We also introduce an application of our algorithm to anomaly detection. With actual traffic data, our method could successfully detect a sudden network scan.

This paper deals with the scattering of a TM plane wave from a periodic grating with single defect, of which position is known. The surface is perfectly conductive and made up with a periodic array of rectangular grooves and a defect where a groove is not formed. The scattered wave above grooves is written as a variation from the diffracted wave for the perfectly periodic case. Then, an integral equation for the scattering amplitude is obtained, which is solved numerically by use of truncation and the iteration method. The differential scattering cross section and the optical theorem are calculated in terms of the scattering amplitude and are illustrated in figures. It is found that incoherent Wood's anomaly appears at critical angles of scattering. The physical mechanisms of Wood's anomaly and incoherent Wood's anomaly are discussed in relation to the guided surface wave excited by the incident plane wave. It is concluded that incoherent Wood's anomaly is caused by the diffraction of the guided surface wave.

Existing anomaly intrusion detection that monitors system calls has two problems: vast false positives and lack of risk information on detection. In order to solve the two problems, we propose an intrusion detection method called "Callchains." Callchains reduces the false positives of existing anomaly intrusion detection by restricting monitoring to the activities with process capabilities prescribed by POSIX 1003.1e. Additionally, Callchains provides an administrator information of used POSIX capabilities in sytem call execution as an indicator of risk. This paper shows Callchains' design, its implementation, and experimental results comparing Callchains with existing approaches.

We propose a model for constructing a multilayered boundary in an information system to defend against intrusive anomalies by correlating a number of parametric anomaly detectors. The model formulation is based on two observations. First, anomaly detectors differ in their detection coverage or blind spots. Second, operating environments of the anomaly detectors reveal different information about system anomalies. The correlation among observation-specific anomaly detectors is first formulated as a Partially Observable Markov Decision Process, and then a policy-gradient reinforcement learning algorithm is developed for an optimal cooperation search, with the practical objectives being broader overall detection coverage and fewer false alerts. A host-based experimental scenario is developed to illustrate the principle of the model and to demonstrate its performance.

This paper deals with the scattering of a TM plane wave from a perfectly conductive sinusoidal surface with finite extent. For comparison, however, we briefly discuss the diffraction by the sinusoidal surface with infinite extent, where we use the concept of the total diffraction cross section per unit surface introduced previously. To solve a case where the sinusoidal corrugation width is much wider than wave length, we propose an undersampling approximation as a new numerical technique. For a small rough case, the total scattering cross section is calculated against the angle of incidence for several different corrugation widths. Then we find remarkable results, which are roughly summarized as follows. When the angle of incidence is apparently different from critical angles and diffraction beams are all scattered into non-grazing directions, the total scattering cross section increases proportional to the corrugation width and hence the total scattering cross section per unit surface (the ratio of the total scattering cross section to the corrugation width) becomes almost constant, which is nearly equal to the total diffraction cross section per unit surface in case of the sinusoidal surface with infinite extent. When the angle of incidence is critical and one of the diffraction beams is scattered into a grazing direction, the total scattering cross section per unit surface strongly depends on the corrugation width and approximately approaches to the total diffraction cross section per unit surface as the corrugation width gets wide.

Large-throughput anomaly prevention mechanism in the upstream side of high-speed (over 10-Gbps) networks is required to prevent various anomalies such as distributed denial of service (DDoS) from causing various network problems. This mechanism requests the processors achieving not only high-speed response for analyzing many packets in a short time but also the flexibility to update the anomaly prevention algorithm. In this research, I assumed a dynamic reconfigurable processor (DRP) was most effective in achieving this anomaly prevention mechanism, for processors used in nodes with the mechanism, and I designed an anomaly prevention mechanism using DRPs. The mechanism can shorten anomaly prevention time in high-speed (10 Gbps) lines using an all-packet analysis. Through a simulation, I achieved the goal of the mechanism achieving a throughput of 83-M packets per second using three DRPs (432 execution elements used). Moreover, with the prototype, it was confirmed that the proposed mechanism prevented anomalies in a short time (constant 0.01 second), which was 3000 times faster than that of a legacy mechanism using a packet sampling method. I also proposed integrated prevention, which was able to reduce the number of execution elements comprising anomaly prevention algorithm against various kinds of anomalies. It was achieved with a simulation that the proposed integrated prevention against three kinds of anomalies (DDoS, worm, and peer to peer (P2P)) reduced the number of execution elements by 24% compared to legacy prevention. In addition, non-stop update was proposed to maintain throughput when updating an anomaly prevention algorithm without packet loss. It was confirmed with a simulation that there was enough time for non-stop update in 10 Gbps 4 lines.

Security protocols flaws represent a substantial portion of security exposures of data networks. In order to evaluate security protocols against any attack, formal methods are equipped with a number of techniques. Unfortunately, formal methods are applicable for static state only, and don't guarantee detecting all possible flaws. Therefore, formal methods should be complemented with dynamic protection. Anomaly detection systems are very suitable for security protocols environments as dynamic activities protectors. This paper presents an intrusion detection system that uses a number of different anomaly detection techniques to detect attacks against security protocols.

Machine learning and data mining algorithms are increasingly being used in the intrusion detection systems (IDS), but their performances are laggard to some extent especially applied in network based intrusion detection: the larger load of network traffic monitoring requires more efficient algorithm in practice. In this paper, we propose and design an anomaly intrusion detection (AID) system based on the vector quantization (VQ) which is widely used for data compression and high-dimension multimedia data index. The design procedure optimizes the performance of intrusion detection by jointly accounting for accurate usage profile modeling by the VQ codebook and fast similarity measures between feature vectors to reduce the computational cost. The former is just the key of getting high detection rate and the later is the footstone of guaranteeing efficiency and real-time style of intrusion detection. Experiment comparisons to other related researches show that the performance of intrusion detection is improved greatly.

This paper proposes "Multipath Control and Proactive Control" to realize a robust QoS control system for mobile multimedia communication in an IP-based cellular network. In this network, all kinds of traffic will share the same backbone network. This requires a QoS system that differentiates services according to the required quality. Though DiffServ is thought to be a promising technique for achieving QoS, an effective path control scheme and a technique that is suitable enough for rapid traffic changes are not yet available. Our solution is multipath control using linear optimization combined with proactive control using traffic anomaly detection. Simulation results show that multipath control and proactive control improve system performance in terms of throughput and packet loss when rapid traffic change takes place.

Masqueraders who impersonate other users pose serious threat to computer security. Unfortunately, firewalls or misuse-based intrusion detection systems are generally ineffective in detecting masqueraders. Anomaly detection techniques have been proposed as a complementary approach to overcome such limitations. However, they are not accurate enough in detection, and the rate of false alarm is too high for the technique to be applied in practice. For example, recent empirical studies on masquerade detection using UNIX commands found the accuracy to be below 70%. In this research, we performed a comparative study to investigate the effectiveness of SVM (Support Vector Machine) technique using the same data set and configuration reported in the previous experiments. In order to improve accuracy of masquerade detection, we used command frequencies in sliding windows as feature sets. In addition, we chose to ignore commands commonly used by all the users and introduce the concept of voting engine. Though still imperfect, we were able to improve the accuracy of masquerade detection to 80.1% and 94.8%, whereas previous studies reported accuracy of 69.3% and 62.8% in the same configurations. This study convincingly demonstrates that SVM is useful as an anomaly detection technique and that there are several advantages SVM offers as a tool to detect masqueraders.

For detecting the anomalous behavior of a user effectively, most researches have concentrated on statistical techniques. However, since statistical techniques mainly analyze the average behavior of a user's activities, some anomalies can be detected inaccurately. In addition, it is difficult to model intermittent activities performed periodically. In order to model the normal behavior of a user closely, a set of various features can be employed. Given an activity of a user, the values of those features that are related to the activity represent the behavior of the activity. Furthermore, activities performed in a session of a user can be regarded as a semantically atomic transaction. Although it is possible to apply clustering technique to these values to extract the normal behavior of a user, most of conventional clustering algorithms do not consider any transactional boundary in a data set. In this paper, a transaction-based clustering algorithm for modeling the normal behavior of a user is proposed. Based on the activities of the past transactions, a set of clusters for each feature can be found to represent the normal behavior of a user as a concise profile. As a result, any anomalous behavior in an online transaction of the user can be effectively detected based on the profile of the user.