Recently, multivariate time-series data has been generated in various environments, such as sensor networks and IoT, making anomaly detection in time-series data an essential research topic. Unsupervised learning anomaly detectors identify anomalies by training a model on normal data and producing high residuals for abnormal observations. However, a fundamental issue arises as anomalies do not consistently result in high residuals, necessitating a focus on the time-series patterns of residuals rather than individual residual sizes. In this paper, we present a novel framework comprising two serialized anomaly detectors: the first model calculates residuals as usual, while the second one evaluates the time-series pattern of the computed residuals to determine whether they are normal or abnormal. Experiments conducted on real-world time-series data demonstrate the effectiveness of our proposed framework.

Seen from the Internet Service Provider (ISP) side, network traffic monitoring is an indispensable part during network service provisioning, which facilitates maintaining the security and reliability of the communication networks. Among the numerous traffic conditions, we should pay extra attention to traffic anomaly, which significantly affects the network performance. With the advancement of Machine Learning (ML), data-driven traffic anomaly detection algorithms have established high reputation due to the high accuracy and generality. However, they are faced with challenges on inefficient traffic feature extraction and high computational complexity, especially when taking the evolving property of traffic process into consideration. In this paper, we proposed an online learning framework for traffic anomaly detection by embracing Gaussian Process (GP) and Sparse Representation (SR) in two steps: 1). To extract traffic features from past records, and better understand these features, we adopt GP with a special kernel, i.e., mixture of Gaussian in the spectral domain, which makes it possible to more accurately model the network traffic for improving the performance of traffic anomaly detection. 2). To combat noise and modeling error, observing the inherent self-similarity and periodicity properties of network traffic, we manually design a feature vector, based on which SR is adopted to perform robust binary classification. Finally, we demonstrate the superiority of the proposed framework in terms of detection accuracy through simulation.

With a rapidly escalating number of sophisticated cyber-attacks, protecting Internet of Things (IoT) networks against unauthorized activity is a major concern. The detection of malicious attack traffic is thus crucial for IoT security to prevent unwanted traffic. However, existing traditional malicious traffic detection systems which relied on supervised machine learning approach need a considerable number of benign and malware traffic samples to train the machine learning models. Moreover, in the cases of zero-day attacks, only a few labeled traffic samples are accessible for analysis. To deal with this, we propose a few-shot malicious IoT traffic detection system with a prototypical graph neural network. The proposed approach does not require prior knowledge of network payload binaries or network traffic signatures. The model is trained on labeled traffic data and tested to evaluate its ability to detect new types of attacks when only a few labeled traffic samples are available. The proposed detection system first categorizes the network traffic as a bidirectional flow and visualizes the binary traffic flow as a color image. A neural network is then applied to the visualized traffic to extract important features. After that, using the proposed few-shot graph neural network approach, the model is trained on different few-shot tasks to generalize it to new unseen attacks. The proposed model is evaluated on a network traffic dataset consisting of benign traffic and traffic corresponding to six types of attacks. The results revealed that our proposed model achieved an F1 score of 0.91 and 0.94 in 5-shot and 10-shot classification, respectively, and outperformed the baseline models.

In this letter, we propose a feature-based knowledge distillation scheme which transfers knowledge between intermediate blocks of teacher and student with flow-based architecture, specifically Normalizing flow in our implementation. In addition to the knowledge transfer scheme, we examine how configuration of the distillation positions impacts on the knowledge transfer performance. To evaluate the proposed ideas, we choose two knowledge distillation baseline models which are based on Normalizing flow on different domains: CS-Flow for anomaly detection and SRFlow-DA for super-resolution. A set of performance comparison to the baseline models with popular benchmark datasets shows promising results along with improved inference speed. The comparison includes performance analysis based on various configurations of the distillation positions in the proposed scheme.

In order to improve the anomaly detection efficiency of network traffic, firstly, the model is established for network flows based on complex networks. Aiming at the uncertainty and fuzziness between network traffic characteristics and network states, the deviation extent is measured from the normal network state using deviation interval uniformly, and the intuitionistic fuzzy sets (IFSs) are established for the various characteristics on the network model that the membership degree, non-membership degree and hesitation margin of the IFSs are used to quantify the ownership of values to be tested and the corresponding network state. Then, the knowledge measure (KM) is introduced into the intuitionistic fuzzy weighted geometry (IFWGω) to weight the results of IFSs corresponding to the same network state with different characteristics together to detect network anomaly comprehensively. Finally, experiments are carried out on different network traffic datasets to analyze the evaluation indicators of network characteristics by our method, and compare with other existing anomaly detection methods. The experimental results demonstrate that the changes of various network characteristics are inconsistent under abnormal attack, and the accuracy of anomaly detection results obtained by our method is higher, verifying our method has a better detection performance.

The issue of a low minority class identification rate caused by data imbalance in anomaly detection tasks is addressed by the proposal of a GAN-SR-based intrusion detection model for industrial control systems. First, to correct the imbalance of minority classes in the dataset, a generative adversarial network (GAN) processes the dataset to reconstruct new minority class training samples accordingly. Second, high-dimensional feature extraction is completed using stacked asymmetric depth self-encoder to address the issues of low reconstruction error and lengthy training times. After that, a random forest (RF) decision tree is built, and intrusion detection is carried out using the features that SNDAE retrieved. According to experimental validation on the UNSW-NB15, SWaT and Gas Pipeline datasets, the GAN-SR model outperforms SNDAE-SVM and SNDAE-KNN in terms of detection performance and stability.

In the Artificial Intelligence for IT Operations scenarios, KPI (Key Performance Indicator) is a very important operation and maintenance monitoring indicator, and research on KPI anomaly detection has also become a hot spot in recent years. Aiming at the problems of low detection efficiency and insufficient representation learning of existing methods, this paper proposes a fast clustering-based KPI anomaly detection method HCE-DWL. This paper firstly adopts the combination of hierarchical agglomerative clustering (HAC) and deep assignment based on CNN-Embedding (CE) to perform cluster analysis (that is HCE) on KPI data, so as to improve the clustering efficiency of KPI data, and then separately the centroid of each KPI cluster and its Transformed Outlier Scores (TOS) are given weights, and finally they are put into the LightGBM model for detection (the Double Weight LightGBM model, referred to as DWL). Through comparative experimental analysis, it is proved that the algorithm can effectively improve the efficiency and accuracy of KPI anomaly detection.

Previous studies on anomaly detection in videos have trained detectors in which reconstruction and prediction tasks are performed on normal data so that frames on which their task performance is low will be detected as anomalies during testing. This paper proposes a new approach that involves sorting video clips, by using a generative network structure. Our approach learns spatial contexts from appearances and temporal contexts from the order relationship of the frames. Experiments were conducted on four datasets, and we categorized the anomalous sequences by appearance and motion. Evaluations were conducted not only on each total dataset but also on each of the categories. Our method improved detection performance on both anomalies with different appearance and different motion from normality. Moreover, combining our approach with a prediction method produced improvements in precision at a high recall.

There exists a great demand for automatic anomaly detection in industrial world. The anomaly has been defined as a group of samples that rarely or never appears. Given a type of products, one has to collect numerous samples and train an anomaly detector. When one diverts a model trained with old types of products with sufficient inventory to the new type, one can detect anomalies of the new type before a production line is established. However, because of the definition of the anomaly, a typical anomaly detector considers the new type of products anomalous even if it is consistent with the standard. Given the above practical demand, this study propose a novel problem setting, few-shot anomaly detection, where an anomaly detector trained in source domains is adapted to a small set of target samples without full retraining. Then, we tackle this problem using a hierarchical probabilistic model based on deep learning. Our empirical results on toy and real-world datasets demonstrate that the proposed model detects anomalies in a small set of target samples successfully.

Predicting the malfunction timing of wind turbines is essential for maintaining the high profitability of the wind power generation business. Machine learning methods have been studied using condition monitoring system data, such as vibration data, and supervisory control and data acquisition (SCADA) data, to detect and predict anomalies in wind turbines automatically. Autoencoder-based techniques have attracted significant interest in the detection or prediction of anomalies through unsupervised learning, in which the anomaly pattern is unknown. Although autoencoder-based techniques have been proven to detect anomalies effectively using relatively stable SCADA data, they perform poorly in the case of deteriorated SCADA data. In this letter, we propose a power-curve filtering method, which is a preprocessing technique used before the application of an autoencoder-based technique, to mitigate the dirtiness of SCADA data and improve the prediction performance of wind turbine degradation. We have evaluated its performance using SCADA data obtained from a real wind-farm.

Since most sensor data depend on each other, time-series anomaly detection is one of practical applications of IoT devices. Such tasks are handled by Recurrent Neural Networks (RNNs) with a feedback structure, such as Long Short Term Memory. However, their learning phase based on Stochastic Gradient Descent (SGD) is computationally expensive for such edge devices. This issue is addressed by executing their learning on high-performance server machines, but it introduces a communication overhead and additional power consumption. On the other hand, Recursive Least-Squares Echo State Network (RLS-ESN) is a simple RNN that can be trained at low cost using the least-squares method rather than SGD. In this paper, we propose its area-efficient hardware implementation for edge devices and adapt it to human activity anomaly detection as an example of interdependent time-series sensor data. The model is implemented in Verilog HDL, synthesized with a 45 nm process technology, and evaluated in terms of the anomaly capability, hardware amount, and performance. The evaluation results demonstrate that the RLS-ESN core with a feedback structure is more robust to hyper parameters than an existing Online Sequential Extreme Learning Machine (OS-ELM) core. It consumes only 1.25 times larger hardware amount and 1.11 times longer latency than the existing OS-ELM core.

We studied an acoustic anomaly detection system for equipments, where the outlier detection method based on recorded sounds is used. In a real environment, the SNR of the target sound against background noise is low, and there is the problem that it is necessary to catch slight changes in sound buried in noise. In this paper, we propose a system in which a sound source extraction process is provided at the preliminary stage of the outlier detection process. In the proposed system, nonnegative matrix factorization based on generalized Gaussian distribution (GGD-NMF) is used as a sound source extraction process. We evaluated the improvement of the anomaly detection performance in a low-SNR environment. In this experiment, SNR capable of detecting an anomaly was greatly improved by providing GGD-NMF for preprocessing.

Blockchain is a distributed ledger system composed of a P2P network and is used for a wide range of applications, such as international remittance, inter-individual transactions, and asset conservation. In Blockchain systems, tamper resistance is enhanced by the property of transaction that cannot be changed or deleted by everyone including the creator of the transaction. However, this property also becomes a problem that unintended transaction created by miss operation or secret key theft cannot be corrected later. Due to this problem, once an illegal transaction such as theft occurs, the damage will expand. To suppress the damage, we need countermeasures, such as detecting illegal transaction at high speed and correcting the transaction before approval. However, anomaly detection in the Blockchain at high speed is computationally heavy, because we need to repeat the detection process using various feature quantities and the feature extractions become overhead. In this paper, to accelerate anomaly detection, we propose to cache transaction information necessary for extracting feature in GPU device memory and perform both feature extraction and anomaly detection in the GPU. We also propose a conditional feature extraction method to reduce computation cost of anomaly detection. We employ anomaly detection using K-means algorithm based on the conditional features. When the number of users is one million and the number of transactions is 100 millions, our proposed method achieves 8.6 times faster than CPU processing method and 2.6 times faster than GPU processing method that does not perform feature extraction on the GPU. In addition, the conditional feature extraction method achieves 1.7 times faster than the unconditional method when the number of users satisfying a given condition is 200 thousands out of one million.

System logs record system states and significant events at various critical points to help debug performance issues and failures. Therefore, the rapid and accurate detection of the system log is crucial to the security and stability of the system. In this paper, proposed is a novel attention-based neural network model, which would learn log patterns from normal execution. Concretely, our model adopts a GRU module with attention mechanism to extract the comprehensive and intricate correlations and patterns embedded in a sequence of log entries. Experimental results demonstrate that our proposed approach is effective and achieve better performance than conventional methods.

Deep learning is gaining more and more lots of attractions and better performance in implementing the Intrusion Detection System (IDS), especially for feature learning. This paper presents the state-of-the-art advances and challenges in IDS using deep learning models, which have been achieved the big performance enhancements in the field of computer vision, natural language processing, and image/audio processing than the traditional methods. After providing a systematic and methodical description of the latest developments in deep learning from the points of the deployed architectures and techniques, we suggest the pros-and-cons of all the deep learning-based IDS, and discuss the importance of deep learning models as feature learning approach. For this, the author has suggested the concept of the Deep-Feature Extraction and Selection (D-FES). By combining the stacked feature extraction and the weighted feature selection for D-FES, our experiment was verified to get the best performance of detection rate, 99.918% and false alarm rate, 0.012% to detect the impersonation attacks in Wi-Fi network which can be achieved better than the previous publications. Summary and further challenges are suggested as a concluding remark.

When people learn a handicraft with instructional contents such as books, videos, and web pages, many of them often give up halfway because the contents do not always assure how to make it. This study aims to provide origami learners, especially beginners, with feedbacks on their folding operations. An approach for recognizing the state of the learner by using a single top-view camera, and pointing out the mistakes made during the origami folding operation is proposed. First, an instruction model that stores easy-to-follow folding operations is defined. Second, a method for recognizing the state of the learner's origami paper sheet is proposed. Third, a method for detecting mistakes made by the learner by means of anomaly detection using a one-class support vector machine (one-class SVM) classifier (using the folding progress and the difference between the learner's origami shape and the correct shape) is proposed. Because noises exist in the camera images due to shadows and occlusions caused by the learner's hands, the shapes of the origami sheet are not always extracted accurately. To train the one-class SVM classifier with high accuracy, a data cleansing method that automatically sifts out video frames with noises is proposed. Moreover, using the statistics of features extracted from the frames in a sliding window makes it possible to reduce the influence by the noises. The proposed method was experimentally demonstrated to be sufficiently accurate and robust against noises, and its false alarm rate (false positive rate) can be reduced to zero. Requiring only a single camera and common origami paper, the proposed method makes it possible to monitor mistakes made by origami learners and support their self-learning.

Threat object recognition in x-ray security images is one of the important practical applications of computer vision. However, research in this field has been limited by the lack of available dataset that would mirror the practical setting for such applications. In this paper, we present a novel GAN-based anomaly detection (GBAD) approach as a solution to the extreme class-imbalance problem in multi-label classification. This method helps in suppressing the surge in false positives induced by training a CNN on a non-practical dataset. We evaluate our method on a large-scale x-ray image database to closely emulate practical scenarios in port security inspection systems. Experiments demonstrate improvement against the existing algorithm.

System logs are useful to understand the status of and detect faults in large scale networks. However, due to their diversity and volume of these logs, log analysis requires much time and effort. In this paper, we propose a log event anomaly detection method for large-scale networks without pre-processing and feature extraction. The key idea is to embed a large amount of diverse data into hidden states by using latent variables. We evaluate our method with 12 months of system logs obtained from a nation-wide academic network in Japan. Through comparisons with Kleinberg's univariate burst detection and a traditional multivariate analysis (i.e., PCA), we demonstrate that our proposed method achieves 14.5% higher recall and 3% higher precision than PCA. A case study shows detected anomalies are effective information for troubleshooting of network system faults.

Since cyber attacks such as cyberterrorism against Industrial Control Systems (ICSs) and cyber espionage against companies managing them have increased, the techniques to detect anomalies in early stages are required. To achieve the purpose, several studies have developed anomaly detection methods for ICSs. In particular, some techniques using packet flow regularity in industrial control networks have achieved high-accuracy detection of attacks disrupting the regularity, i.e. normal behaviour, of ICSs. However, these methods cannot identify scanning attacks employed in cyber espionage because the probing packets assimilate into a number of normal ones. For example, the malware called Havex is customised to clandestinely acquire information from targeting ICSs using general request packets. The techniques to detect such scanning attacks using widespread packets await further investigation. Therefore, the goal of this study was to examine high performance methods to identify anomalies even if elaborate packets to avoid alert systems were employed for attacks against industrial control networks. In this paper, a novel detection model for anomalous packets concealing behind normal traffic in industrial control networks was proposed. For the proposal of the sophisticated detection method, we took particular note of packet flow regularity and employed the Markov-chain model to detect anomalies. Moreover, we regarded not only original packets but similar ones to them as normal packets to reduce false alerts because it was indicated that an anomaly detection model using the Markov-chain suffers from the ample false positives affected by a number of normal, irregular packets, namely noise. To calculate the similarity between packets based on the packet flow regularity, a vector representation tool called word2vec was employed. Whilst word2vec is utilised for the culculation of word similarity in natural language processing tasks, we applied the technique to packets in ICSs to calculate packet similarity. As a result, the Markov-chain with word2vec model identified scanning packets assimulating into normal packets in higher performance than the conventional Markov-chain model. In conclusion, employing both packet flow regularity and packet similarity in industrial control networks contributes to improving the performance of anomaly detection in ICSs.

Detecting anomalies, such as network failure or intentional attack in Internet, is a vital but challenging task. Although numerous techniques have been developed based on Internet traffic, detecting anomalies from the perspective of Internet topology structure is going to be possible because the anomaly detection of structured datasets based on complex network theory has become a focus of attention recently. In this paper, an anomaly detection method for the large-scale Internet topology is proposed to detect local structure crashes caused by the cascading failure. In order to quantify the dynamic changes of Internet topology, the network path changes coefficient (NPCC) is put forward which highlights the Internet abnormal state after it is attacked continuously. Furthermore, inspired by Fibonacci Sequence, we proposed the decision function that can determine whether the Internet is abnormal or not. That is the current Internet is abnormal if its NPCC is out of the normal domain calculated using the previous k NPCCs of Internet topology. Finally the new Internet anomaly detection method is tested against the topology data of three Internet anomaly events. The results show that the detection accuracy of all events are over 97%, the detection precision for three events are 90.24%, 83.33% and 66.67%, when k=36. According to the experimental values of index F1, larger values of k offer better detection performance. Meanwhile, our method has better performance for the anomaly behaviors caused by network failure than those caused by intentional attack. Compared with traditional anomaly detection methods, our work is more simple and powerful for the government or organization in items of detecting large-scale abnormal events.