A classification function maps a set of vectors into several classes. A machine learning problem is treated as a design problem for partially defined classification functions. To realize classification functions for MNIST hand written digits, three different architectures are considered: Single-unit realization, 45-unit realization, and 45-unit ×r realization. The 45-unit realization consists of 45 ternary classifiers, 10 counters, and a max selector. Test accuracy of these architectures are compared using MNIST data set.

Predictive spatial-monitoring, which predicts spatial information such as road traffic, has attracted much attention in the context of smart cities. Machine learning enables predictive spatial-monitoring by using a large amount of aggregated sensor data. Since the capacity of mobile networks is strictly limited, serious transmission delays occur when loads of communication traffic are heavy. If some of the data used for predictive spatial-monitoring do not arrive on time, prediction accuracy degrades because the prediction has to be done using only the received data, which implies that data for prediction are ‘delay-sensitive’. A utility-based allocation technique has suggested modeling of temporal characteristics of such delay-sensitive data for prioritized transmission. However, no study has addressed temporal model for prioritized transmission in predictive spatial-monitoring. Therefore, this paper proposes a scheme that enables the creation of a temporal model for predictive spatial-monitoring. The scheme is roughly composed of two steps: the first involves creating training data from original time-series data and a machine learning model that can use the data, while the second step involves modeling a temporal model using feature selection in the learning model. Feature selection enables the estimation of the importance of data in terms of how much the data contribute to prediction accuracy from the machine learning model. This paper considers road-traffic prediction as a scenario and shows that the temporal models created with the proposed scheme can handle real spatial datasets. A numerical study demonstrated how our temporal model works effectively in prioritized transmission for predictive spatial-monitoring in terms of prediction accuracy.

Machine learning is becoming an attractive topic for researchers and industrial firms in the area of computational intelligence because of its proven effectiveness and performance in resolving real-world problems. However, some challenges such as precise search, intelligent discovery and intelligent learning need to be addressed and solved. One most important challenge is the non-steady performance of various machine learning models during online learning and operation. Online learning is the ability of a machine-learning model to modernize information without retraining the scheme when new information is available. To address this challenge, we evaluate and analyze four widely used online machine learning models: Online Sequential Extreme Learning Machine (OSELM), Feature Adaptive OSELM (FA-OSELM), Knowledge Preserving OSELM (KP-OSELM), and Infinite Term Memory OSELM (ITM-OSELM). Specifically, we provide a testbed for the models by building a framework and configuring various evaluation scenarios given different factors in the topological and mathematical aspects of the models. Furthermore, we generate different characteristics of the time series to be learned. Results prove the real impact of the tested parameters and scenarios on the models. In terms of accuracy, KP-OSELM and ITM-OSELM are superior to OSELM and FA-OSELM. With regard to time efficiency related to the percentage of decreases in active features, ITM-OSELM is superior to KP-OSELM.

Since most sensor data depend on each other, time-series anomaly detection is one of practical applications of IoT devices. Such tasks are handled by Recurrent Neural Networks (RNNs) with a feedback structure, such as Long Short Term Memory. However, their learning phase based on Stochastic Gradient Descent (SGD) is computationally expensive for such edge devices. This issue is addressed by executing their learning on high-performance server machines, but it introduces a communication overhead and additional power consumption. On the other hand, Recursive Least-Squares Echo State Network (RLS-ESN) is a simple RNN that can be trained at low cost using the least-squares method rather than SGD. In this paper, we propose its area-efficient hardware implementation for edge devices and adapt it to human activity anomaly detection as an example of interdependent time-series sensor data. The model is implemented in Verilog HDL, synthesized with a 45 nm process technology, and evaluated in terms of the anomaly capability, hardware amount, and performance. The evaluation results demonstrate that the RLS-ESN core with a feedback structure is more robust to hyper parameters than an existing Online Sequential Extreme Learning Machine (OS-ELM) core. It consumes only 1.25 times larger hardware amount and 1.11 times longer latency than the existing OS-ELM core.

Network function virtualization (NFV) enables network operators to flexibly provide diverse virtualized functions for services such as Internet of things (IoT) and mobile applications. To meet multiple quality of service (QoS) requirements against time-varying network environments, infrastructure providers must dynamically adjust the amount of computational resources, such as CPU, assigned to virtual network functions (VNFs). To provide agile resource control and adaptiveness, predicting the virtual server load via machine learning technologies is an effective approach to the proactive control of network systems. In this paper, we propose an adjustment mechanism for regressors based on forgetting and dynamic ensemble executed in a shorter time than that of our previous work. The framework includes a reducing training data method based on sparse model regression. By making a short list of training data derived from the sparse regression model, the relearning time can be reduced to about 57% without degrading provisioning accuracy.

With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving images that are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacy concerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.

Machine learning models inherently memorize significant amounts of information, and thus hiding not only prediction processes but also trained models, i.e., model obliviousness, is desirable in the cloud setting. Several works achieved model obliviousness with the MNIST dataset, but datasets that include complicated samples, e.g., CIFAR-10 and CIFAR-100, are also used in actual applications, such as face recognition. Secret sharing-based secure prediction for CIFAR-10 is difficult to achieve. When a deep layer architecture such as CNN is used, the calculation error when performing secret calculation becomes large and the accuracy deteriorates. In addition, if detailed calculations are performed to improve accuracy, a large amount of calculation is required. Therefore, even if the conventional method is applied to CNN as it is, good results as described in the paper cannot be obtained. In this paper, we propose two approaches to solve this problem. Firstly, we propose a new protocol named Batch-normalizedActivation that combines BatchNormalization and Activation. Since BatchNormalization includes real number operations, when performing secret calculation, parameters must be converted into integers, which causes a calculation error and decrease accuracy. By using our protocol, calculation errors can be eliminated, and accuracy degradation can be eliminated. Further, the processing is simplified, and the amount of calculation is reduced. Secondly, we explore a secret computation friendly and high accuracy architecture. Related works use a low-accuracy, simple architecture, but in reality, a high accuracy architecture should be used. Therefore, we also explored a high accuracy architecture for the CIFAR10 dataset. Our proposed protocol can compute prediction of CIFAR-10 within 15.05 seconds with 87.36% accuracy while providing model obliviousness.

Autonomous mobility machines, such as self-driving cars, transportation robots, and automated construction machines, are promising to support or enrich human lives. To further improve such machines, they will be connected to the network via wireless links to be managed, monitored, or remotely operated. The autonomous mobility machines must have self-status based on their positioning system to safely conduct their operations without colliding with other objects. The self-status is not only essential for machine operation but also it is valuable for wireless link quality management. This paper presents self-status-based wireless link quality prediction and evaluates its performance by using a prototype mobility robot combined with a wireless LAN system. The developed robot has functions to measure the throughput and receive signal strength indication and obtain self-status details such as location, direction, and odometry data. Prediction performance is evaluated in offline processing by using the dataset gathered in an indoor experiment. The experiments clarified that, in the 5.6 GHz band, link quality prediction using self-status of the robot forecasted the throughput several seconds into the future, and the prediction accuracies were investigated as dependent on time window size of the target throughput, bandwidth, and frequency gap.

We propose a privacy-preserving machine learning scheme with encryption-then-compression (EtC) images, where EtC images are images encrypted by using a block-based encryption method proposed for EtC systems with JPEG compression. In this paper, a novel property of EtC images is first discussed, although EtC ones was already shown to be compressible as a property. The novel property allows us to directly apply EtC images to machine learning algorithms non-specialized for computing encrypted data. In addition, the proposed scheme is demonstrated to provide no degradation in the performance of some typical machine learning algorithms including the support vector machine algorithm with kernel trick and random forests under the use of z-score normalization. A number of facial recognition experiments with are carried out to confirm the effectiveness of the proposed scheme.

Direction of arrival (DOA) estimation of wireless signals has a long history but is still being investigated to improve the estimation accuracy. Non-linear algorithms such as compressed sensing are now applied to DOA estimation and achieve very high performance. If the large computational loads of compressed sensing algorithms are acceptable, it may be possible to apply a deep neural network (DNN) to DOA estimation. In this paper, we verify on-grid DOA estimation capability of the DNN under a simple estimation situation and discuss the effect of training data on DNN design. Simulations show that SNR of the training data strongly affects the performance and that the random SNR data is suitable for configuring the general-purpose DNN. The obtained DNN provides reasonably high performance, and it is shown that the DNN trained using the training data restricted to close DOA situations provides very high performance for the close DOA cases.

Social media channels, such as Facebook, Twitter, and Instagram, have altered our world forever. People are now increasingly connected than ever and reveal a sort of digital persona. Although social media certainly has several remarkable features, the demerits are undeniable as well. Recent studies have indicated a correlation between high usage of social media sites and increased depression. The present study aims to exploit machine learning techniques for detecting a probable depressed Twitter user based on both, his/her network behavior and tweets. For this purpose, we trained and tested classifiers to distinguish whether a user is depressed or not using features extracted from his/her activities in the network and tweets. The results showed that the more features are used, the higher are the accuracy and F-measure scores in detecting depressed users. This method is a data-driven, predictive approach for early detection of depression or other mental illnesses. This study's main contribution is the exploration part of the features and its impact on detecting the depression level.

Return-oriented programming (ROP) has been crucial for attackers to evade the security mechanisms of recent operating systems. Although existing ROP detection approaches mainly focus on host-based intrusion detection systems (HIDSes), network-based intrusion detection systems (NIDSes) are also desired to protect various hosts including IoT devices on the network. However, existing approaches are not enough for network-level protection due to two problems: (1) Dynamic approaches take the time with second- or minute-order on average for inspection. For applying to NIDSes, millisecond-order is required to achieve near real time detection. (2) Static approaches generate false positives because they use heuristic patterns. For applying to NIDSes, false positives should be minimized to suppress false alarms. In this paper, we propose a method for statically detecting ROP chains in malicious data by learning the target libraries (i.e., the libraries that are used for ROP gadgets). Our method accelerates its inspection by exhaustively collecting feasible ROP gadgets in the target libraries and learning them separated from the inspection step. In addition, we reduce false positives inevitable for existing static inspection by statically verifying whether a suspicious byte sequence can link properly when they are executed as a ROP chain. Experimental results showed that our method has achieved millisecond-order ROP chain detection with high precision.

Vision systems are widely adopted in industrial fields for monitoring and automation. As a typical example, industrial vision systems are extensively implemented in vibrator parts feeder to ensure orientations of parts for assembling are aligned and disqualified parts are eliminated. An efficient parts orientation recognition and counting method is thus critical to adopt. In this paper, an integrated method for fast parts counting and orientation recognition using industrial vision systems is proposed. Original 2D spatial image signal of parts is decomposed to 1D signal with its temporal variance, thus efficient recognition and counting is achievable, feeding speed of each parts is further leveraged to elaborate counting in an adaptive way. Experiments on parts of different types are conducted, the experimental results revealed that our proposed method is both more efficient and accurate compared to other relevant methods.

This paper addresses the problem of selecting a significant subset of candidate features to use for multiple linear regression. Bertsimas et al. [5] recently proposed the discrete first-order (DFO) algorithm to efficiently find near-optimal solutions to this problem. However, this algorithm is unable to escape from locally optimal solutions. To resolve this, we propose a stochastic discrete first-order (SDFO) algorithm for feature subset selection. In this algorithm, random perturbations are added to a sequence of candidate solutions as a means to escape from locally optimal solutions, which broadens the range of discoverable solutions. Moreover, we derive the optimal step size in the gradient-descent direction to accelerate convergence of the algorithm. We also make effective use of the L2-regularization term to improve the predictive performance of a resultant subset regression model. The simulation results demonstrate that our algorithm substantially outperforms the original DFO algorithm. Our algorithm was superior in predictive performance to lasso and forward stepwise selection as well.

This paper presents a real-time body motion classification system using the radio channel characteristics of a wearable body area network (BAN). We developed a custom wearable BAN radio channel measurement system by modifying an off-the-shelf ZigBee-based sensor network system, where the link quality indicator (LQI) values of the wireless links between the coordinator and four sensor nodes can be measured. After interpolating and standardizing the raw data samples in a pre-processing stage, the time-domain features are calculated, and the body motion is classified by a decision-tree based random forest machine learning algorithm which is most suitable for real-time processing. The features were carefully chosen to exclude those that exhibit the same tendency based on the mean and variance of the features to avoid overfitting. The measurements demonstrated successful real-time body motion classification and revealed the potential for practical use in various daily-life applications.

Mobile wireless sensor networks (WSNs) are facing threats from malicious nodes that disturb packet transmissions, leading to poor mobile WSN performance. Existing studies have proposed a number of methods, such as decision tree-based classification methods and reputation based methods, to detect these malicious nodes. These methods assume that the malicious nodes follow only pre-defined attack models and have no learning ability. However, this underestimation of the capability of malicious node is inappropriate due to recent rapid progresses in machine learning technologies. In this study, we design reinforcement learning-based malicious nodes, and define a novel observation space and sparse reward function for the reinforcement learning. We also design an adaptive learning method to detect these smart malicious nodes. We construct a robust classifier, which is frequently updated, to detect these smart malicious nodes. Extensive experiments show that, in contrast to existing attack models, the developed malicious nodes can degrade network performance without being detected. We also investigate the performance of our detection method, and confirm that the method significantly outperforms the state-of-the-art methods in terms of detection accuracy and false detection rate.

We propose a multi-targeted backdoor that misleads different models to different classes. The method trains multiple models with data that include specific triggers that will be misclassified by different models into different classes. For example, an attacker can use a single multi-targeted backdoor sample to make model A recognize it as a stop sign, model B as a left-turn sign, model C as a right-turn sign, and model D as a U-turn sign. We used MNIST and Fashion-MNIST as experimental datasets and Tensorflow as a machine learning library. Experimental results show that the proposed method with a trigger can cause misclassification as different classes by different models with a 100% attack success rate on MNIST and Fashion-MNIST while maintaining the 97.18% and 91.1% accuracy, respectively, on data without a trigger.

Malicious attackers on the Internet use automated attack programs to disrupt the use of services via mass spamming, unnecessary bulletin boarding, and account creation. Completely automated public turing test to tell computers and humans apart (CAPTCHA) is used as a security solution to prevent such automated attacks. CAPTCHA is a system that determines whether the user is a machine or a person by providing distorted letters, voices, and images that only humans can understand. However, new attack techniques such as optical character recognition (OCR) and deep neural networks (DNN) have been used to bypass CAPTCHA. In this paper, we propose a method to generate CAPTCHA images by using the fast-gradient sign method (FGSM), iterative FGSM (I-FGSM), and the DeepFool method. We used the CAPTCHA image provided by python as the dataset and Tensorflow as the machine learning library. The experimental results show that the CAPTCHA image generated via FGSM, I-FGSM, and DeepFool methods exhibits a 0% recognition rate with ε=0.15 for FGSM, a 0% recognition rate with α=0.1 with 50 iterations for I-FGSM, and a 45% recognition rate with 150 iterations for the DeepFool method.

As one type of machine-learning model, a “decision-tree ensemble model” (DTEM) is represented by a set of decision trees. A DTEM is mainly known to be valid for structured data; however, like other machine-learning models, it is difficult to train so that it returns the correct output value (called “prediction value”) for any input value (called “attribute value”). Accordingly, when a DTEM is used in regard to a system that requires reliability, it is important to comprehensively detect attribute values that lead to malfunctions of a system (failures) during development and take appropriate countermeasures. One conceivable solution is to install an input filter that controls the input to the DTEM and to use separate software to process attribute values that may lead to failures. To develop the input filter, it is necessary to specify the filtering condition for the attribute value that leads to the malfunction of the system. In consideration of that necessity, we propose a method for formally verifying a DTEM and, according to the result of the verification, if an attribute value leading to a failure is found, extracting the range in which such an attribute value exists. The proposed method can comprehensively extract the range in which the attribute value leading to the failure exists; therefore, by creating an input filter based on that range, it is possible to prevent the failure. To demonstrate the feasibility of the proposed method, we performed a case study using a dataset of house prices. Through the case study, we also evaluated its scalability and it is shown that the number and depth of decision trees are important factors that determines the applicability of the proposed method.

With the recent advances in e-commerce, it has become important to recommend not only mass-produced daily items, such as books, but also items that are not mass-produced. In this study, we present an algorithm for real estate recommendations. Automatic property recommendations are a highly difficult task because no identical properties exist in the world, occupied properties cannot be recommended, and users rent or buy properties only a few times in their lives. For the first step of property recommendation, we predict users' preferences for properties by combining content-based filtering and Multi-Layer Perceptron (MLP). In the MLP, we use not only attribute data of users and properties, but also deep features extracted from property floor plan images. As a result, we successfully predict users' preference with a Matthews Correlation Coefficient (MCC) of 0.166.