Representation learning is a crucial and complex task for multivariate time series data analysis, with a wide range of applications including trend analysis, time series data search, and forecasting. In practice, unsupervised learning is strongly preferred owing to sparse labeling. However, most existing studies focus on the representation of individual subseries without considering relationships between different subseries. In certain scenarios, this may lead to downstream task failures. Here, an unsupervised representation learning model is proposed for multivariate time series that considers the semantic relationship among subseries of time series. Specifically, the covariance calculated by the Gaussian process (GP) is introduced to the self-attention mechanism, capturing relationship features of the subseries. Additionally, a novel unsupervised method is designed to learn the representation of multivariate time series. To address the challenges of variable lengths of input subseries, a temporal pyramid pooling (TPP) method is applied to construct input vectors with equal length. The experimental results show that our model has substantial advantages compared with other representation learning models. We conducted experiments on the proposed algorithm and baseline algorithms in two downstream tasks: classification and retrieval. In classification task, the proposed model demonstrated the best performance on seven of ten datasets, achieving an average accuracy of 76%. In retrieval task, the proposed algorithm achieved the best performance under different datasets and hidden sizes. The result of ablation study also demonstrates significance of semantic relationship in multivariate time series representation learning.

Social relationships (e.g., couples, opponents) are the foundational part of society. Social relation atmosphere describes the overall interaction environment between social relationships. Discovering social relation atmosphere can help machines better comprehend human behaviors and improve the performance of social intelligent applications. Most existing research mainly focuses on investigating social relationships, while ignoring the social relation atmosphere. Due to the complexity of the expressions in video data and the uncertainty of the social relation atmosphere, it is even difficult to define and evaluate. In this paper, we innovatively analyze the social relation atmosphere in video data. We introduce a Relevant Visual Concept (RVC) from the social relationship recognition task to facilitate social relation atmosphere recognition, because social relationships contain useful information about human interactions and surrounding environments, which are crucial clues for social relation atmosphere recognition. Our approach consists of two main steps: (1) we first generate a group of visual concepts that preserve the inherent social relationship information by utilizing a 3D explanation module; (2) the extracted relevant visual concepts are used to supplement the social relation atmosphere recognition. In addition, we present a new dataset based on the existing Video Social Relation Dataset. Each video is annotated with four kinds of social relation atmosphere attributes and one social relationship. We evaluate the proposed method on our dataset. Experiments with various 3D ConvNets and fusion methods demonstrate that the proposed method can effectively improve recognition accuracy compared to end-to-end ConvNets. The visualization results also indicate that essential information in social relationships can be discovered and used to enhance social relation atmosphere recognition.

In this paper, we propose a selective membership inference attack method that determines whether certain data corresponding to a specific class are being used as training data for a machine learning model or not. By using the proposed method, membership or non-membership can be inferred by generating a decision model from the prediction of the inference models and training the confidence values for the data corresponding to the selected class. We used MNIST as an experimental dataset and Tensorflow as a machine learning library. Experimental results show that the proposed method has a 92.4% success rate with 5 inference models for data corresponding to a specific class.

Salient ship detection plays an important role in ensuring the safety of maritime transportation and navigation. However, due to the influence of waves, special weather, and illumination on the sea, existing saliency methods are still unable to achieve effective ship detection in a complex marine environment. To solve the problem, this paper proposed a novel saliency method based on an attention nested U-Structure (AU2Net). First, to make up for the shortcomings of the U-shaped structure, the pyramid pooling module (PPM) and global guidance paths (GGPs) are designed to guide the restoration of feature information. Then, the attention modules are added to the nested U-shaped structure to further refine the target characteristics. Ultimately, multi-level features and global context features are integrated through the feature aggregation module (FAM) to improve the ability to locate targets. Experiment results demonstrate that the proposed method could have at most 36.75% improvement in F-measure (Favg) compared to the other state-of-the-art methods.

During the detection of maritime targets, the jitter of the shipborne camera usually causes the video instability and the false or missed detection of targets. Aimed at tackling this problem, a novel algorithm for maritime target detection based on the electronic image stabilization technology is proposed in this study. The algorithm mainly includes three models, namely the points line model (PLM), the points classification model (PCM), and the image classification model (ICM). The feature points (FPs) are firstly classified by the PLM, and stable videos as well as target contours are obtained by the PCM. Then the smallest bounding rectangles of the target contours generated as the candidate bounding boxes (bboxes) are sent to the ICM for classification. In the experiments, the ICM, which is constructed based on the convolutional neural network (CNN), is trained and its effectiveness is verified. Our experimental results demonstrate that the proposed algorithm outperformed the benchmark models in all the common metrics including the mean square error (MSE), peak signal to noise ratio (PSNR), structural similarity index (SSIM), and mean average precision (mAP) by at least -47.87%, 8.66%, 6.94%, and 5.75%, respectively. The proposed algorithm is superior to the state-of-the-art techniques in both the image stabilization and target ship detection, which provides reliable technical support for the visual development of unmanned ships.

The progress of immersive technology enables researchers and developers to construct work spaces that are freed from real-world constraints. This has motivated us to investigate the role of the human body. In this research, we examine human cognitive behaviors in obtaining an understanding of the width of their virtual body through simple yet meaningful experiments using virtual reality (VR). In the experiments, participants were modeled as an invisible board, and a spherical object was thrown at the participants to provide information for exploring the width of their invisible body. Audio and visual feedback were provided when the object came into contact with the board (body). We first explored how precisely the participants perceived the virtual body width. Next, we examined how the body perception was generated and changed as the trial proceeded when the participants tried to move right or left actively for the avoidance of collision with approaching objects. The results of the experiments indicated that the participants could become successful in avoiding collision within a limited number of trials (14 at most) under the experimental conditions. It was also found that they postponed deciding how much they should move at the beginning and then started taking evasive action earlier as they become aware of the virtual body.

The membership check of a group is an important operation to implement discrete logarithm-based cryptography in practice securely. Since this check requires costly scalar multiplication or exponentiation operation, several efficient methods have been investigated. In the case of pairing-based cryptography, this is an extended research area of discrete logarithm-based cryptography, Barreto et al. (LATINCRYPT 2015) proposed a parameter choice called subgroup-secure elliptic curves. They also claimed that, in some schemes, if an elliptic curve is subgroup-secure, costly scalar multiplication or exponentiation operation can be omitted from the membership check of bilinear groups, which results in faster schemes than the original ones. They also noticed that some schemes would not maintain security with this omission. However, they did not show the explicit condition of what schemes become insecure with the omission. In this paper, we show a concrete example of insecurity in the sense of subgroup security to help developers understand what subgroup security is and what properties are preserved. In our conclusion, we recommend that the developers use the original membership check because it is a general and straightforward method to implement schemes securely. If the developers want to use the subgroup-secure elliptic curves and to omit the costly operation in a scheme for performance reasons, it is critical to carefully analyze again that correctness and security are preserved with the omission.

Incident ticket classification plays an important role in the complex system maintenance. However, low classification accuracy will result in high maintenance costs. To solve this issue, this paper proposes a fuzzy output support vector machine (FOSVM) based incident ticket classification approach, which can be implemented in the context of both two-class SVMs and multi-class SVMs such as one-versus-one and one-versus-rest. Our purpose is to solve the unclassifiable regions of multi-class SVMs to output reliable and robust results by more fine-grained analysis. Experiments on both benchmark data sets and real-world ticket data demonstrate that our method has better performance than commonly used multi-class SVM and fuzzy SVM methods.

Figure-ground (FG) segregation has been considered as a fundamental step towards object recognition. We explored plausible mechanisms that estimate global figure-ground segregation from local image features by investigating the human visual system. Physiological studies have reported border-ownership (BO) selective neurons in V2 which signal the local direction of figure (DOF) along a border; however, how local BO signals contribute to global FG segregation has not been clarified. The BO and FG processing could be independent, dependent on each other, or inseparable. The investigation on the differences and similarities between the BO and FG judgements is important for exploring plausible mechanisms that enable global FG estimation from local clues. We performed psychophysical experiments that included two different tasks each of which focused on the judgement of either BO or FG. The perceptual judgments showed consistency between the BO and FG determination while a longer distance in gaze movement was observed in FG segregation than BO discrimination. These results suggest the involvement of distinct neural mechanism for local BO determination and global FG segregation.

A regular pattern is a string consisting of constant symbols and distinct variable symbols. The language of a regular pattern is the set of all constant strings obtained by replacing all variable symbols in the regular pattern with non-empty strings. The present paper deals with the learning problem of languages of regular patterns within Angluin's query learning model, which is an established mathematical model of learning via queries in computational learning theory. The class of languages of regular patterns was known to be identifiable from one positive example using a polynomial number of membership queries, in the query learning model. In present paper, we show that the class of languages of regular patterns is identifiable from one positive example using a linear number of membership queries, with respect to the length of the positive example.

In historical learning, to grasp the causal relationship between historical events and to understand factors that bring about important events are significant for fostering the historical thinking. However, some students are not able to find historical events that have causal relationships. The view of observing the historical events is different among individuals, so it is not appropriate to define the historical events that have causal relationships and impose students to remember them. The students need to understand the definition of the causal relationships and find the historical events that satisfy the definition according to their viewpoints. The objective of this paper is to develop a support system for understanding the meaning of a causal relationship and creating causal relation graphs that represent the causal relationships between historical events. When historical events have a causal relationship, a state change caused by one event becomes the cause of the other event. To consider these state changes is critically important to connect historical events. This paper proposes steps for considering causal relationships between historical events by arranging the state changes of historical people along with them. It also develops the system that supports students to create the causal relation graph according to the state changes. In our system, firstly, the interface for arranging state changes of historical people according to the historical events is given. Then, the interface for drawing the causal relation graph of historical events is provided in which state changes are automatically indicated on the created links in the causal relation graph. By observing the indicated state changes on the links, students are able to check by themselves whether their causal relation graphs correctly represent the causal relationships between historical events.

Advanced metering infrastructure (AMI) is a kind of wireless sensor network that provides two-way communication between smart meters and city utilities in the neighborhood area of the smart grid. And the routing protocol for low-power and lossy network (RPL) is being considered for use in AMI networks. However, there still exist several problems that need to be solved, especially with respect to QoS guarantees. To address these problems, an improved algorithm of RPL based on triangle module operator named as TMO is proposed. TMO comprehensively evaluates routing metrics: end-to-end delay, number of hops, expected transmission count, node remaining energy, and child node count. Moreover, TMO uses triangle module operator to fuse membership functions of these routing metrics. Then, the node with minimum rank value will be selected as preferred parent (the next hop). Consequently, the QoS of RPL-based AMI networks can be guaranteed effectively. Simulation results show that TMO offers a great improvement over several the most popular schemes for RPL like ETXOF, OF-FL and additive composition metric manners in terms of network lifetime, average end-to-end delay, average packet loss ratio, average hop count from nodes to root, etc.

Malware phylogeny refers to inferring evolutionary relationships between instances of families. It has gained a lot of attention over the past several years, due to its efficiency in accelerating reverse engineering of new variants within families. Previous researches mainly focused on tree-based models. However, those approaches merely demonstrate lineage of families using dendrograms or directed trees with rough evolution information. In this paper, we propose a novel malware phylogeny construction method taking advantage of persistent phylogeny tree model, whose nodes correspond to input instances and edges represent the gain or lost of functional characters. It can not only depict directed ancestor-descendant relationships between malware instances, but also show concrete function inheritance and variation between ancestor and descendant, which is significant in variants defense. We evaluate our algorithm on three malware families and one benign family whose ground truth are known, and compare with competing algorithms. Experiments demonstrate that our method achieves a higher mean accuracy of 61.4%.

Object detection has been a hot topic of image processing, computer vision and pattern recognition. In recent years, training a model from labeled images using machine learning technique becomes popular. However, the relationship between training samples is usually ignored by existing approaches. To address this problem, a novel approach is proposed, which trains Siamese convolutional neural network on feature pairs and finely tunes the network driven by a small amount of training samples. Since the proposed method considers not only the discriminative information between objects and background, but also the relationship between intraclass features, it outperforms the state-of-arts on real images.

Subspace membership encryption (SME), a generalization of inner product encryption (IPE), was recently formalized by Boneh, Raghunathan, and Segev in Asiacrypt 2013. The main motivation for SME was that traditional predicate encryptions did not yield function privacy, a security notion introduced by Boneh et al. in Crypto 2013 that captures the privacy of the predicate associated to the secret key. Although they gave a generic construction of SME based on any IPE, we show that their construction of SME for small attribute space was incorrect and provide an attack that breaks the attribute hiding security, a baseline security notion for predicate encryptions that captures the privacy of the attribute associated with the ciphertext. Then, we propose a generalized construction of SME and prove that the attribute hiding security can not be achieved even in the newly defined setting. Finally, we further extend our generalized construction of SME and propose a SME that achieves the attribute hiding property even when the attribute space is small. In exchange our proposed scheme does not yield function privacy and the construction is rather inefficient. Although we did not succeed in constructing a SME both yielding function privacy and attribute hiding security, ours is the first attribute hiding SME scheme whose attribute space is polynomial in the security parameter, and we formalized a richer framework for constructing SMEs and discovered a trade-off like relationship between the two security notions.

Radio Frequency Identifications (RFID) are useful low-cost devices for identification or authentication systems through wireless communication. The ownership of the RFID tag is frequently changed in the life cycle of the tag, it may fall in to the hands of a malicious adversary. The privacy problem in this situation is studied in the RFID ownership transfer protocol. However, almost all previous works provide only heuristic analysis and many protocols are broken. Elkhiyaoui et al. defined the security model for RFID ownership transfer protocols and proposed the detailed security proof to their protocol, but we show that their protocol does not provide enough privacy and cover the realistic attack. We investigate a suitable security model for RFID ownership transfer protocols and provide a new provably secure RFID ownership transfer protocol.

Routing is a challenging issue in mobile social networks (MSNs) because of time-varying links and intermittent connectivity. In order to enable nodes to make right decisions while forwarding messages, exploiting social relationship has become an important method for designing efficient routing protocols in MSNs. In this paper, we first use the temporal evolution graph model to accurately capture the dynamic topology of the MSN. Based on the model, we introduce the social relationship metric for detecting the quality of human social relationship from contact history records. Utilizing this metric, we propose social relationship based betweenness centrality metric to identify influential nodes to ensure messages forwarded by the nodes with stronger social relationship and higher likelihood of contacting other nodes. Then, we present SRBet, a novel social-based forwarding algorithm, which utilizes the aforementioned metric to enhance routing performance. Simulations have been conducted on two real world data sets and results demonstrate that the proposed forwarding algorithm achieves better performances than the existing algorithms.

Ultra high definition (UHD) imaging systems have attracted much attention as a next generation television (TV) broadcasting service and video streaming service. However, the state of the art video coding standards including H.265/HEVC has not enough compression rate for streaming, broadcasting and storing UHD. Existing coding standard such as H.265/HEVC normaly use RGB-YCbCr color transform before compressing RGB color image since that procedure can decorrelate color components well. However, there is room for improvement on the coding efficiency for color image based on an observation that the luminance and chrominance components changes in same locations. This observation inspired us to propose a new post-processing method for compressed images by using weighted least square (WLS) filter with coded luminance component as a guide image, for refining the edges of chrominance components. Since the computational cost of WLS tends to superlinearly increase with increasing image size, it is difficult to apply it to UHD images. To overcome this problem, we propose slightly overlapped block partitioning and a new variant of WLS (constrained WLS, CWLS). Experimental results of objective quality comparison and subjective assessment test using 4K images show that our proposed method can outperform the conventional method and reduce the bit amount for chrominance component drastically with preserving the subjective quality.

In this paper, we propose a method for reconstructing 3D sequential patterns from multiple images without knowing exact image correspondences and without calibrating linear camera sensitivity parameters on intensity. The sequential pattern is defined as a series of colored 3D points. We assume that the series of the points are obtained in multiple images, but the correspondence of individual points is not known among multiple images. For reconstructing sequential patterns, we consider a camera projection model which combines geometric and photometric information of objects. Furthermore, we consider camera projections in the frequency space. By considering the multi-view relationship on the new projection model, we show that the 3D sequential patterns can be reconstructed without knowing exact correspondence of individual image points in the sequential patterns; moreover, the recovered 3D patterns do not suffer from changes in linear camera sensitivity parameters. The efficiency of the proposed method is tested using real images.

The bag-of-words model (BOW) has been extensively adopted by recent human action recognition methods. The pooling operation, which aggregates local descriptor encodings into a single representation, is a key determiner of the performance of the BOW-based methods. However, the spatio-temporal relationship among interest points has rarely been considered in the pooling step, which results in the imprecise representation of human actions. In this paper, we propose a novel pooling strategy named contextual max pooling (CMP) to overcome this limitation. We add a constraint term into the objective function under the framework of max pooling, which forces the weights of interest points to be consistent with their probabilities. In this way, CMP explicitly considers the spatio-temporal contextual relationships among interest points and inherits the positive properties of max pooling. Our method is verified on three challenging datasets (KTH, UCF Sports and UCF Films datasets), and the results demonstrate that our method achieves better results than the state-of-the-art methods in human action recognition.