Fine-grained image classification is one of the key basic tasks of computer vision. The appearance of traditional deep convolutional neural network (DCNN) combined with attention mechanism can focus on partial and local features of fine-grained images, but it still lacks the consideration of the embedding mode of different attention modules in the network, leading to the unsatisfactory result of classification model. To solve the above problems, three different attention mechanisms are introduced into the DCNN network (like ResNet, VGGNet, etc.), including SE, CBAM and ECA modules, so that DCNN could better focus on the key local features of salient regions in the image. At the same time, we adopt three different embedding modes of attention modules, including serial, residual and parallel modes, to further improve the performance of the classification model. The experimental results show that the three attention modules combined with three different embedding modes can improve the performance of DCNN network effectively. Moreover, compared with SE and ECA, CBAM has stronger feature extraction capability. Among them, the parallelly embedded CBAM can make the local information paid attention to by DCNN richer and more accurate, and bring the optimal effect for DCNN, which is 1.98% and 1.57% higher than that of original VGG16 and Resnet34 in CUB-200-2011 dataset, respectively. The visualization analysis also indicates that the attention modules can be easily embedded into DCNN networks, especially in the parallel mode, with stronger generality and universality.

Graph layouts reveal global or local structures of graph data. However, there are few studies on assisting readers in better reconstructing a graph from a layout. This paper attempts to generate a layout whose edges can be reestablished. We reformulate the graph layout problem as an edge classification problem. The inputs are the vertex pairs, and the outputs are the edge existences. The trainable parameters are the laid-out coordinates of the vertices. We propose a binary classification-based graph layout (BCGL) framework in this paper. This layout aims to preserve the local structure of the graph and does not require the total similarity relationships of the vertices. We implement two concrete algorithms under the BCGL framework, evaluate our approach on a wide variety of datasets, and draw comparisons with several other methods. The evaluations verify the ability of the BCGL in local neighborhood preservation and its visual quality with some classic metrics.

OpenFlow is a widely adopted implementation of the Software-Defined Networking (SDN) architecture. Since conventional network monitoring systems are unable to cope with OpenFlow networks, researchers have developed various monitoring systems tailored for OpenFlow networks. However, these existing systems either rely on a specific controller framework or an API, both of which are not part of the OpenFlow specification, and thus limit their applicability. This article proposes a transparent and low-overhead monitoring system for OpenFlow networks, referred to as Opimon. Opimon monitors the network topology, switch statistics, and flow tables in an OpenFlow network and visualizes the result through a web interface in real-time. Opimon monitors a network by interposing a proxy between the controller and switches and intercepting every OpenFlow message exchanged. This design allows Opimon to be compatible with any OpenFlow switch or controller. We tested the functionalities of Opimon on a virtual network built using Mininet and a large-scale international OpenFlow testbed (PRAGMA-ENT). Furthermore, we measured the performance overhead incurred by Opimon and demonstrated that the overhead in terms of latency and throughput was less than 3% and 5%, respectively.

The binary quadratic knapsack problem (QKP) aims at optimizing a quadratic cost function within a single knapsack. Its applications and difficulty make it appealing for various industrial fields. In this paper we present an efficient strategy to solve the problem by modeling it as an Ising spin model using an Ising machine to search for its ground state which translates to the optimal solution of the problem. Secondly, in order to facilitate the search, we propose a novel technique to visualize the landscape of the search and demonstrate how difficult it is to solve QKP on an Ising machine. Finally, we propose two software solution improvement algorithms to efficiently solve QKP on an Ising machine.

The Tokyo subway is one of the most complex subway networks in the world and it is difficult to compute a visually readable metro map using existing layout methods. In this paper, we present a new method that can generate complex metro maps such as the Tokyo subway network. Our method consists of two phases. The first phase generates rough metro maps. It decomposes the metro networks into smaller subgraphs and partially generates rough metro maps. In the second phase, we use a local search technique to improve the aesthetic quality of the rough metro maps. The experimental results including the Tokyo metro map are shown.

Outlier detection in a data set is very important in performing proper data mining. In this paper, we propose a method for efficiently detecting outliers by performing cluster analysis using the DS algorithm improved from the k-means algorithm. This method is simpler to detect outliers than traditional methods, and these detected outliers can quantitatively indicate “the degree of outlier”. Using this method, we detect abnormal trading days from OHLCs for S&P500 and FTSA, which are typical and world-wide stock indexes, from the beginning of 2005 to the end of 2015. They are defined as non-steady trading days, and the conditions for becoming the non-steady markets are mined as new knowledge. Applying the mined knowledge to OHLCs from the beginning of 2016 to the end of 2018, we can predict the non-steady trading days during that period. By verifying the predicted content, we show the fact that the appropriate knowledge has been successfully mined and show the effectiveness of the outlier detection method proposed in this paper. Furthermore, we mutually reference and comparatively analyze the results of applying this method to multiple stock indexes. This analyzes possible to visualize when and where social and economic impacts occur and how they propagate through the earth. This is one of the new applications using this method.

Power line communications (PLC) is a communication technology that uses a power-line as a transmission medium. Previous studies have shown that connecting an AC adapter such as a mobile phone charger to the power-line affects signal quality. Therefore, in this paper, the authors analyze the influence of chargers on inter-computer communications using packet capture to evaluate communications quality. The analysis results indicate the occurrence of a short duration in which packets are not detected once in a half period of the power-line supply: named communication forbidden time. For visualizing the communication forbidden time and for evaluating the communications quality of the inter-computer communications using PLC, the authors propose an instantaneous power-line frequency synchronized superimposed chart and its plotting algorithm. Further, in order to analyze accurately, the position of the communication forbidden time can be changed by altering the initial burst signal plotting position. The difference in the chart, which occurs when the plotting start position changes, is also discussed. We show analysis examples using the chart for a test bed data assumed an ideal environment, and show the effectiveness of the chart for analyzing PLC inter-computer communications.

We propose a novel compositing pipeline and a dynamic load balancing technique for volume rendering which utilizes a two-layered group structure to achieve effective and scalable load balancing. The technique enables each process to render data from non-contiguous regions of the volume with minimal impact on the total render time. We demonstrate the effectiveness of the proposed technique by performing a set of experiments on a modern GPU cluster. The experiments show that using the technique results in up to a 35.7% lower worst-case memory usage as compared to a dynamic k-d tree load balancing technique, whilst simultaneously achieving similar or higher render performance. The proposed technique was also able to lower the amount of transferred data during the load balancing stage by up to 72.2%. The technique has the potential to be used in many scenarios where other dynamic load balancing techniques have proved to be inadequate, such as during large-scale visualization.

Heat map is an important tool for eye tracking data analysis and visualization. It is very intuitive to express the area watched by observer, but ignores saccade information that expresses gaze shift. Based on conventional heat map generation method, this paper presents a novel heat map generation method for eye tracking data. The proposed method introduces a mixed data structure of fixation points and saccades, and considers heat map deformation for saccade type data. The proposed method has advantages on indicating gaze transition direction while visualizing gaze region.

Defects in spacecraft software may result in loss of life and serious economic damage. To avoid such consequences, the software development process incorporates code review activity. A code review conducted by a third-party organization independently of a software development team can effectively identify defects in software. However, such review activity is difficult for third-party reviewers, because they need to understand the entire structure of the code within a limited time and without prior knowledge. In this study, we propose a tool to visualize inter-module dataflow for source code of spacecraft software systems. To evaluate the method, an autonomous rover control program was reviewed using this visualization. While the tool does not decreases the time required for a code review, the reviewers considered the visualization to be effective for reviewing code.

Spatio-temporal dependent data, such as weather observation data, are data of which the attribute values depend on both time and space. Typical methods for the visualization of such data include plotting the attribute values at each point in time on a map and displaying series of the maps in chronological order with animation, or displaying them by juxtaposing horizontally or vertically. However, these methods are problematic in that they compel readers interested in grasping the spatial changes of the attribute values to memorize the representations on the maps. The problem is exacerbated by considering that the longer the time-period covered by the data, the higher the cognitive load. In order to solve these problems, the authors propose a visualization method capable of overlaying the representations of multiple instantaneous values on a single static map. This paper explains the design of the proposed method and reports two experiments conducted by the authors to investigate the usefulness of the method. The experimental results show that the proposed method is useful in terms of the speed and accuracy with which it reads the spatial changes and its ability to present data with long time series efficiently.

This paper proposes a method to extract and visualize a library update history in a project. The method identifies reused library versions by comparing source code in a product with existing versions of the library so that developers can understand when their own copy of a library has been copied, modified, and updated.

Studies have shown that program visualization(PV) is effective for student programming exercise or self-study support. However, very few instructors actively use PV tools for programming lectures. This article discussed the impediments the instructors meet during combining PV tools into lecture classrooms and proposed a C programming classroom instruction support tool based on program visualization — PROVIT-CI (PROgram VIsualization Tool for Classroom Instruction). PROVIT-CI has been consecutively and actively used by the instructors in author's university to enhance their lectures since 2015. The evaluation of application results in an introductory C programming course shows that PROVIT-CI is effective and helpful for instructors classroom use.

A novel dimensionality reduction method, Fisher Discriminant Locality Preserving Canonical Correlation Analysis (FDLP-CCA), for visualizing Web images is presented in this paper. FDLP-CCA can integrate two modalities and discriminate target items in terms of their semantics by considering unique characteristics of the two modalities. In this paper, we focus on Web images with text uploaded on Social Networking Services for these two modalities. Specifically, text features have high discriminate power in terms of semantics. On the other hand, visual features of images give their perceptual relationships. In order to consider both of the above unique characteristics of these two modalities, FDLP-CCA estimates the correlation between the text and visual features with consideration of the cluster structure based on the text features and the local structures based on the visual features. Thus, FDLP-CCA can integrate the different modalities and provide separated manifolds to organize enhanced compactness within each natural cluster.

Biomimetics is a new research field that creates innovation through the collaboration of different existing research fields. However, the collaboration, i.e., the exchange of deep knowledge between different research fields, is difficult for several reasons such as differences in technical terms used in different fields. In order to overcome this problem, we have developed a new retrieval platform, “Biomimetics image retrieval platform,” using a visualization-based image retrieval technique. A biological database contains a large volume of image data, and by taking advantage of these image data, we are able to overcome limitations of text-only information retrieval. By realizing such a retrieval platform that does not depend on technical terms, individual biological databases of various species can be integrated. This will allow not only the use of data for the study of various species by researchers in different biological fields but also access for a wide range of researchers in fields ranging from materials science, mechanical engineering and manufacturing. Therefore, our platform provides a new path bridging different fields and will contribute to the development of biomimetics since it can overcome the limitation of the traditional retrieval platform.

This paper presents a novel defense scheme for DDoS attacks that uses an image processing method. This scheme especially focused on the prevalence of adjacent neighbor spoofing, called subnet spoofing. It is rarely studied and there is few or no feasible approaches than other spoofing attacks. The key idea is that a “DDoS attack with IP spoofing” is represented as a specific pattern such as a “line” on the spatial image planes, which can be recognized through an image processing technique. Applying the clustering technique to the lines makes it possible to identify multiple attack source networks simultaneously. For the identified networks in which the zombie hosts reside, we then employ a signature-based pattern extraction algorithm, called a pivoted movement, and the DDoS attacks are filtered by correlating the IP and media access control pairing signature. As a result, this proposed scheme filters attacks without disturbing legitimate traffic. Unlike previous IP traceback schemes such as packet marking and path fingerprinting, which try to diagnose the entire attack path, our proposed scheme focuses on identifying only the attack source. Our approach can achieve an adaptive response to DDoS attacks, thereby mitigating them at the source, while minimizing the disruption of legitimate traffic. The proposed scheme is analyzed and evaluated on the IPv4 and IPv6 network topology from CAIDA, the results of which show its effectiveness.

Blood pressure measurement by auscultatory method is a compulsory skill that is required by all healthcare practitioners. During the measurement, they must concentrate on recognizing the Korotkoff sounds, looking at the sphygmomanometer scale, and constantly deflating the cuff pressure simultaneously. This complex operation is difficult for the new learners and they need a lot of practice with the supervisor in order to guide them on their measurements. However, the supervisor is not always available and consequently, they always face the problem of lack of enough training. In order to help them mastering the skill of measuring blood pressure by auscultatory method more efficiently and effectively, we propose using a sensor device to capture the signals of Korotkoff sounds and cuff pressure during the measurement, and display the signal changes on a visualization tool through wireless connection. At the end of the measurement, the learners can verify their skill on deflation speed and recognition of Korotkoff sounds using the graphical view, and compare their measurements with the machine instantly. By using this device, the new learners do not need to wait for their supervisor for training but can practice with their colleagues more frequently. As a result, they will be able to acquire the skill in a shorter time and be more confident with their measurements.

Some unilateral lower-limb amputees, have through continued exertion, increase the foot reaction force of the sound leg. The asymmetric gait with a prosthetic leg may thus negatively affect the musculoskeletal health of the leg on the healthy side. Therefore, it is important for these amputees to learn how to adjust the balance of each foot load in training. The aim of this study is to develop a training support system visualizing floor-reaction forces using a color-depth sensor. The pose of the entire body of the amputee is measured by the depth sensor, and the floor reaction force is estimated based on Zero Moment Point (ZMP), which is calculated using the center of mass of the amputee. Evaluation experiments of the proposed method were performed and they confirmed the effectiveness of the estimation method and the training with the visualization of reaction force.

Isosurface extraction is one of the most popular techniques for visualizing scalar volume data. However, volume data contains infinitely many isosurfaces. Furthermore, a single isosurface might contain many connected components, or contours, with each representing a different object surface. Hence, it is often a tedious and time-consuming manual process to find and extract contours that are interesting to users. This paper describes a novel method for automatically extracting salient contours from volume data. For this purpose, we propose a contour gradient tree (CGT) that contains the information of salient contours and their saliency magnitude. We organize the CGT in a hierarchical way to generate a sequence of contours in saliency order. Our method was applied to various medical datasets. Experimental results show that our method can automatically extract salient contours that represent regions of interest in the data.

Once a software product has been released, a large number of software products may be derived from an original single product. Management and maintenance of product variants are important, but those are hardly cared because developers do not make efforts for the further maintainability in the initial phase of software development. However, history of products would be lost in typical cases and developers have only source code of products in the worst case. In this paper, we approximate the evolution history of software products using source code of them. Our key idea is that two successive products are the most similar pair of products in evolution history, and have many similar source files. We did an experiment to compare the analysis result with actual evolution history. The result shows 78% (on average) of edges in the extracted trees are consistent with the actual evolution history of the products.