Convolutional Neural Networks (CNNs) have recently demonstrated outstanding performance in image retrieval tasks. Local convolutional features extracted by CNNs, in particular, show exceptional capability in discrimination. Recent research in this field has concentrated on pooling methods that incorporate local features into global features and assess the global similarity of two images. However, the pooling methods sacrifice the image's local region information and spatial relationships, which are precisely known as the keys to the robustness against occlusion and viewpoint changes. In this paper, instead of pooling methods, we propose an alternative method based on local similarity, determined by directly using local convolutional features. Specifically, we first define three forms of local similarity tensors (LSTs), which take into account information about local regions as well as spatial relationships between them. We then construct a similarity CNN model (SCNN) based on LSTs to assess the similarity between the query and gallery images. The ideal configuration of our method is sought through thorough experiments from three perspectives: local region size, local region content, and spatial relationships between local regions. The experimental results on a modified open dataset (where query images are limited to occluded ones) confirm that the proposed method outperforms the pooling methods because of robustness enhancement. Furthermore, testing on three public retrieval datasets shows that combining LSTs with conventional pooling methods achieves the best results.

Pose estimation is a research hot spot in computer vision tasks and the key to computer perception of human activities. The core concept of human pose estimation involves describing the motion of the human body through major joint points. Large receptive fields and rich spatial information facilitate the keypoint localization task, and how to capture features on a larger scale and reintegrate them into the feature space is a challenge for pose estimation. To address this problem, we propose a multi-scale convergence network (MSCNet) with a large receptive field and rich spatial information. The structure of the MSCNet is based on an hourglass network that captures information at different scales to present a consistent understanding of the whole body. The multi-scale receptive field (MSRF) units provide a large receptive field to obtain rich contextual information, which is then selectively enhanced or suppressed by the Squeeze-Excitation (SE) attention mechanism to flexibly perform the pose estimation task. Experimental results show that MSCNet scores 73.1% AP on the COCO dataset, an 8.8% improvement compared to the mainstream CMUPose method. Compared to the advanced CPN, the MSCNet has 68.2% of the computational complexity and only 55.4% of the number of parameters.

The finger-vein-based deep neural network authentication system has been applied widely in real scenarios, such as countries' banking and entrance guard systems. However, to ensure performance, the deep neural network should train many parameters, which needs lots of time and computing resources. This paper proposes a method that introduces artificial features with prior knowledge into the convolution layer. First, it designs a multi-direction pattern base on the traditional local binary pattern, which extracts general spatial information and also reduces the spatial dimension. Then, establishes a sample effective deep convolutional neural network via combination with convolution, with the ability to extract deeper finger vein features. Finally, trains the model with a composite loss function to increase the inter-class distance and reduce the intra-class distance. Experiments show that the proposed methods achieve a good performance of higher stability and accuracy of finger vein recognition.

Cross-modal hashing technology has attracted much attention for its favorable retrieval performance and low storage cost. However, for existing cross-modal hashing methods, the heterogeneity of data across modalities is still a challenge and how to fully explore and utilize the intra-modality features has not been well studied. In this paper, we propose a novel cross-modal hashing approach called Modality-fused Graph Network (MFGN). The network architecture consists of a text channel and an image channel that are used to learn modality-specific features, and a modality fusion channel that uses the graph network to learn the modality-shared representations to reduce the heterogeneity across modalities. In addition, an integration module is introduced for the image and text channels to fully explore intra-modality features. Experiments on two widely used datasets show that our approach achieves better results than the state-of-the-art cross-modal hashing methods.

This letter proposes a novel lightweight deep learning object detector named LW-YOLOv4-tiny, which incorporates the convolution block feature addition module (CBFAM). The novelty of LW-YOLOv4-tiny is the use of channel-wise convolution and element-wise addition in the CBFAM instead of utilizing the concatenation of different feature maps. The model size and computation requirement are reduced by up to 16.9 Mbytes, 5.4 billion FLOPs (BFLOPS), and 11.3 FPS, which is 31.9%, 22.8%, and 30% smaller and faster than the most recent version of YOLOv4-tiny. From the MSCOCO2017 and PASCAL VOC2012 benchmarks, LW-YOLOv4-tiny achieved 40.2% and 69.3% mAP, respectively.

The acquisition of distant sound has always been a hot research topic. Since sound is caused by vibration, one of the best methods for measuring distant sound is to use a laser Doppler vibrometer (LDV). This laser has high directivity, that enables it to acquire sound from far away, which is of great practical use for disaster relief and other situations. However, due to the vibration characteristics of the irradiated object itself and the reflectivity of its surface (or other reasons), the acquired sound is often lacking frequency components in certain frequency bands and is mixed with obvious noise. Therefore, when using LDV to acquire distant speech, if we want to recognize the actual content of the speech, it is necessary to enhance the acquired speech signal in some way. Conventional speech enhancement methods are not generally applicable due to the various types of degradation in observed speech. Moreover, while several speech enhancement methods for LDV have been proposed, they are only effective when the irradiated object is known. In this paper, we present a speech enhancement method for LDV that can deal with unknown irradiated objects. The proposed method is composed of noise reduction, pitch detection, power spectrum envelope estimation, power spectrum reconstruction, and phase estimation. Experimental results demonstrate the effectiveness of our method for enhancing the acquired speech with unknown irradiated objects.

To accelerate research on Beyond 5G (B5G) technologies in Japan, we propose an algorithm that designs mesh-type metropolitan area network (MAN) models based on a priori Japanese regional railway information, because ground-truth communication network information is unavailable. Instead, we use the information of regional railways, which is expected to express the necessary geometric structure of our metropolitan cities while remaining strongly correlated with their population densities and demographic variations. We provide an additional compression algorithm for use in reducing a small-scale network model from the original MAN model designed using the proposed algorithm. Two Tokyo MAN models are created, and we provide day and night variants for each while highlighting the number of passengers alighting/boarding at each station and the respective population densities. The validity of the proposed algorithm is verified through comparisons with the Japan Photonic Network model and another model designed using the communication network information, which is not ground-truth. Comparison results show that our proposed algorithm is effective for designing MAN models and that our result provides a valid Tokyo MAN model.

A drone swarm is a robotic architecture having multiple drones cooperate to accomplish a mission. Nowadays, heterogeneous drone swarms, in which a small number of gateway drones (GWs) act as protocol translators to enable the mixing of multiple swarms that use independent wireless protocols, have attracted much attention from many researchers. Our previous work proposed Path Optimizer — a method to minimize the number of end-to-end path-hops in a remote video monitoring system using heterogeneous drone swarms by autonomously relocating GWs to create a shortcut in the network for each communication request. However, Path Optimizer has limitations in improving communication quality when more video sessions than the number of GWs are requested simultaneously. Path Coordinator, which we propose in this paper, achieves a uniform reduction in end-to-end hops and maximizes the allowable hop satisfaction rate regardless of the number of sessions by introducing the cooperative and synchronous relocation of all GWs. Path Coordinator consists of two phases: first, physical optimization is performed by geographically relocating all GWs (relocation phase), and then logical optimization is achieved by modifying the relaying GWs of each video flow (rerouting phase). Computer simulations reveal that Path Coordinator adapts to various environments and performs as well as we expected. Furthermore, its performance is comparable to the upper limits possible with brute-force search.

Automatic modulation recognition (AMR) plays a critical role in modern communication systems. Owing to the recent advancements of deep learning (DL) techniques, the application of DL has been widely studied in AMR, and a large number of DL-AMR algorithms with high recognition rates have been developed. Most DL-AMR algorithm models have high recognition accuracy but have numerous parameters and are huge, complex models, which make them hard to deploy on resource-constrained platforms, such as satellite platforms. Some lightweight and low-complexity DL-AMR algorithm models also struggle to meet the accuracy requirements. Based on this, this paper proposes a lightweight and high-recognition-rate DL-AMR algorithm model called Lightweight Densely Connected Convolutional Network (DenseNet) Long Short-Term Memory network (LDLSTM). The model cascade of DenseNet and LSTM can achieve the same recognition accuracy as other advanced DL-AMR algorithms, but the parameter volume is only 1/12 that of these algorithms. Thus, it is advantageous to deploy LDLSTM in resource-constrained systems.

An approach to dedicated computers is discussed in this study as a possibility for portable, low-cost, and low-power consumption high-performance computing technologies. Particularly, dataflow architecture dedicated computer of the finite integration technique (FIT) for 2D magnetostatic field simulation is considered for use in industrial applications. The dataflow architecture circuit of the BiCG-Stab matrix solver of the FIT matrix calculation is designed by the very high-speed integrated circuit hardware description language (VHDL). The operation of the dedicated computer's designed circuit is considered by VHDL logic circuit simulation.

Influential user detection is critical in supporting the human facilitator-based facilitation in the online forum. Traditional approaches to detect influential users in the online forum focus on the statistical activity information such as the number of posts. However, statistical activity information cannot fully reflect the influence that users bring to the online forum. In this paper, we propose to detect the influencers from the influence propagation perspective and focus on the influential maximization (IM) problem which aims at choosing a set of users that maximize the influence propagation from the entire social network. An online forum influence propagation network (OFIPN) is proposed to model the influence from an individual user perspective and influence propagation between users, and a heuristic algorithm that is proposed to find influential users in OFIPN. Experiments are conducted by simulations with a real-world social network. Our empirical results show the effectiveness of the proposed algorithm.

As the Internet has become prevalent, the popularity of net media has been growing, to a point that it has taken over conventional mass media. However, TWtrends, the Twitter trends visualization system operated by our research team since 2019, indicates that many topics on TV programs frequently appear on Twitter trendlines. This study investigates the relationship between Twitter and TV programs by collecting information on Twitter trends and TV programs simultaneously. Although this study provides a rough estimation of the volume of tweets that mention TV programs, the results show that several tweets mention TV programs at a constant rate, which tends to increase on the weekend. This tendency of TV-related tweets stems from the audience rating survey results. Considering the study outcome, and the fact that many TV programs introduce topics popular in social media, implies codependency between Internet media (social media) and mass media.

We propose GConvLoc, a WiFi fingerprinting-based indoor localization method utilizing graph convolutional networks. Using the graph structure, we can consider the fingerprint data of the reference points and their location labels in addition to the fingerprint data of the test point at inference time. Experimental results show that GConvLoc outperforms baseline methods that do not utilize graphs.

In real world applications of multiclass classification models, misclassification in an important class (e.g., stop sign) can be significantly more harmful than in other classes (e.g., no parking). Thus, it is crucial to improve the recall of an important class while maintaining overall accuracy. For this problem, we found that improving the separation of important classes relative to other classes in the feature space is effective. Existing methods that give a class-sensitive penalty for cross-entropy loss do not improve the separation. Moreover, the methods designed to improve separations between all classes are unsuitable for our purpose because they do not consider the important classes. To achieve the separation, we propose a loss function that explicitly gives loss for the feature space, called class-sensitive additive angular margin (CAMRI) loss. CAMRI loss is expected to reduce the variance of an important class due to the addition of a penalty to the angle between the important class features and the corresponding weight vectors in the feature space. In addition, concentrating the penalty on only the important class hardly sacrifices separating the other classes. Experiments on CIFAR-10, GTSRB, and AwA2 showed that CAMRI loss could improve the recall of a specific class without sacrificing accuracy. In particular, compared with GTSRB's second-worst class recall when trained with cross-entropy loss, CAMRI loss improved recall by 9%.

We study a new transistor-level side-channel leakage caused by charges trapped in between stacked transistors namely residual electric charges (RECs). Building leakage models is important in designing countermeasures against side-channel attacks (SCAs). The conventional work showed that even a transistor-level leakage is measurable with a local electromagnetic measurement. One example is the current-path leak [1], [2]: an attacker can distinguish the number of transistors in the current path activated during a signal transition. Addressing this issue, Sugawara et al. proposed to use a mirror circuit that has the same number of transistors on its possible current paths. We show that this countermeasure is insufficient by showing a new transistor-level leakage, caused by RECs, not covered in the previous work. RECs can carry the history of the gate's state over multiple clock cycles and changes the gate's electrical behavior. We experimentally verify that RECs cause exploitable side-channel leakage. We also propose a countermeasure against REC leaks and designed advanced encryption standard-128 (AES-128) circuits using IO-masked dual-rail read-only memory with a 180-nm complementary metal-oxide-semiconductor (CMOS) process. We compared the resilience of our AES-128 circuits against EMA attacks with and without our countermeasure and investigated an RECs' effect on physically unclonable functions (PUFs). We further extend RECs to physically unclonable function. We demonstrate that RECs affect the performance of arbiter and ring-oscillator PUFs through experiments using our custom chips fabricated with 180- and 40-nm CMOS processes*.

The BGPsec protocol, which is an extension of the border gateway protocol (BGP) for Internet routing known as BGPsec, uses digital signatures to guarantee the validity of routing information. However, the use of digital signatures in routing information on BGPsec causes a lack of memory in BGP routers, creating a gaping security hole in today's Internet. This problem hinders the practical realization and implementation of BGPsec. In this paper, we present APVAS (AS path validation based on aggregate signatures), a new protocol that reduces the memory consumption of routers running BGPsec when validating paths in routing information. APVAS relies on a novel aggregate signature scheme that compresses individually generated signatures into a single signature. Furthermore, we implement a prototype of APVAS on BIRD Internet Routing Daemon and demonstrate its efficiency on actual BGP connections. Our results show that the routing tables of the routers running BGPsec with APVAS have 20% lower memory consumption than those running the conventional BGPsec. We also confirm the effectiveness of APVAS in the real world by using 800,000 routes, which are equivalent to the full route information on a global scale.

Proof of Work (PoW), which is a consensus algorithm for blockchain, entails a large number of meaningless hash calculations and wastage of electric power and computational resources. In 2021, it is estimated that the PoW of Bitcoin consumes as much electricity as Pakistan's annual power consumption (91TWh). This is a serious problem against sustainable development goals. To solve this problem, this study proposes Meaningful-PoW (mPoW), which involves a meaningful calculation, namely the application of a genetic algorithm (GA) to PoW. Specifically, by using the intermediate values that are periodically generated through GA calculations as an input to the Hashcash used in Bitcoin, it is possible to make this scheme a meaningful calculation (GA optimization problem) while maintaining the properties required for PoW. Furthermore, by applying a device-binding technology, mPoW can be ASIC resistant without the requirement of a large memory. Thus, we show that mPoW can reduce the excessive consumption of both power and computational resources.

The reliability of deep neural networks (DNN) against hardware errors is essential as DNNs are increasingly employed in safety-critical applications such as automatic driving. Transient errors in memory, such as radiation-induced soft error, may propagate through the inference computation, resulting in unexpected output, which can adversely trigger catastrophic system failures. As a first step to tackle this problem, this paper proposes constructing a vulnerability model (VM) with a small number of fault injections to identify vulnerable model parameters in DNN. We reduce the number of bit locations for fault injection significantly and develop a flow to incrementally collect the training data, i.e., the fault injection results, for VM accuracy improvement. We enumerate key features (KF) that characterize the vulnerability of the parameters and use KF and the collected training data to construct VM. Experimental results show that VM can estimate vulnerabilities of all DNN model parameters only with 1/3490 computations compared with traditional fault injection-based vulnerability estimation.

With the advent of the Internet of Things (IoT), short packet transmissions will dominate the future wireless communication. However, traditional coherent demodulation and channel estimation schemes require large pilot overhead, which may be highly inefficient for short packets in multipath fading scenarios. This paper proposes a novel pilot-free short packet structure based on the association of modulation on conjugate-reciprocal zeros (MOCZ) and tail-biting convolutional codes (TBCC), where a noncoherent demodulation and decoding scheme is designed without the channel state information (CSI) at the transceivers. We provide a construction method of constellation sets and demodulation rule for M-ary MOCZ. By deriving low complexity log-likelihood ratios (LLR) for M-ary MOCZ, we offer a reasonable balance between energy and bandwidth efficiency for joint coding and modulation scheme. Simulation results show that our proposed scheme can attain significant performance and throughput gains compared to the pilot-based coherent modulation scheme over multipath fading channels.

This paper proposes a sub-signal channel modulation scheme for hitless redundancy switching systems that offers highly confidential communications. A hitless redundancy switching system prevents frame loss by using multiple routes to forward the same frame. Although most studies on redundancy switching systems deal with frame duplication, elimination, and selection of redundant paths for the main signal, we focus on the transmission of the sub-signal channel. We introduce mathematical expressions to model the transmission rate and bit error rate of the sub-signal channel. To evaluate the validity of the models, we conduct numerical simulations to calculate the sub-signal transmission rate, main-signal transmission rate, and bit error rate of the sub-signal channel at physical transmission rates of 100Mb/s, 1Gb/s, and 10Gb/s. We discuss how to design sub-signal channel modulation on the basis of the evaluation results. We further discuss applications of sub-signal modulation in terms of network size and jitter.