High frame rate and ultra-low delay vision system, which can finish reading and processing of 1000fps sequence within 1ms/frame, draws increasing attention in the field of robotics that requires immediate feedback from image process core. Meanwhile, tracking task plays an important role in many computer vision applications. Among various tracking algorithms, Lucas Kanade (LK)-based template tracking, which tracks targets with high accuracy over the sub-pixel level, is one of the keys for robotic applications, such as factory automation (FA). However, the substantial spatial iterative processing and complex computation in the LK algorithm, make it difficult to achieve a high frame rate and ultra-low delay tracking with limited resources. Aiming at an LK-based template tracking system that reads and processes 1000fps sequences within 1ms/frame with small resource costs, this paper proposes: 1) High temporal resolution-based temporal iterative tracking, which maps the spatial iterations into the temporal domain, efficiently reduces resource cost and delay caused by spatial iterative processing. 2) Label scanner-based multi-stream spatial processing, which maps the local spatial processing into the labeled input pixel stream and aggregates them with a label scanner, makes the local spatial processing in the LK algorithm possible be implemented with a small resource cost. Algorithm evaluation shows that the proposed temporal iterative tracking performs dynamic tracking, which tracks object with coarse accuracy when it's moving fast and achieves higher accuracy when it slows down. Hardware evaluation shows that the proposed label scanner-based multi-stream architecture makes the system implemented on FPGA (zcu102) with resource cost less than 20%, and the designed tracking system supports to read and process 1000fps sequence within 1ms/frame.

Although deep neural networks (DNNs) have achieved high performance across a variety of applications, they can often be deceived by adversarial examples that are generated by adding small perturbations to the original images. Adversaries may generate adversarial examples using the property of transferability, in which adversarial examples that deceive one model can also deceive other models because adversaries do not obtain any information on the DNNs deployed in real scenarios. Recent studies show that adversarial examples with feature space perturbations are more transferable than others. Adversarial training is an effective method to defend against adversarial attacks. However, it results in a decrease in the classification accuracy for natural images, and it is not sufficiently robust against transferable adversarial examples because it does not consider adversarial examples with feature space perturbations. We propose a novel adversarial training method to train DNNs to be robust against transferable adversarial examples and maximize their classification accuracy for natural images. The proposed method trains DNNs to correctly classify natural images and adversarial examples and also minimize the feature differences between them. The robustness of the proposed method was similar to those of the previous adversarial training methods for MNIST dataset and was up to average 6.13% and 9.24% more robust against transfer adversarial examples for CIFAR-10 and CIFAR-100 datasets, respectively. In addition, the proposed method yielded an average classification accuracy that was approximately 0.53%, 6.82%, and 10.60% greater than some state-of-the-art adversarial training methods for all datasets, respectively. The proposed method is robust against a variety of transferable adversarial examples, which enables its implementation in security applications that may benefit from high-performance classification but are at high risk of attack.

Speech separation is the task of extracting target speech while suppressing background interference components. In applications like video telephones, visual information about the target speaker is available, which can be leveraged for multi-speaker speech separation. Most previous multi-speaker separation methods are mainly based on convolutional or recurrent neural networks. Recently, Transformer-based Seq2Seq models have achieved state-of-the-art performance in various tasks, such as neural machine translation (NMT), automatic speech recognition (ASR), etc. Transformer has showed an advantage in modeling audio-visual temporal context by multi-head attention blocks through explicitly assigning attention weights. Besides, Transformer doesn't have any recurrent sub-networks, thus supporting parallelization of sequence computation. In this paper, we propose a novel speaker-independent audio-visual speech separation method based on Transformer, which can be flexibly applied to unknown number and identity of speakers. The model receives both audio-visual streams, including noisy spectrogram and speaker lip embeddings, and predicts a complex time-frequency mask for the corresponding target speaker. The model is made up by three main components: audio encoder, visual encoder and Transformer-based mask generator. Two different structures of encoders are investigated and compared, including ResNet-based and Transformer-based. The performance of the proposed method is evaluated in terms of source separation and speech quality metrics. The experimental results on the benchmark GRID dataset show the effectiveness of the method on speaker-independent separation task in multi-talker environments. The model generalizes well to unseen identities of speakers and noise types. Though only trained on 2-speaker mixtures, the model achieves reasonable performance when tested on 2-speaker and 3-speaker mixtures. Besides, the model still shows an advantage compared with previous audio-visual speech separation works.

Translation of long and complex sentence has always been a challenge for machine translation. In recent years, neural machine translation (NMT) has achieved substantial progress in modeling the semantic connection between words in a sentence, but it is still insufficient in capturing discourse structure information between clauses within complex sentences, which often leads to poor discourse coherence when translating long and complex sentences. On the other hand, the hypotactic structure, a main component of the discourse structure, plays an important role in the coherence of discourse translation, but it is not specifically studied. To tackle this problem, we propose a novel Chinese-English NMT approach that incorporates the hypotactic structure knowledge of complex sentences. Specifically, we first annotate and build a hypotactic structure aligned parallel corpus to provide explicit hypotactic structure knowledge of complex sentences for NMT. Then we propose three hypotactic structure-aware NMT models with three different fusion strategies, including source-side fusion, target-side fusion, and both-side fusion, to integrate the annotated structure knowledge into NMT. Experimental results on WMT17, WMT18 and WMT19 Chinese-English translation tasks demonstrate that the proposed method can significantly improve the translation performance and enhance the discourse coherence of machine translation.

Fifth-generation (5G) mobile communication systems employ beamforming technology using massive multiple-input and multiple-output (MIMO) to improve the reception quality and spectrum efficiency within a cell. Meanwhile, coordinated beamforming among multiple base stations is an effective approach to improving the spectrum efficiency at the cell edges, in which massive MIMO is deployed at geographically distant base stations and beamforming control is conducted in a cooperative manner. Codebook-based beamforming is a method for realizing multi-cell coordinated beamforming, in which each base station selects one of multiple beams that are predefined in a codebook. In codebook-based beamforming, it is important to design an efficient codebook that takes into account the beam allocation and the number of beams. In general, the larger the number of beams defined in a codebook, the more finely tuned the beam control can be and a greater improvement in spectrum efficiency can be expected. However, it requires a huge signal processing to optimize the beam combinations with a large number of beams by coordinated beamforming. This paper proposes a novel codebook design that efficiently assigns beam directions and widths in a vertical plane. Computer simulations showed that the proposed codebook performs as well as the conventional method while requiring fewer beam combinations.

Cow behavior monitoring is critical for understanding the current state of cow welfare and developing an effective planning strategy for pasture management, such as early detection of disease and estrus. One of the most powerful and cost-effective methods is a neural-network-based monitoring system that analyzes time series data from inertial sensors attached to cows. For this method, a significant challenge is to improve the quality and quantity of teaching data in the development of neural network models, which requires us to collect data that can cover various realistic conditions and assign labels to them. As a result, the cost of data collection is significantly high. This work proposes a data augmentation method to solve two major quality problems in the collection process of teaching data. One is the difficulty and randomicity of teaching data acquisition and the other is the sensor position changes during actual operation. The proposed method can computationally emulate different rotating states of the collar-type sensor device from the measured acceleration data. Furthermore, it generates data for actions that occur less frequently. The verification results showed significantly higher estimation performance with an average accuracy of over 98% for five main behaviors (feeding, walking, drinking, rumination, and resting) based on learning with long short-term memory (LSTM) network. Compared with the estimation performance without data augmentation, which was insufficient with a minimum of 60.48%, the recognition rate was improved by 2.52-37.05pt for various behaviors. In addition, comparison of different rotation intervals was investigated and a 30-degree increment was selected based on the accuracy performances analysis. In conclusion, the proposed data expansion method can improve the accuracy in cow behavior estimation by a neural network model. Moreover, it contributes to a significant reduction of the teaching data collection cost for machine learning and opens many opportunities for new research.

This paper presents a deep network based on morphological filters for Gaussian denoising. The morphological filters can be applied with only addition, max, and min functions and require few computational resources. Therefore, the proposed network is suitable for implementation using a small microprocessor. Each layer of the proposed network consists of a top-hat transform, which extracts small peaks and valleys of noise components from the input image. Noise components are iteratively reduced in each layer by subtracting the noise components from the input image. In this paper, the extensions of opening and closing are introduced as linear combinations of the morphological filters for the top-hat transform of this deep network. Multiplications are only required for the linear combination of the morphological filters in the proposed network. Because almost all parameters of the network are structuring elements of the morphological filters, the feature maps and parameters can be represented in short bit-length integer form, which is suitable for implementation with single instructions, multiple data (SIMD) instructions. Denoising examples show that the proposed network obtains denoising results comparable to those of BM3D [1] without linear convolutions and with approximately one tenth the number of parameters of a full-scale deep convolutional neural network [2]. Moreover, the computational time of the proposed method using SIMD instructions of a microprocessor is also presented.

Reducing the power consumption of capsule endoscopy is essential for its further development. We introduce K-SVD dictionary learning to design a dictionary for sparse coding, and improve reconstruction accuracy of capsule endoscopic images captured using compressed sensing. At a compression ratio of 20%, the proposed method improves image quality by approximately 4.4 dB for the peak signal-to-noise ratio.

High-altitude platform stations (HAPSs) are recognized as a promising technology for coverage extension in the sixth generation (6G) mobile communications and beyond. The purpose of this study is to develop a HAPS system with a coverage radius of 100km and high capacity by focusing on the following two aspects: array antenna structure and user selection. HAPS systems must jointly use massive multiple-input multiple-output (mMIMO) and multiuser MIMO techniques to increase their capacity. However, the coverage achieved by a conventional planar array antenna is limited to a circular area with a radius of only tens of kilometers. A conventional semi-orthogonal user selection (SUS) scheme based on the orthogonality of channel vectors achieves high capacity, but it has high complexity. First, this paper proposes a cylindrical mMIMO system to achieve an ultra-wide coverage radius of 100km and high capacity. Second, this paper presents a novel angle-based user selection (AUS) scheme, where a user selection problem is formulated as a maximization of the minimum angular difference between users over all user groups. Finally, a low-complexity suboptimal algorithm (SA) for AUS is also proposed. Assuming an area with a 100km radius, simulation results demonstrate that the proposed cylindrical mMIMO system improves the signal-to-interference-plus-noise ratio by approx. 12dB at the boundary of the area, and it achieves approx. 1.5 times higher capacity than the conventional mMIMO which uses a planar array antenna. In addition, the results show that the proposed AUS scheme improves the lower percentiles in the system capacity distribution compared with SUS and basic random user selection. Furthermore, the computational complexity of the proposed SA is in the order of only 1/4000 that of SUS.

OpenFlow is a widely adopted implementation of the Software-Defined Networking (SDN) architecture. Since conventional network monitoring systems are unable to cope with OpenFlow networks, researchers have developed various monitoring systems tailored for OpenFlow networks. However, these existing systems either rely on a specific controller framework or an API, both of which are not part of the OpenFlow specification, and thus limit their applicability. This article proposes a transparent and low-overhead monitoring system for OpenFlow networks, referred to as Opimon. Opimon monitors the network topology, switch statistics, and flow tables in an OpenFlow network and visualizes the result through a web interface in real-time. Opimon monitors a network by interposing a proxy between the controller and switches and intercepting every OpenFlow message exchanged. This design allows Opimon to be compatible with any OpenFlow switch or controller. We tested the functionalities of Opimon on a virtual network built using Mininet and a large-scale international OpenFlow testbed (PRAGMA-ENT). Furthermore, we measured the performance overhead incurred by Opimon and demonstrated that the overhead in terms of latency and throughput was less than 3% and 5%, respectively.

To sustain and expand the agricultural economy even as its workforce shrinks, the efficiency of farm operations must be improved. One key to efficiency improvement is completely unmanned driving of farm machines, which requires stable monitoring and control of machines from remote sites, a safety system to ensure safe autonomous driving even without manual operations, and precise positioning in not only small farm fields but also wider areas. As possible solutions for those issues, we have developed technologies of wireless network quality prediction, an end-to-end overlay network, machine vision for safety and positioning, network cooperated vehicle control and autonomous tractor control and conducted experiments in actual field environments. Experimental results show that: 1) remote monitoring and control can be seamlessly continued even when connection between the tractor and the remote site needs to be switched across different wireless networks during autonomous driving; 2) the safety of the autonomous driving can automatically be ensured by detecting both the existence of people in front of the unmanned tractor and disturbance of network quality affecting remote monitoring operation; and 3) the unmanned tractor can continue precise autonomous driving even when precise positioning by satellite systems cannot be performed.

Linear codes with a few-weight have important applications in combinatorial design, strongly regular graphs and cryptography. In this paper, we first construct a class of Boolean functions with at most five-valued Walsh spectra, and determine their spectrum distribution. Then, we derive two classes of linear codes with at most six-weight from the new functions. Meanwhile, the length, dimension and weight distributions of the codes are obtained. Results show that both of the new codes are minimal and among them, one is wide minimal code and the other is a narrow minimal code and thus can be used to design secret sharing scheme with good access structures. Finally, some Magma programs are used to verify the correctness of our results.

A method to generate color palettes from images is proposed. Here, deep neural networks (DNN) are utilized in order to consider human perception. Two aspects of human perception are considered; one is attention to image, and the other is human preference for colors. This method first extracts N regions with dominant color categories from the image considering human attention. Here, N is the number of colors in a color palette. Then, the representative color is obtained from each region considering the human preference for color. Two deep neural-net systems are adopted here, one is for estimating the image area which attracts human attention, and the other is for estimating human preferable colors from image regions to obtain representative colors. The former is trained with target images obtained by an eye tracker, and the latter is trained with dataset of color selection by human. Objective and subjective evaluation is performed to show high performance of the proposed system compared with conventional methods.

The narrow bandwidth limitation of 300-3400Hz on the public switching telephone network results in speech quality deterioration. In this paper, we propose an artificial bandwidth extension approach that reconstructs the missing lower bandwidth of 50-300Hz using sinusoidal synthesis based on the first formant location. Sinusoidal synthesis generates sinusoidal waves with a harmonic structure. The proposed method detects the fundamental frequency using an autocorrelation method based on YIN algorithm, where a threshold processing avoids the false fundamental frequency detection on unvoiced sounds. The amplitude of the sinusoidal waves is calculated in the time domain from the weighted energy of 300-600Hz. In this case, since the first formant location corresponds to the first peak of the spectral envelope, we reconstruct the harmonic structure to avoid attenuating and overemphasizing by increasing the weight when the first formant location is lower, and vice versa. Consequently, the subjective and objective evaluations show that the proposed method reduces the speech quality difference between the original speech signal and the bandwidth extended speech signal.

Estimating the spatial parameters of the signals by using the effective data of a single snapshot is essential in the field of reconnaissance and confrontation. Major drawback of existing algorithms is that its constructed covariance matrix has a great degree of rank loss. The performance of existing algorithms gets degraded with low signal-to-noise ratio. In this paper, a three-parallel linear array based algorithm is proposed to achieve two-dimensional direction of arrival estimates in a single snapshot scenario. The key points of the proposed algorithm are: 1) construct three pseudo matrices with full rank and no rank loss by using the single snapshot data from the received signal model; 2) by using the rotation relation between pseudo matrices, the matched 2D-DOA is obtained with an efficient parameter matching method. Main objective of this work is on improving the angle estimation accuracy and reducing the loss of degree of freedom in single snapshot 2D-DOA estimation.

Computer networks tend to be subjected to the proliferation of mobile demands, therefore it poses a great challenge to guarantee the quality of network service. For real-time systems, the QoS performance bound analysis for the complex network topology and background traffic in modern networks is often difficult. Network calculus, nevertheless, converts a complex non-linear network system into an analyzable linear system to accomplish more accurate delay bound analysis. The existing network environment contains complex network resource allocation schemes, and delay bound analysis is generally pessimistic, hence it is essential to modify the analysis model to improve the bound accuracy. In this paper, the main research approach is to obtain the measurement results of an actual network by building a measurement environment and the corresponding theoretical results by network calculus. A comparison between measurement data and theoretical results is made for the purpose of clarifying the scheme of bandwidth scheduling. The measurement results and theoretical analysis results are verified and corrected, in order to propose an accurate per-flow end-to-end delay bound analytic model for a large-scale scheduling network. On this basis, the instructional significance of the analysis results for the engineering construction is discussed.

For beyond 5G era, in network function virtualization (NFV) environments, service chaining can be utilized to provide the flexible network infrastructures needed to support the creation of various application services. In this paper, we propose a dynamic service chain construction based on model predictive control (MPC) to utilize network resources. In the proposed method, the number of data packets in the buffer at each node is modeled as a dynamical system for MPC. Then, we formulate an optimization problem with the predicted amount of traffic injecting into each service chain from users for the dynamical system. In the optimization problem, the transmission route of each service chain, the node where each VNF is placed, and the amount of resources for each VNF are determined simultaneously by using MPC so that the amount of resources allocated to VNFs and the number of VNF migrations are minimized. In addition, the performance of data transmission is also controlled by considering the maximum amount of data packets stored in buffers. The performance of the proposed method is evaluated by simulation, and the effectiveness of the proposed method with different parameter values is investigated.

To reduce the complexity of direct radio frequency (RF) undersampling real-time spectrum monitoring in wireless Internet of Things (IoT) bands (920MHz, 2.4GHz, and 5 GHz bands), a design method of sampling frequencies is proposed in this paper. The Direct RF Undersampling receiver architecture enables the use of ADC with sampling clock lower frequency than receiving RF signal, but it needs RF signal identification signal processing from folded spectrums with multiple sampling clock frequencies. The proposed design method allows fewer sampling frequencies to be used than the conventional design method for continuous frequency range (D.C. to 5GHz-band). The proposed method reduced 2 sampling frequencies in wireless IoT bands case compared with the continuous range. The design result using the proposed method is verified by measurement.

A periodic array of InSb spheres on a substrate is numerically analyzed at terahertz frequencies. The incident field is shown to be coupled to the substrate due to the guided-mode resonance. The effect of the background refractive index on the transmission characteristics is investigated for sensor applications.

Volume integral equations combined with orthogonality of guided mode and non-guided field are proposed for the TE incidence of two-dimensional optical slab waveguide. The slab waveguide is assumed to satisfy the single mode condition. The formulation of the integral equations are described in detail. The matrix equation obtained by applying the method of moments to the integral equations is shown. Numerical results for step, gap, and grating waveguides are given. They are compared to published papers to validate the proposed method.