Reinforcement Learning (RL) is an efficient method for solving Markov Decision Processes (MDPs) without a priori knowledge about an environment, and can be classified into the exploitation oriented method and the exploration oriented method. Q-learning is a representative RL and is classified as an exploration oriented method. It is guaranteed to obtain an optimal policy, however, Q-learning needs numerous trials to learn it because there is not action-selecting mechanism in Q-learning. For accelerating the learning rate of the Q-learning and realizing exploitation and exploration at a learning process, the Q-ee learning system has been proposed, which uses pre-action-selector, action-selector and back propagation of Q values to improve the performance of Q-learning. But the Q-ee learning is merely suitable for deterministic MDPs, and its convergent guarantee to derive an optimal policy has not been proved. In this paper, based on discussing different exploration methods, replacing the pre-action-selector in the Q-ee learning, we introduce a method that can be used to implement an active exploration to an environment, the Active Exploration Planning (AEP), into the learning system, which we call the Q-ae learning. With this replacement, the Q-ae learning not only maintains advantages of the Q-ee learning but also is adapted to a stochastic environment. Moreover, under deterministic MDPs, this paper presents the convergent condition and its proof for an agent to obtain the optimal policy by the method of the Q-ae learning. Further, by discussions and experiments, it is shown that by adjusting the relation between the learning factor and the discounted rate, the exploration process to an environment can be controlled on a stochastic environment. And, experimental results about the exploration rate to an environment and the correct rate of learned policies also illustrate the efficiency of the Q-ae learning on the stochastic environment.

MF (medium frequency) radars (MFR) are powerful tools for understanding the upper atmosphere, by measuring horizontal wind velocity and electron density. This article introduces three MFR systems, two in Japan, Yamagawa (31.20N, 130.62E) and Wakkanai (45.36N, 141.81E) radars, and one at Poker Flat, Alaska (65.1N, 147.5W). Experimental techniques, and their observed results are briefly shown. Horizontal wind velocity was observed by those MFRs, in height ranges of 60-100 km (day) and 80-100 km (night) at Yamagawa and Wakkanai, while the data coverage is unusually low, >54 km (day) and >68 km (night), at Poker Flat. Comparison of MFR winds with temperature observed by a collocated Rayleigh lidar at Poker Flat shows consistency of those two instrument results in terms of atmospheric wave theory, implying validity of MFR data at such low altitudes. Electron density results at Poker Flat agree reasonably with International Reference Ionosphere model values at 74-84 km, and agree well with variation of cosmic noise absorption by the Poker Flat imaging riometer, suggesting valid electron density estimation by MFR at least below 80-85 km.

Fourier analysis of sinusoidal and/or quasi-periodic signals in additive noise has been used in various fields. So far, many analysis algorithms including the well-known DFT have been developed. In particular, many adaptive algorithms have been proposed to handle non-stationary signals whose discrete Fourier coefficients (DFCs) are time-varying. Notch Fourier Transform (NFT) and Constrained Notch Fourier Transform(CNFT) proposed by Tadokoro et al. and Kilani et al., respectively, are two of them, which are implemented by filter banks and estimate the DFCs via simple sliding algorithms of their own. This paper presents, for the first time, statistical performance analyses of the NFT and the CNFT. Estimation biases and mean square errors (MSEs) of their sliding algorithms will be derived in closed form. As a result, it is revealed that both algorithms are unbiased, and their estimation MSEs are related to the signal frequencies, the additive noise variance and orders of comb filters used in their filter banks. Extensive simulations are performed to confirm the analytical findings.

An optimization of the smoothing preprocessing for the correlated signal parameter estimation was considered. Although the smoothing factor (the number of subarrays) is a free parameter in the smoothing preprocessing, a useful strategy to determine it has not yet been established. In this paper, we investigated thoroughly about the smoothing factor and also proposed a new scheme to optimize it. The proposed method, using the smoothed equivalent diversity profile (SED profile), is able to evaluate the effect of smoothing preprocessing without any a priori information. Therefore, this method is applicable in the real multipath parameter estimation.

This paper presents a novel competitive learning algorithm for the design of variable-rate vector quantizers (VQs). The algorithm, termed variable-rate competitive learning (VRCL) algorithm, designs a VQ having minimum average distortion subject to a rate constraint. The VRCL performs the weight vector training in the wavelet domain so that required training time is short. In addition, the algorithm enjoys a better rate-distortion performance than that of other existing VQ design algorithms and competitive learning algorithms. The learning algorithm is also more insensitive to the selection of initial codewords as compared with existing design algorithms. Therefore, the VRCL algorithm can be an effective alternative to the existing variable-rate VQ design algorithms for the applications of signal compression.

In this paper, we describe a method of determining meat quality using the concepts of "marbling score" and texture analysis. The marbling score is a measure of the density distribution of fat in the rib-eye region. Based on the results of an investigation carried out by handing out questionnaires to graders, we consider the marbling of meat to be a texture pattern and propose a method for the implementation of a grading system using a texture feature. In this system, we use a gray level co-occurrence matrix as the texture feature, which is a typical second-order statistic of gray levels of a texture image, and determine standard texture-feature vectors for each grade based on it. The grade of an unevaluated image is determined by comparing the texture-feature vector of this unevaluated image with the standard texture-feature vectors. Experimental results show the proposed method to be effective.

The interconnect analysis of on- and off-chips is very important in the design of high-speed signal processing, digital communication, and microwave electronic systems. When the interconnects are characterized by sampled data via electromagnetic analysis, the circuit-level simulation of the network requires rational approximation of the sampled data. Since the frequency band of the sampled data is more than 10 GHz, the rational function must fit into it at many frequency points. The rational function is approximated using the orthogonal least-squares method. With an increase in the number of the fitting data, the least-squares method suffers from a singularity problem. To avoid this, the sampled data are hierarchically approximated in this paper. Moreover, to reduce the computational cost of the circuit-level simulation, the parameter matrix of the interconnects is approximated by a rational matrix with one common denominator polynomial, and the selective orthogonalization procedure is presented.

This paper presents a three-dimensional polarimetric detection result of targets buried in snowpack by synthetic aperture FM-CW radar system. Since the FM-CW radar is suitable for short range sensing and can be equipped with fully polarimetric capability, we further extended it to a polarimetric three-dimensional SAR system. A field experiment was carried out to image and detect targets in a natural snowpack of 280 cm deep. The polarimetric detection and identification schemes are the polarimetric filtering, three-component decomposition, and the power polarization anisotropy coefficient. These approaches to acquired data show the usefulness of three-dimensional polarimetric FM-CW SAR system.

A constraint resolution scheme in the Hopfield-type neural network named "Neuron Filter" is presented for efficiently solving combinatorial optimization problems. The neuron filter produces an output that satisfies the constraints of the problem as best as possible according to both neuron inputs and outputs. This paper defines the neuron filter and shows its introduction into existing neural networks for N-queens problems and FPGA board-level routing problems. The performance is evaluated through simulations where the results show that our neuron filter improves the searching capability of the neural network with the shorter computation time.

In this paper, performance of a PN code acquisition scheme is analyzed and simulated for a DS/CDMA overlay system where a CDMA user and a narrowband user coexist in the same frequency band. A narrowband user is modelled as a narrowband interference (NBI) located at the fraction of a CDMA user's bandwidth. To suppress the NBI, an interference suppression filter is employed at the receiver frontend. Acquisition performance is evaluated in terms of mean acquisition time using state transition diagram for acquisition process. To apply for a DS/CDMA mobile cellular environments, multiple access interference and imperfection of power control are taken into account in the analysis of acquisition performance. The imperfect power control is considered by modelling the power of each user to be lognormally distributed about nominal received power. From the simulation results, it is shown that for the cases of perfect and imperfect power control, the interference suppression filter is very effective for supprssion of the NBI and rapid PN code acquisition in a DS/CDMA overlay environment. It is also shown that the one-sided tap number of 5 for interference suppression filter is sufficient to suppress the NBI. And, capacity estimates are compared based on acquisition and BER performance. The analysis in this paper can be applied to the practical situations for a DS/CDMA overlay environment.

We present several challenging gridding problems for multi-dimensional device and process simulation and discuss how new strategies might contribute to their solution. Formulating grid quality requirements for the standard Scharfetter-Gummel box method discretization in device simulation, we demonstrate how the offsetting techniques compares with quadtree grid generation methods and how they apply to modern device designs. Further we present a grid adaptation approach which respects the grid quality criteria and touch upon the main adaptation difficulties within device simulation. For the 3D moving boundary grids in process simulation we present a new algorithm.

In this paper, a new set of difference equations is derived for transient analysis of the convergence of adaptive FIR filters using the Sign-Sign Algorithm with Gaussian reference input and additive Gaussian noise. The analysis is based on the assumption that the tap weights are jointly Gaussian distributed. Residual mean squared error after convergence and simpler approximate difference equations are further developed. Results of experiment exhibit good agreement between theoretically calculated convergence and that of simulation for a wide range of parameter values of adaptive filters.

This paper proposes radio resource control scheme for ABR service that execute flow-control on the transmission rate and assignable bandwidth according to the congestion conditions in both wireless and wired networks. The proposed scheme is useful in improving frequency utilization and meeting the QoS requirements. There are two methods to realize the proposed scheme: explicit rate control (ERC) and binary control (BC). We estimate the performance of the proposed scheme by simulation in comparison with a scheme without flow control in a wireless network under the conditions of a finite buffer, wired network congestion, and RM-cell errors. Consequently, we confirm that the proposed scheme is more effective than the scheme without flow-control under all service conditions. In addition, we clarify that both ERC and BC are effective under the conditions of a finite buffer, wired network congestion, and RM-cell errors.

In this article, an efficient vector quantization (VQ) scheme called side-match finite-state vector quantization with adaptive block classification is presented for image compression. It makes use of edge information contained in image in additional to the average values of blocks forming the image. In order to achieve low bit rate coding while preserving good quality images, neighboring blocks are utilized to predict the class of current block. Image blocks are mainly classified as edge blocks and non-edge blocks in this coding scheme. To improve the coding efficiency, edge blocks and non-edge blocks are further reclassified into different classes, respectively. Moreover, the number of bits for encoding an image is greatly reduced by foretelling the class of input block and applying small state codebook in corresponding class. The improvement of the proposed coding scheme is attractive as compared with other VQ techniques.

Conventional multiuser interference canceling is always done in the uplink. In this paper, a novel multiuser interference canceling scheme is introduced. It can be used in the downlink of the TDD-CDMA systems. The multiuser interference canceling is done through the joint optimization of both the spreading and despreading waveform for different users, instead of the despreading code only. The idea is to take the advantages of exploitation, rather than mitigation, of the multipath propagation effects. In TDD-CDMA systems, the channel parameter of the downlink can be estimated from the uplink. We can calculate the spreading and despreading code by using the instantaneous channel parameters, so they can accommodate to the characteristic of the multipath channel. The maximum receive signal to interference plus noise ration (SINR) criterion is used to optimize the spreading and despreading code. An iteration algorithm is given to create the optimal spreading and despreading code. Numerical results show that our scheme can greatly decrease the multiple access interference (MAI), hence greatly increase the SINR, compared with the conventional spreading code such as Gold code. Numerical results of the proposed scheme assuming different number of active users, Doppler frequency and coefficient estimation error are also given.

In this paper, we propose a projection based design of near perfect reconstruction QMF banks. An advantage of this method is that additional design specifications are easily implemented by defining new convex sets. To apply convex projection technique, the main difficulty is how to approximate the design specifications by some closed convex sets. In this paper, introducing a notion of Magnitude Product Space where a pair of magnitude responses of analysis filters is expressed as a point, we approximate design requirements of QMF banks by multiple closed convex sets in this space. The proposed method iteratively applies a convex projection technique, Hybrid Steepest Descent Method, to find a point corresponding to the optimal analysis filters at each stage, where the closed convex sets are dynamically improved. Design examples show that the proposed design method leads to significant improvement over conventional design methods.

The Infostation concept has been proposed to provide convenient and cost effective access to high-speed mobile data services. An efficient IP packet transmission protocol is required to compensate for the high error rate inherent to fading radio channels. In this paper, a novel link layer retransmission scheme is proposed. Using the channel state and fading state estimators, the scheme adjusts the retransmission parameters dynamically in order to achieve the optimum performance under time-varying channel conditions. A theoretical analysis is presented for the case of a random error channel. Furthermore, a simulation tool is developed for evaluating the performance of the scheme in a fading channel with various parameters. The analysis and simulation results show that this new retransmission scheme can provide substantial improvement over traditional schemes. It gives a robust performance in both slow and fast fading conditions. In addition, the algorithm's sensitivity to parameter values and channel characteristics, such as Doppler frequency and fading statistics, is investigated. A unique attribute of this algorithm and performance analysis is that throughput is evaluated in IP packets rather than in physical layer packets.

In this paper the novel method of "weighted OFDM" is addressed. Different types of weighting factors (including Rectangular, Bartlett, Gaussian, Raised cosine, Half-sin and Shanon) are considered. The impact of weighting of OFDM on the peak-to-average power ratio (PAPR) is investigated by means of simulation and is compared for the above mentioned weighting factors. Results show that by weighting of the OFDM signal the PAPR reduces. Bit error performance of weighted multicarrier transmission over a multipath channel is also investigated. Results indicate that there is a trade off between PAPR reduction and bit error performance degradation by weighting.

This paper presents an accelerated image halftoning technique using an improved genetic algorithm with tiny populations. The algorithm is based on a new cooperative model for genetic operators in GA. Two kinds of operators are used in parallel to produce offspring: (i) SRM (Self-Reproduction with Mutation) to introduce diversity by means of Adaptive Dynamic-Block (ADB) mutation inducing the appearance of beneficial mutations. (ii) CM (Crossover and Mutation) to promote the increase of beneficial mutations in the population. SRM applies qualitative mutation only to the bits inside a mutation block and controls the required exploration-exploitation balance through its adaptive mechanism. An extinctive selection mechanism subjects SRM's and CM's offspring to compete for survival. The simulation results show that our scheme impressively reduces computer memory and processing time required to obtain high quality halftone images. For example, compared to the conventional image halftoning technique with GA, the proposed algorithm using only a 2% population size required about 15% evaluations to generate high quality images. The results make our scheme appealing for practical implementations of the image halftoning technique using GA.

We compare the numerical results for electron direct tunneling currents for single gate oxides, ON- and ONO-structures. We demonstrate that stacked dielectrics can keep the tunneling currents a few orders of magnitude lower than electrostatically equivalent single oxides. We also discuss the impact of gate material and of the modeling of electron transport in silicon.