The solution of the ordinary kernel ridge regression, based on the squared loss function and the squared norm-based regularizer, can be easily interpreted as a stochastic linear estimator by considering the autocorrelation prior for an unknown true function. As is well known, a stochastic affine estimator is one of the simplest extensions of the stochastic linear estimator. However, its corresponding kernel regression problem is not revealed so far. In this paper, we give a formulation of the kernel regression problem, whose solution is reduced to a stochastic affine estimator, and also give interpretations of the formulation.

In signal restoration problems, we expect to improve the restoration performance with a priori information about unknown target signals. In this paper, the parametric Wiener filter with linear constraints for unknown target signals is discussed. Since the parametric Wiener filter is usually defined as the minimizer of the criterion not for the unknown target signal but for the filter, it is difficult to impose constraints for the unknown target signal in the criterion. To overcome this difficulty, we introduce a criterion for the parametric Wiener filter defined for the unknown target signal whose minimizer is equivalent to the solution obtained by the original formulation. On the basis of the newly obtained criterion, we derive a closed-form solution for the parametric Wiener filter with linear constraints.

In terms of the formulation of the optimality, image restoration filters can be divided into two streams. One is formulated as an optimization problem in which the fidelity of a restored image is indirectly evaluated, and the other is formulated as an optimization problem based on a direct evaluation. Originally, the formulation of the optimality and the solutions derived from the formulation are identical each other. However in many studies adopting the former stream, an arbitrary choice of a solution without a mathematical ground passes unremarked. In this paper, we discuss the relation between the formulation of the optimality and the solution derived from the formulation from a mathematical point of view, and investigate the relation between a direct style formulation and an indirect one. Through these analyses, we show that the both formulations yield the identical filter in practical situations.

In this paper, we discuss the problem of active training data selection for improving the generalization capability of a neural network. We look at the learning problem from a function approximation perspective and formalize it as an inverse problem. Based on this framework, we analytically derive a method of choosing a training data set optimized with respect to the Wiener optimization criterion. The final result uses the apriori correlation information on the original function ensemble to devise an efficient sampling scheme which, when used in conjunction with the learning scheme described here, is shown to result in optimal generalization. This result is substantiated through a simulated example and a learning problem in high dimensional function space.