This paper develops the double generating function method for the discrete-time linear quadratic optimal control problem. This method can give generators for optimal solutions only in terms of pre-computed coefficients and boundary conditions, which is useful for the on-line repetitive computation for different boundary conditions. Moreover, since each generator contains inverse terms, the invertibility analysis is also performed to conclude that the terms in the generators constructed by double generating functions with opposite time directions are invertible under some mild conditions, while the terms with the same time directions will become singular when the time goes infinity which may cause instability in numerical computations. Examples demonstrate the effectiveness of the developed method.

Recently the data-driven learning of dynamic systems has become a promising approach because no physical knowledge is needed. Pure machine learning approaches such as Gaussian process regression (GPR) learns a dynamic model from data, with all physical knowledge about the system discarded. This goes from one extreme, namely methods based on optimizing parametric physical models derived from physical laws, to the other. GPR has high flexibility and is able to model any dynamics as long as they are locally smooth, but can not generalize well to unexplored areas with little or no training data. The analytic physical model derived under assumptions is an abstract approximation of the true system, but has global generalization ability. Hence the optimal learning strategy is to combine GPR with the analytic physical model. This paper proposes a method to learn dynamic systems using GPR with analytic ordinary differential equations (ODEs) as prior information. The one-time-step integration of analytic ODEs is used as the mean function of the Gaussian process prior. The total parameters to be trained include physical parameters of analytic ODEs and parameters of GPR. A novel method is proposed to simultaneously learn all parameters, which is realized by the fully Bayesian GPR and more promising to learn an optimal model. The standard Gaussian process regression, the ODE method and the existing method in the literature are chosen as baselines to verify the benefit of the proposed method. The predictive performance is evaluated by both one-time-step prediction and long-term prediction. By simulation of the cart-pole system, it is demonstrated that the proposed method has better predictive performances.

Particle filters have been widely used for state estimation problems in nonlinear and non-Gaussian systems. Their performance depends on the given system and measurement models, which need to be designed by the user for each target system. This paper proposes a novel method to design these models for a particle filter. This is a numerical optimization method, where the particle filter design process is interpreted into the framework of reinforcement learning by assigning the randomnesses included in both models of the particle filter to the policy of reinforcement learning. In this method, estimation by the particle filter is repeatedly performed and the parameters that determine both models are gradually updated according to the estimation results. The advantage is that it can optimize various objective functions, such as the estimation accuracy of the particle filter, the variance of the particles, the likelihood of the parameters, and the regularization term of the parameters. We derive the conditions to guarantee that the optimization calculation converges with probability 1. Furthermore, in order to show that the proposed method can be applied to practical-scale problems, we design the particle filter for mobile robot localization, which is an essential technology for autonomous navigation. By numerical simulations, it is demonstrated that the proposed method further improves the localization accuracy compared to the conventional method.

This paper develops the generating function method for the discrete-time nonlinear optimal control problem. This method can analytically give the optimal input as state feedforward control in terms of the generating functions. Since the generating functions are nonlinear, we also develop numerical implementations to find their Taylor series expressions. This finally gives optimal solutions expressed only in terms of the pre-computed generating function coefficients and state boundary conditions, such that it is useful for the on-demand optimal solution generation for different boundary conditions. Examples demonstrate the effectiveness of the developed method.

Recently, control theory using machine learning, which is useful for the control of unknown systems, has attracted significant attention. This study focuses on such a topic with optimal control problems for unknown nonlinear systems. Because optimal controllers are designed based on mathematical models of the systems, it is challenging to obtain models with insufficient knowledge of the systems. Kernel functions are promising for developing data-driven models with limited knowledge. However, the complex forms of such kernel-based models make it difficult to design the optimal controllers. The design corresponds to solving Hamilton-Jacobi (HJ) equations because their solutions provide optimal controllers. Therefore, the aim of this study is to derive certain kernel-based models for which the HJ equations are solved in an exact sense, which is an extended version of the authors' former work. The HJ equations are decomposed into tractable algebraic matrix equations and nonlinear functions. Solving the matrix equations enables us to obtain the optimal controllers of the model. A numerical simulation demonstrates that kernel-based models and controllers are successfully developed.