Optimal Regularization for System Identification from Noisy Input and Output Signals

Jingmin XIN; Hiromitsu OHMORI; Akira SANO

Optimal Regularization for System Identification from Noisy Input and Output Signals

Jingmin XIN, Hiromitsu OHMORI, Akira SANO

Full Text Views

0

Cite this

Summary :

In identification of a finite impulse response (FIR) model using noise-corrupted input and output data, the least squares type of estimation schemes such as the ordinary least squares (LS), the corrected least squares (CLS) and the total least squares (TLS) method become often numerically unstable, when the true input signal to the system is strongly correlated. To overcome this ill-conditioned problem, we propose a regularized CLS estimation method by introducing multiple regularization parameters to minimize the mean squares error (MSE) of the regularized CLS estimate of the FIR model. The asymptotic MSE can be evaluated by considering the third and fourth order cross moments of the input and output measurement noises, and an analytical expression of the optimal regularization parameters minimizing the MSE is also clarified. Furthermore, an effective regularization algorithm is given by using the only accessible input-output data without using any true unknown parameters. The effectiveness of the proposed data-based regularization algorithm is demonstrated and compared with the ordinary LS, CLS and TLS estimates through numerical examples.

Publication: IEICE TRANSACTIONS on Fundamentals Vol.E78-A No.12 pp.1805-1815

Publication Date: 1995/12/25

Publicized

Online ISSN

DOI

Type of Manuscript: PAPER

Category: Digital Signal Processing

Cite this

Copy

Jingmin XIN, Hiromitsu OHMORI, Akira SANO, "Optimal Regularization for System Identification from Noisy Input and Output Signals" in IEICE TRANSACTIONS on Fundamentals, vol. E78-A, no. 12, pp. 1805-1815, December 1995, doi: .
Abstract: In identification of a finite impulse response (FIR) model using noise-corrupted input and output data, the least squares type of estimation schemes such as the ordinary least squares (LS), the corrected least squares (CLS) and the total least squares (TLS) method become often numerically unstable, when the true input signal to the system is strongly correlated. To overcome this ill-conditioned problem, we propose a regularized CLS estimation method by introducing multiple regularization parameters to minimize the mean squares error (MSE) of the regularized CLS estimate of the FIR model. The asymptotic MSE can be evaluated by considering the third and fourth order cross moments of the input and output measurement noises, and an analytical expression of the optimal regularization parameters minimizing the MSE is also clarified. Furthermore, an effective regularization algorithm is given by using the only accessible input-output data without using any true unknown parameters. The effectiveness of the proposed data-based regularization algorithm is demonstrated and compared with the ordinary LS, CLS and TLS estimates through numerical examples.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/e78-a_12_1805/_p

Copy

@ARTICLE{e78-a_12_1805,
author={Jingmin XIN, Hiromitsu OHMORI, Akira SANO, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Optimal Regularization for System Identification from Noisy Input and Output Signals},
year={1995},
volume={E78-A},
number={12},
pages={1805-1815},
abstract={In identification of a finite impulse response (FIR) model using noise-corrupted input and output data, the least squares type of estimation schemes such as the ordinary least squares (LS), the corrected least squares (CLS) and the total least squares (TLS) method become often numerically unstable, when the true input signal to the system is strongly correlated. To overcome this ill-conditioned problem, we propose a regularized CLS estimation method by introducing multiple regularization parameters to minimize the mean squares error (MSE) of the regularized CLS estimate of the FIR model. The asymptotic MSE can be evaluated by considering the third and fourth order cross moments of the input and output measurement noises, and an analytical expression of the optimal regularization parameters minimizing the MSE is also clarified. Furthermore, an effective regularization algorithm is given by using the only accessible input-output data without using any true unknown parameters. The effectiveness of the proposed data-based regularization algorithm is demonstrated and compared with the ordinary LS, CLS and TLS estimates through numerical examples.},
keywords={},
doi={},
ISSN={},
month={December},}

Copy

TY - JOUR
TI - Optimal Regularization for System Identification from Noisy Input and Output Signals
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 1805
EP - 1815
AU - Jingmin XIN
AU - Hiromitsu OHMORI
AU - Akira SANO
PY - 1995
DO -
JO - IEICE TRANSACTIONS on Fundamentals
SN -
VL - E78-A
IS - 12
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - December 1995
AB - In identification of a finite impulse response (FIR) model using noise-corrupted input and output data, the least squares type of estimation schemes such as the ordinary least squares (LS), the corrected least squares (CLS) and the total least squares (TLS) method become often numerically unstable, when the true input signal to the system is strongly correlated. To overcome this ill-conditioned problem, we propose a regularized CLS estimation method by introducing multiple regularization parameters to minimize the mean squares error (MSE) of the regularized CLS estimate of the FIR model. The asymptotic MSE can be evaluated by considering the third and fourth order cross moments of the input and output measurement noises, and an analytical expression of the optimal regularization parameters minimizing the MSE is also clarified. Furthermore, an effective regularization algorithm is given by using the only accessible input-output data without using any true unknown parameters. The effectiveness of the proposed data-based regularization algorithm is demonstrated and compared with the ordinary LS, CLS and TLS estimates through numerical examples.
ER -