Acceleration and Stabilization Techniques for the Levenberg-Marquardt Method
Summary :
In this paper, two techniques are proposed for accelerating and stabilizing the Levenberg-Marquardt (LM) method where its conventional stabilizer matrix (identity matrix) is superseded by (1) a diagonal matrix whose elements are column norms of Jacobian matrix J, or (2) a non-diagonal square root matrix of J TJ. Geometrically, these techniques make constraint conditions of the LM method fitted better to relevant cost function than conventional one. Results of numerical simulations show that proposed techniques are effective when both column norm ratio of J and mutual interactions between arguments of the cost function are large. Especially, the technique (2) introduces a new LM method of damped Gauss-Newton (GN) type which satisfies both properties of global convergence and quadratic convergence by controlling Marquardt factor and can stabilize convergence numerically. Performance of the LMM techniques are compared also with a damped GN method with line search procedure.
- Publication
- IEICE TRANSACTIONS on Fundamentals Vol.E88-A No.7 pp.1971-1978
- Publication Date
- 2005/07/01
- Publicized
- Online ISSN
- DOI
- 10.1093/ietfec/e88-a.7.1971
- Type of Manuscript
- PAPER
- Category
- Numerical Analysis and Optimization
Authors
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Hiroyasu SAKAMOTO, Katsuya MATSUMOTO, Azusa KUWAHARA, Yoshiteru HAYAMI, "Acceleration and Stabilization Techniques for the Levenberg-Marquardt Method" in IEICE TRANSACTIONS on Fundamentals,
vol. E88-A, no. 7, pp. 1971-1978, July 2005, doi: 10.1093/ietfec/e88-a.7.1971.
Abstract: In this paper, two techniques are proposed for accelerating and stabilizing the Levenberg-Marquardt (LM) method where its conventional stabilizer matrix (identity matrix) is superseded by (1) a diagonal matrix whose elements are column norms of Jacobian matrix J, or (2) a non-diagonal square root matrix of J TJ. Geometrically, these techniques make constraint conditions of the LM method fitted better to relevant cost function than conventional one. Results of numerical simulations show that proposed techniques are effective when both column norm ratio of J and mutual interactions between arguments of the cost function are large. Especially, the technique (2) introduces a new LM method of damped Gauss-Newton (GN) type which satisfies both properties of global convergence and quadratic convergence by controlling Marquardt factor and can stabilize convergence numerically. Performance of the LMM techniques are compared also with a damped GN method with line search procedure.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1093/ietfec/e88-a.7.1971/_p
Copy
@ARTICLE{e88-a_7_1971,
author={Hiroyasu SAKAMOTO, Katsuya MATSUMOTO, Azusa KUWAHARA, Yoshiteru HAYAMI, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Acceleration and Stabilization Techniques for the Levenberg-Marquardt Method},
year={2005},
volume={E88-A},
number={7},
pages={1971-1978},
abstract={In this paper, two techniques are proposed for accelerating and stabilizing the Levenberg-Marquardt (LM) method where its conventional stabilizer matrix (identity matrix) is superseded by (1) a diagonal matrix whose elements are column norms of Jacobian matrix J, or (2) a non-diagonal square root matrix of J TJ. Geometrically, these techniques make constraint conditions of the LM method fitted better to relevant cost function than conventional one. Results of numerical simulations show that proposed techniques are effective when both column norm ratio of J and mutual interactions between arguments of the cost function are large. Especially, the technique (2) introduces a new LM method of damped Gauss-Newton (GN) type which satisfies both properties of global convergence and quadratic convergence by controlling Marquardt factor and can stabilize convergence numerically. Performance of the LMM techniques are compared also with a damped GN method with line search procedure.},
keywords={},
doi={10.1093/ietfec/e88-a.7.1971},
ISSN={},
month={July},}
Copy
TY - JOUR
TI - Acceleration and Stabilization Techniques for the Levenberg-Marquardt Method
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 1971
EP - 1978
AU - Hiroyasu SAKAMOTO
AU - Katsuya MATSUMOTO
AU - Azusa KUWAHARA
AU - Yoshiteru HAYAMI
PY - 2005
DO - 10.1093/ietfec/e88-a.7.1971
JO - IEICE TRANSACTIONS on Fundamentals
SN -
VL - E88-A
IS - 7
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - July 2005
AB - In this paper, two techniques are proposed for accelerating and stabilizing the Levenberg-Marquardt (LM) method where its conventional stabilizer matrix (identity matrix) is superseded by (1) a diagonal matrix whose elements are column norms of Jacobian matrix J, or (2) a non-diagonal square root matrix of J TJ. Geometrically, these techniques make constraint conditions of the LM method fitted better to relevant cost function than conventional one. Results of numerical simulations show that proposed techniques are effective when both column norm ratio of J and mutual interactions between arguments of the cost function are large. Especially, the technique (2) introduces a new LM method of damped Gauss-Newton (GN) type which satisfies both properties of global convergence and quadratic convergence by controlling Marquardt factor and can stabilize convergence numerically. Performance of the LMM techniques are compared also with a damped GN method with line search procedure.
ER -
English
