A Mathematical Solution to a Network Designing Problem

Yoshikane TAKAHASHI

A Mathematical Solution to a Network Designing Problem

Yoshikane TAKAHASHI

Full Text Views

0

Cite this

Summary :

One of the major open issues in neural network research includes a Network Designing Problem (NDP): find a polynomial-time procedure that produces minimal structures (the minimum intermediate size, thresholds and synapse weights) of multilayer threshold feed-forward networks so that they can yield outputs consistent with given sample sets of input-output data. The NDP includes as a sub-problem a Network Training Problem (NTP) where the intermediate size is given. The NTP has been studied mainly by use of iterative algorithms of network training. This paper, making use of both rate distortion theory in information theory and linear algebra, solves the NDP mathematically rigorously. On the basis of this mathematical solution, it furthermore develops a mathematical solution Procedure to the NDP that computes the minimal structure straightforwardly from the sample set. The Procedure precisely attains the minimum intermediate size, although its computational time complexity can be of non-polynomial order at worst cases. The paper also refers to a polynomial-time shortcut to the Procedure for practical use that can reach an approximately minimum intermediate size with its error measurable. The shortcut, when the intermediate size is pre-specified, reduces to a promising alternative as well to current network training algorithms to the NTP.

Publication: IEICE TRANSACTIONS on Fundamentals Vol.E78-A No.10 pp.1381-1411

Publication Date: 1995/10/25

Publicized

Online ISSN

DOI

Type of Manuscript: PAPER

Category: Neural Networks

Cite this

Copy

Yoshikane TAKAHASHI, "A Mathematical Solution to a Network Designing Problem" in IEICE TRANSACTIONS on Fundamentals, vol. E78-A, no. 10, pp. 1381-1411, October 1995, doi: .
Abstract: One of the major open issues in neural network research includes a Network Designing Problem (NDP): find a polynomial-time procedure that produces minimal structures (the minimum intermediate size, thresholds and synapse weights) of multilayer threshold feed-forward networks so that they can yield outputs consistent with given sample sets of input-output data. The NDP includes as a sub-problem a Network Training Problem (NTP) where the intermediate size is given. The NTP has been studied mainly by use of iterative algorithms of network training. This paper, making use of both rate distortion theory in information theory and linear algebra, solves the NDP mathematically rigorously. On the basis of this mathematical solution, it furthermore develops a mathematical solution Procedure to the NDP that computes the minimal structure straightforwardly from the sample set. The Procedure precisely attains the minimum intermediate size, although its computational time complexity can be of non-polynomial order at worst cases. The paper also refers to a polynomial-time shortcut to the Procedure for practical use that can reach an approximately minimum intermediate size with its error measurable. The shortcut, when the intermediate size is pre-specified, reduces to a promising alternative as well to current network training algorithms to the NTP.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/e78-a_10_1381/_p

Copy

@ARTICLE{e78-a_10_1381,
author={Yoshikane TAKAHASHI, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={A Mathematical Solution to a Network Designing Problem},
year={1995},
volume={E78-A},
number={10},
pages={1381-1411},
abstract={One of the major open issues in neural network research includes a Network Designing Problem (NDP): find a polynomial-time procedure that produces minimal structures (the minimum intermediate size, thresholds and synapse weights) of multilayer threshold feed-forward networks so that they can yield outputs consistent with given sample sets of input-output data. The NDP includes as a sub-problem a Network Training Problem (NTP) where the intermediate size is given. The NTP has been studied mainly by use of iterative algorithms of network training. This paper, making use of both rate distortion theory in information theory and linear algebra, solves the NDP mathematically rigorously. On the basis of this mathematical solution, it furthermore develops a mathematical solution Procedure to the NDP that computes the minimal structure straightforwardly from the sample set. The Procedure precisely attains the minimum intermediate size, although its computational time complexity can be of non-polynomial order at worst cases. The paper also refers to a polynomial-time shortcut to the Procedure for practical use that can reach an approximately minimum intermediate size with its error measurable. The shortcut, when the intermediate size is pre-specified, reduces to a promising alternative as well to current network training algorithms to the NTP.},
keywords={},
doi={},
ISSN={},
month={October},}

Copy

TY - JOUR
TI - A Mathematical Solution to a Network Designing Problem
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 1381
EP - 1411
AU - Yoshikane TAKAHASHI
PY - 1995
DO -
JO - IEICE TRANSACTIONS on Fundamentals
SN -
VL - E78-A
IS - 10
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - October 1995
AB - One of the major open issues in neural network research includes a Network Designing Problem (NDP): find a polynomial-time procedure that produces minimal structures (the minimum intermediate size, thresholds and synapse weights) of multilayer threshold feed-forward networks so that they can yield outputs consistent with given sample sets of input-output data. The NDP includes as a sub-problem a Network Training Problem (NTP) where the intermediate size is given. The NTP has been studied mainly by use of iterative algorithms of network training. This paper, making use of both rate distortion theory in information theory and linear algebra, solves the NDP mathematically rigorously. On the basis of this mathematical solution, it furthermore develops a mathematical solution Procedure to the NDP that computes the minimal structure straightforwardly from the sample set. The Procedure precisely attains the minimum intermediate size, although its computational time complexity can be of non-polynomial order at worst cases. The paper also refers to a polynomial-time shortcut to the Procedure for practical use that can reach an approximately minimum intermediate size with its error measurable. The shortcut, when the intermediate size is pre-specified, reduces to a promising alternative as well to current network training algorithms to the NTP.
ER -