Concatenated coding techniques are applied to slow frequency-hop packet radio communications for channels with partial-band interference. Binary orthogonal signaling (e.g., binary FSK) is employed with noncoherent demodulation. The outer codes are Reed-Solomon codes and the inner codes are convolutional codes. Two concatenated coding schemes are compared. The first employs an interleaver between the outer Reed-Solomon code and the inner convolutional code. The second scheme employs an additional interleaver following the convolutional code. Comparisons are made between the performance of these concatenated coding schemes and the performance of Reed-Solomon codes alone.

In this paper we consider multiuser detection using a neural network in a synchronous code-division multiple-access channel. In a code-division multiple-access channel, a matched filter is widely used as a receiver. However, when the relative powers of the interfering signals are large, i.e. the near-far problem, the performances of the matched filter receiver degrade. Although the optimum receiver for multiuser detection is superior to the matched filter receiver in such situations, the optimum receiver is too complex to be implemented. A simple technique to implement the optimum multiuser detection is required. Recurrent neural networks which consist of a number of simple processing units can rapidly provide a collectively-computed solution. Moreover, the network can seek out a minimum in the energy function. On the other hand, the optimum multiuser detection in a synchronous channel is carried out by the maximization of a likelihood function. In this paper, it is shown that the energy function of the neural network is identical to the likelihood function of the optimum multiuser detection and the neural network can be used to implement the optimum multiuser detection. Performance comparisons among the optimum receiver, the matched filter one and the neural network one are carried out by computer simulations. It is shown that the neural network receiver has a capability to achieve near-optimum performance in several situations and local minimum problems are few serious.

A new type of synchronous code division multiple access (S/CDMA) scheme for optical subscriber systems is reported. Passive channel multiplexing is promising for optical subscriber systems because it realizes high system performance at low cost. Unfortunately, passive channel multiplexing suffers from phase differences among the upstream channels, and these differences prevent the usage of traditional synchronous CDMA techniques that reduce cross channel interference. This paper proposes the new technique of block-interleaving & redundancy code sequences to overcome this problem. This combination realizes S/CDMA even in the presence of phase differences and eliminates cross channel interference completely. Therefore, in an optical subscriber system using the new type S/CDMA, the bit error rate performance is independent of phase difference levels and the number of multiplexed channels.

Three dimensional (3-D) optics offers potential advantages to the massively-parallel systems over electronics from the view point of information transfer. The purpose of this paper is to survey some aspects of the 3-D optical interconnection technology for the future massively-parallel computing systems. At first, the state-of-art of the current optoelectronic array devices to build the interconnection networks are described, with emphasis on those based on the semiconductor technology. Next, the principles, basic architectures, several examples of the 3-D optical interconnection systems in neural networks and multiprocessor systems are described. Finally, the issues that are needed to be solved for putting such technology into practical use are summarized.

In this paper we discuss a neural network model that can recognize patterns rotated at various angles. The model employs copy learning, a learning method entirely different from those used in conventional models. Copy-Learning is an effective learning method to attain the desired objective in a short period of time by making a copy of the result of basic learning through the application of certain rules. Our model using this method is capable of recognizing patterns rotated at various angles without requiring mathematical preprocessing. It involves two processes: first, it learns only the standard patterns by using part of the network. Then, it copies the result of the learning to the unused part of the network and thereby recognizes unknown input patterns by using all parts of the network. The model has merits over the conventional models in that it substantially reduces the time required for learning and recognition and can also recognize the rotation angle of the input pattern.

This paper describes the learning performance of the deterministic Boltzmann machine (DBM), which is a promising neural network model suitable for analog LSI implementation. (i) A new learning procedure suitable for LSI implementation is proposed. This is fully-on-line learning in which different sample patterns are presented in consecutive clamped and free phases and the weights are modified in each phase. This procedure is implemented without extra memories for learning operation, and reduces the chip area and power consumption for learning by 50 percent. (ii) Learning in a layer-type DBM with one output unit has characteristic local minima which reduce the effective number of available hidden units. Effective methods to avoid reaching these local minima are proposed. (iii) Although DBM learning is not suitable for mapping problems with analog target values, it is useful for analog data discrimination problems.

We propose a fully digital architecture for Kohonen network suitable for VLSI implementation. The proposed architecture adopts a functional memory type parallel processor (FMPP) architecture which has a structure similar to a content addressable memory (CAM). One word of CAM is regarded as a processing element and a group of elements forms a neuron. All processing elements execute the same operation in bit-serial but in processor-parallel. Thus the number of instructions for realizing the network algorithm is independent of the number of neurons in the network. With reference to a previously reported CAM, we estimate a network with 96 neurons for speech recognition could be integrated on three chips using a 1.2 µm process, and it operates 50 times faster than a sequential hardware. Owing to its highly regular structure of memories, the proposed hardware architecture is well compatible with current VLSI technology.

Digital neural networks are suitable for WSI implementation because their noise immunity is high, they have a fault tolerant structure, and the use of bus architecture can reduce the number of interconnections between neurons. To investigate the feasibility of WSIs, we integrated either 576 conventional neurons or 288 self-learning neurons on a 5-inch wafer, by using 0.8-µm CMOS technology and three metal layers. We also developed a new electron-beam direct-writing technology which enables easier fabrication of VLSI chips and wafer-level interconnections. We fabricated 288 self-learning neuron WSIs having as many as 230 good neurons.

To investigate necessary conditions for the object recognition by simulations using neural network models is one of ways to acquire suggestions for understanding the neuronal representation of objects in the brain. In the present study, we trained a three layered neural network to form a geometrical feature representation in its output layer using back-propagation algorithm. After training using 73 learning examples, 65 testing patterns made by various combinations of above features could be recognized with the network at a rate of 95.3% appropriate response. We could classify four types of hidden layer units on the basis of effects on the output layer.

We developed programmable high-performance and high-speed neurocomputer for a large neural network using ASIC neurocomputing chips made by CMOS VLSI technology. The neurocomputer consists of one master node and multiple slave nodes which are connected by two data paths, a broadcast bus and a ring bus. The nodes are made by ASIC chips and each chip has plural nodes in it. The node has four types of computation hardware that can be cascaded in series forming a pipeline. Processing speed is proportional to the number of nodes. The neurocomputer is built on one printed circuit board having 65 VLSI chips that offers 1.5 billion connections/sec. The neurocomputer uses SIMD for easy programming and simple hardware. It can execute complicated computations, memory access and memory address control, and data paths control in a single instruction and in a single time step using the pipeline. The neurocomputer processes forward and backward calculations of multilayer perceptron type neural networks, LVQ, feedback type neural networks such as Hopfield model, and any other types by programming. To compute neural computation effectively and simply in a SIMD type neurocomputer, new processing methods are proposed for parallel computation such as delayed instruction execution, and reconfiguration.

We have developed a digital coprocessor with a dynamically reconfigurable pipeline architecture specified for a layered neural network which executes on-chip learning. The coprocessor attains a learning speed of 18 MCUPS that is approximately twenty times that of the conventional DSP. This coprocessor obtains expansibility in the calculation through a larger multi-layer, network by means of a network decomposition and a distributed processing approach.

The advantages of a neuro-chip architecture based on a DRAM are demonstrated through a discussion of the general issuse regarding a memory based neuro-chip architecture and a comparison with a chip based on an SRAM. The performance of both chips is compared assuming digital operation, a 1.5-V supply voltage, a 106-synapse neural network capability, and a 0.5-µm CMOS design rule. The use of a one-transistor DRAM cell array for the storage of synapse weights results in a chip 55% smaller than an SRAM based chip with the same 8-Mbit memory capacity and the same number of processing elements. No additional operations for refreshing the DRAM cell array are necessary during the processing of the neural networks. This is because all the synapse weights in the array are transferred to the processing elements during the processing and the DRAM cells in the array are automatically refreshed when they are selected. The precharge operation of the DRAM cell array degrades the processing speed, however a processing speed of 1.37 GCPS is expected for the DRAM based chip. That speed is comparable to the 1.71 GCPS for the SRAM based chip with the same 256 parallel-processing elements. A DRAM cell array has the additional advantage of lower power dissipation in this specific usage for the neuro-chip. The dynamic operation of the DRAM cell array results in a 10% lower operating power dissipation than a chip using an SRAM cell array at the same processing speed of 1.37 GCPS. That lower operating power dissipation enables a DRAM based chip to run on a 1.5-V dry cell for longer under intermittent daily use even though the SRAM cell array has little power dissipation in data-holding mode.

A genetic method to generate a neural network which has both structure and connection weights adequate for a given task is proposed. A neural network having arbitrary connections is regarded as a virtual living thing which has genes representing its connections among neural units. Effectiveness of the network is estimated from its time sequential input and output signals. Excellent individuals, namely appropriate neural networks, are generated through generation iterations. The basic principle of the method and its applications are described. As an example of evolution from randomly generated networks to feedforward networks, an XOR problem is dealt with, and an action control problem is used for making networks containing feedback and mutual connections. The proposed method is available for designing a neural network whose adequate structure is unknown.

The self-organizing feature map is one of the most widely used neural network paradigm based on unsupervised competitive learning. However, the learning algorithm introduced by Kohonen is very slow when the size of the map is large. The slowness is caused by the search for large map in each training steps of the learning. In this paper, a fast learning algorithm based on incremental ordering is proposed. The new learning starts with only a few units evenly distributed on a large topological feature map, and gradually increases the number of units until it covers the entire map. In middle phases of the learning, some units are well-ordered and others are not, while all units are weekly-ordered in Kohonen learning. The ordered units, during the learning, help to accelerate the search speed of the algorithm and accelerate the movements of the remaining unordered units to their topological locations. It is shown by theoretical analysis as well as experimental analysis that the proposed learning algorithm reduces the training time from O(M2) to O(log M) for M by M map without any additional working space, while preserving the ordering properties of the Kohonen learning algorithm.

It has been reported that generalization performance of multilayer feedformard networks strongly depends on the attainment of saturated hidden outputs in response to the training set. Usually standard Backpropagation (BP) network mostly uses intermediate values of hidden units as the internal representation of the training patterns. In this letter, we propose construction of a 3-layer cascaded network in which two 2-layer networks are first trained independently by delta rule and then cascaded. After cascading, the intermediate layer can be viewed as hidden layer which is trained to attain preassigned saturated outputs in response to the training set. This network is particularly easier to construct for linearly separable training set, and can also be constructed for nonlinearly separable tasks by using higher order inputs at the input layer or by assigning proper codes at the intermediate layer which can be obtained from a trained Fahlman and Lebiere's network. Simulation results show that, at least, when the training set is linearly separable, use of the proposed cascaded network significantly enhances the generalization performance compared to BP network, and also maintains high generalization ability for nonlinearly separable training set. Performance of cascaded network depending on the preassigned codes at the intermediate layer is discussed and a suggestion about the preassigned coding is presented.

High-speed learning of neural networks using the multifrequency oscillation method is demonstrated for first time. During the learning of an analog neural network integrated circuit implementing the exclusive-OR' logic, weight and threshold values converge to steady states within 2 ms for a learning speed of 2 mega-patterns per second.

A learning procedure of a three layer neural network with limited structure, called a multi-valued neural network, is proposed. The three layer net has a single linear neuron in its output layer. All input weights of a number of hidden neurons are identical. The network takes k+1 distinct stable values, where k is the number of hidden neurons. The proposed learning procedure consists of two parts, Phase and Phase . The former is one for the learning of weights between the hidden and output layers, and the latter is one for those between the input and the hidden layers. The network is applied to classification of numerals, which shows the effectiveness of the proposed learning procedure.

The performance of processing elements can be improved by the progress of VLSI circuit technology, while the communication overhead can not be negligible in parallel processing system. This paper presents a unified scheduling that allocates tasks having different task processing times in multiple processing elements. The objective function is formulated to measure communication time between processing elements. By employing constraint conditions, the scheduling efficiently generates an optimal solution using an integer programming so that minimum communication time can be achieved. We also propose a VLSI processor for robotics whose latency is very small. In the VLSI processor, the data transfer between two processing elements can be done very quickly, so that the communication cycle time is greatly reduced.

This paper describes a single-bit parallel processor specified to Boltzmann Machine. The processor has SIMD (Shingle Instruction Multiple Data stream) type parallel architecture and every processing element (PE) has a single-bit ALU and a local memory storing connected weights between neurons. Features of the processor are large scale parallel processing a number of the simple single-bit PEs and effective expansion realized by multiple chips connected simple bus lines. Moreover, it is enhanced that the processing speed can be independent of the number of the neurons. We designed the PE using 1.2 µm CMOS process standard cells and confirmed the high performance using CAD simulations.

With the arrival of B-ISDN, point-to-point routing alone is no longer adequate. A new class of computer and video related services, such as mass mailing, TV broadcasting, teleconferencing, and video 900 service, requires the network to handle multiple destination routing (MDR). Multiple destination routing enables widespread usage of multipoint services at a lower cost than networks using point-to-point routing. With this in mind, network providers are researching more into MDR algorithms. However, the MDR problem itself is very complex. Furthermore, its optimal solution, the Steiner tree problem, is NP-complete and thus not suitable for real-time applications. Recently, various algorithms which approximate the Steiner tree problem have been proposed and, in this invited paper, we will summarize the simulation results of these algorithms. But first, we will define the MDR problem, the issues involved, and the benchmark used to compare MDR algorithms. Then, we will categorize the existing MDR algorithms into a five-level classification tree. Lastly, we will present various published results of static algorithms and our own simulation results of quasi-static algorithms.