In the present paper we present a mathematical theory for the transient analysis of probabilistic models relevant to communication networks. First we review the z-transform method, the matrix method, and the Laplace transform, as applied to a class of birth-and-death process model that is relevant to characterize network traffic sources. We then show how to develop transient solutions in terms of the eigenvalues and spectral expansions. In the latter half the paper we develop a general theory to solve dynamic behavior of statistical multiplexer for multiple types of traffic sources, which will arise in the B-ISDN environment. We transform the partial differential equation that governs the system into a concise form by using the theory of linear operator. We present a closed form expression (in the Laplace transform domain) for transient solutions of the joint probability distribution of the number of on sources and buffer content for an arbitrary initial condition. Both finite and infinite buffer capacity cases are solved exactly. The essence of this general result is based on the unique determination of unknown boundary conditions of the probability distributions. Other possible applications of this general theory are discussed, and several problems for future investigations are identified.