We have defined the number of degrees of freedom of order n (the n-th order NDF) kn(T) of a piece of an ergodic stationary random process X(t) of length T by using the variance of an estimate Zn(T) of the n-th moment of X(t). Zn(T) is here calculated by averaging Xn(t) over a time interval T. The NDF denotes a quantitative measure of the effective number of independent or uncorrelated random variables Xn(ti) included in the time interval T. Correlation times and equivalent bandwidths, which are important in random processes and some fields in physics, are deduced from the NDF's. In this paper, we study the NDF's from the viewpoint of information theory. First, a simple information measure Jn(T) based on the uncertainty (Shannon's entropy) of Zn(T) is defined. It is natural that we introduce such Jn(T) because kn(T) has been defined based on the variance of Zn(T). We show that the relation J1(T)(log k1(T))/2 and the asymptotic relation (log kn(T))/(2Jn(T)) 1(n2, 3, ) as T . Second, a measure In(T) of the information gained about the n-th moment of X(t) by the observation of the Zn(T) is defined in the style of Shannon. The relation I1(T)(log k1(T))/2 and the asymptotic relation I2(T) (log k2(T))/2 as T are deduced. Last, we define a measure Fn(T) as the information in the style of Fisher about the n-th moment contained in the estimate Zn(T) and we show that F1(T)k1(T) and F2(T) approaches asymptotically k2(T) as T . As a result, the n-th order NDF kn(T), which was originally defined by using the variance of Zn(T), is almost equivalent to one defined by using the entropy of Zn(T). Further, the 1st and 2nd order NDF's show the amount of information about the 1st and 2nd moment of X(t) which are contained in Z1(T) and Z2(T) respectively.