The achievability part of the rate-distortion theorem is proved by showing existence of good codes. For i.i.d. sources, two methods showing existence are known; random coding and non-random coding. For general sources, however, no proof in which good codes are constructed with non-random coding is found. In this paper, with a non-random method of code construction, we prove the achievability part of the rate-distortion theorem for general sources. Moreover, we also prove a stochastic variation of the rate-distortion theorem with the same method.

The asymptotic behavior of the recurrence time with fidelity criterion is discussed. Let X= be a source and Y= a database. For a Δ>0 and an integer l>0 define (Y,X,Δ) as the minimum integer N satisfying dl(,) Δ subject to a fidelity criterion dl. In this paper the following two i. i. d. cases are considered: (A) Xi P and Yi Q, where P and Q are probability distributions on a finite alphabet, and (B) Xi N(0,1) and Yi N(0,1). In case (A) it is proved that (1/l)log2(Y,X,Δ) almost surely converges to a certain constant determined by P, Q and Δ as l. The Kac's lemma plays an important role in the proof on the convergence. In case (B) it is shown that there is a quantity related to (1/l)log2 (Y,X,Δ) that converges to the rate-distortion bound in almost sure sense.

This article proposes, given an independently-and-identically distributed binary source, an arithmetic code-like variable-to-variable length source code whose compression efficiency achieves nearly the rate function in a range of small distortion. Inheriting advantages of arithmetic codes, the proposed code requires neither large memory capacity nor large computation time for management of messages and codewords. The Elias code, which can be regarded as an antecedent of arithmetic codes, is defined originally in terms of the first-in-first-out (FIFO) coding form. The proposed code corresponds to an extension from the Elias code refined in terms of the last-in-first-out (LIFO) coding form into one considered a fidelity criterion.