Equivalences among Some Information Measures for Individual Sequences and Their Applications for Fixed-Length Coding Problems
Summary :
This paper proposes three new information measures for individual sequences and clarifies their properties. Our new information measures are called as the non-overlapping max-entropy, the overlapping smooth max-entropy, and the non-overlapping smooth max-entropy, respectively. These measures are related to the fixed-length coding of individual sequences. We investigate these measures, and show the following three properties: (1) The non-overlapping max-entropy coincides with the topological entropy. (2) The overlapping smooth max-entropy and the non-overlapping smooth max-entropy coincide with the Ziv-entropy. (3) When an individual sequence is drawn from an ergodic source, the overlapping smooth max-entropy and the non-overlapping smooth max-entropy coincide with the entropy rate of the source. Further, we apply these information measures to the fixed-length coding of individual sequences, and propose some new universal coding schemes which are asymptotically optimum.
- Publication
- IEICE TRANSACTIONS on Fundamentals Vol.E107-A No.3 pp.393-403
- Publication Date
- 2024/03/01
- Publicized
- 2023/08/16
- Online ISSN
- 1745-1337
- DOI
- 10.1587/transfun.2023TAP0005
- Type of Manuscript
- Special Section PAPER (Special Section on Information Theory and Its Applications)
- Category
- Source Coding and Data Compression
Authors
Tomohiko UYEMATSU
Tokyo Institute of Technology
Tetsunao MATSUTA
Saitama University
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Tomohiko UYEMATSU, Tetsunao MATSUTA, "Equivalences among Some Information Measures for Individual Sequences and Their Applications for Fixed-Length Coding Problems" in IEICE TRANSACTIONS on Fundamentals,
vol. E107-A, no. 3, pp. 393-403, March 2024, doi: 10.1587/transfun.2023TAP0005.
Abstract: This paper proposes three new information measures for individual sequences and clarifies their properties. Our new information measures are called as the non-overlapping max-entropy, the overlapping smooth max-entropy, and the non-overlapping smooth max-entropy, respectively. These measures are related to the fixed-length coding of individual sequences. We investigate these measures, and show the following three properties: (1) The non-overlapping max-entropy coincides with the topological entropy. (2) The overlapping smooth max-entropy and the non-overlapping smooth max-entropy coincide with the Ziv-entropy. (3) When an individual sequence is drawn from an ergodic source, the overlapping smooth max-entropy and the non-overlapping smooth max-entropy coincide with the entropy rate of the source. Further, we apply these information measures to the fixed-length coding of individual sequences, and propose some new universal coding schemes which are asymptotically optimum.
URL: https://global.ieice.org/en_transactions/fundamentals/10.1587/transfun.2023TAP0005/_p
Copy
@ARTICLE{e107-a_3_393,
author={Tomohiko UYEMATSU, Tetsunao MATSUTA, },
journal={IEICE TRANSACTIONS on Fundamentals},
title={Equivalences among Some Information Measures for Individual Sequences and Their Applications for Fixed-Length Coding Problems},
year={2024},
volume={E107-A},
number={3},
pages={393-403},
abstract={This paper proposes three new information measures for individual sequences and clarifies their properties. Our new information measures are called as the non-overlapping max-entropy, the overlapping smooth max-entropy, and the non-overlapping smooth max-entropy, respectively. These measures are related to the fixed-length coding of individual sequences. We investigate these measures, and show the following three properties: (1) The non-overlapping max-entropy coincides with the topological entropy. (2) The overlapping smooth max-entropy and the non-overlapping smooth max-entropy coincide with the Ziv-entropy. (3) When an individual sequence is drawn from an ergodic source, the overlapping smooth max-entropy and the non-overlapping smooth max-entropy coincide with the entropy rate of the source. Further, we apply these information measures to the fixed-length coding of individual sequences, and propose some new universal coding schemes which are asymptotically optimum.},
keywords={},
doi={10.1587/transfun.2023TAP0005},
ISSN={1745-1337},
month={March},}
Copy
TY - JOUR
TI - Equivalences among Some Information Measures for Individual Sequences and Their Applications for Fixed-Length Coding Problems
T2 - IEICE TRANSACTIONS on Fundamentals
SP - 393
EP - 403
AU - Tomohiko UYEMATSU
AU - Tetsunao MATSUTA
PY - 2024
DO - 10.1587/transfun.2023TAP0005
JO - IEICE TRANSACTIONS on Fundamentals
SN - 1745-1337
VL - E107-A
IS - 3
JA - IEICE TRANSACTIONS on Fundamentals
Y1 - March 2024
AB - This paper proposes three new information measures for individual sequences and clarifies their properties. Our new information measures are called as the non-overlapping max-entropy, the overlapping smooth max-entropy, and the non-overlapping smooth max-entropy, respectively. These measures are related to the fixed-length coding of individual sequences. We investigate these measures, and show the following three properties: (1) The non-overlapping max-entropy coincides with the topological entropy. (2) The overlapping smooth max-entropy and the non-overlapping smooth max-entropy coincide with the Ziv-entropy. (3) When an individual sequence is drawn from an ergodic source, the overlapping smooth max-entropy and the non-overlapping smooth max-entropy coincide with the entropy rate of the source. Further, we apply these information measures to the fixed-length coding of individual sequences, and propose some new universal coding schemes which are asymptotically optimum.
ER -
English
