Entropy and relative entropy from information-theoretic principles

Gour, G; Tomamichel, M

Entropy and relative entropy from information-theoretic principles

Gour, G Tomamichel, M

Permalink

Publication Type:: Journal Article
Citation:: IEEE Trans. Inf. Theory, 2020, 67, pp. 6313-6327
Issue Date:: 2020-06-20

Closed Access

	Filename	Description	Size
	2006.11164v2.pdf		912.19 kB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Gour, G
dc.contributor.author	Tomamichel, M https://orcid.org/0000-0001-5410-3329
dc.date.accessioned	2023-03-14T01:29:41Z
dc.date.available	2023-03-14T01:29:41Z
dc.date.issued	2020-06-20
dc.identifier.citation	IEEE Trans. Inf. Theory, 2020, 67, pp. 6313-6327
dc.identifier.uri	http://hdl.handle.net/10453/167283
dc.description.abstract	We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find that these axioms induce sufficient structure to establish continuity in the interior of the probability simplex and meaningful upper and lower bounds, e.g., we find that every relative entropy must lie between the R\'enyi divergences of order $0$ and $\infty$. We further show simple conditions for positive definiteness of such relative entropies and a characterisation in term of a variant of relative trumping. Our main result is a one-to-one correspondence between entropies and relative entropies.
dc.language	en
dc.relation.ispartof	IEEE Trans. Inf. Theory
dc.rights	info:eu-repo/semantics/closedAccess
dc.title	Entropy and relative entropy from information-theoretic principles
dc.type	Journal Article
utslib.citation.volume	67
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
utslib.copyright.status	closed_access	*
dc.date.updated	2023-03-14T01:29:38Z
pubs.volume	67

Abstract:

We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find that these axioms induce sufficient structure to establish continuity in the interior of the probability simplex and meaningful upper and lower bounds, e.g., we find that every relative entropy must lie between the R\'enyi divergences of order $0$ and $\infty$. We further show simple conditions for positive definiteness of such relative entropies and a characterisation in term of a variant of relative trumping. Our main result is a one-to-one correspondence between entropies and relative entropies.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/167283