Bias in O-Information Estimation

Johanna Gehlen, Jie Li, Cillian Hourican, Stavroula Tassi, Pashupati P. Mishra, Terho Lehtimäki, Mika Kähönen, Olli Raitakari, Jos A. Bosch, Rick Quax

Research output: Contribution to journalArticleScientificpeer-review

3 Downloads (Pure)

Abstract

Higher-order relationships are a central concept in the science of complex systems. A popular method of attempting to estimate the higher-order relationships of synergy and redundancy from data is through the O-information. It is an information–theoretic measure composed of Shannon entropy terms that quantifies the balance between redundancy and synergy in a system. However, bias is not yet taken into account in the estimation of the O-information of discrete variables. In this paper, we explain where this bias comes from and explore it for fully synergistic, fully redundant, and fully independent simulated systems of (Formula presented.) variables. Specifically, we explore how the sample size and number of bins affect the bias in the O-information estimation. The main finding is that the O-information of independent systems is severely biased towards synergy if the sample size is smaller than the number of jointly possible observations. This could mean that triplets identified as highly synergistic may in fact be close to independent. A bias approximation based on the Miller–Maddow method is derived for the O-information. We find that for systems of (Formula presented.) variables the bias approximation can partially correct for the bias. However, simulations of fully independent systems are still required as null models to provide a benchmark of the bias of the O-information.

Original languageEnglish
Article number837
Number of pages24
JournalEntropy
Volume26
Issue number10
DOIs
Publication statusPublished - Oct 2024
Publication typeA1 Journal article-refereed

Keywords

  • bias
  • complex systems
  • higher-order relationships
  • information synergy
  • O-information

Publication forum classification

  • Publication forum level 0

ASJC Scopus subject areas

  • Information Systems
  • Mathematical Physics
  • Physics and Astronomy (miscellaneous)
  • General Physics and Astronomy
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Bias in O-Information Estimation'. Together they form a unique fingerprint.

Cite this