IDTxl基本使用教程

安装和依赖请查看:

Installation and Requirements

%reload_ext autoreload
%autoreload 2
%matplotlib inline

使用IDTxl做第一个网络推断

# %load test_first_example.py
# Import classes
from idtxl.multivariate_te import MultivariateTE
from idtxl.data import Data
from idtxl.visualise_graph import plot_network
import matplotlib.pyplot as plt
# a) Generate test data
data = Data()
data.generate_mute_data(n_samples=1000, n_replications=1)

# b) Initialise analysis object and define settings
network_analysis = MultivariateTE()
settings = {'cmi_estimator': 'JidtGaussianCMI',
            'max_lag_sources': 5,
            'min_lag_sources': 1}

# c) Run analysis
results = network_analysis.analyse_network(settings=settings, data=data)

# d) Plot inferred network to console and via matplotlib
results.print_edge_list(weights='max_te_lag', fdr=False)
plot_network(results=results, weights='max_te_lag', fdr=False)
plt.show()
Adding data with properties: 5 processes, 1000 samples, 1 replications
overwriting existing data

####### analysing target with index 0 from list [0, 1, 2, 3, 4]

Target: 0 - testing sources [1, 2, 3, 4]

---------------------------- (1) include target candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (0, 1) maximum statistic, n_perm: 200
testing candidate: (0, 2) maximum statistic, n_perm: 200
testing candidate: (0, 3) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) include source candidates
candidate set: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (1, 5) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
no sources selected, nothing to prune ...

---------------------------- (4) final statistics
no sources selected ...
final source samples: []
final target samples: [(0, 4), (0, 1), (0, 2)]



####### analysing target with index 1 from list [0, 1, 2, 3, 4]

Target: 1 - testing sources [0, 2, 3, 4]

---------------------------- (1) include target candidates
candidate set: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (1, 5) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (3, 3) maximum statistic, n_perm: 200
testing candidate: (4, 5) maximum statistic, n_perm: 200
testing candidate: (4, 1) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
selected candidates: [(3, 2), (3, 1), (3, 3), (4, 5)]
testing candidate: (4, 5) minimum statistic, n_perm: 500
 -- significant

---------------------------- (4) final statistics
selected variables: [(1, 4), (1, 1), (1, 5), (1, 2), (1, 3), (3, 2), (3, 1), (3, 3), (4, 5)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 4 selected sources
final source samples: [(3, 2), (3, 1), (3, 3), (4, 5)]
final target samples: [(1, 4), (1, 1), (1, 5), (1, 2), (1, 3)]



####### analysing target with index 2 from list [0, 1, 2, 3, 4]

Target: 2 - testing sources [0, 1, 3, 4]

---------------------------- (1) include target candidates
candidate set: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (2, 1) maximum statistic, n_perm: 200
testing candidate: (2, 3) maximum statistic, n_perm: 200
testing candidate: (2, 5) maximum statistic, n_perm: 200
testing candidate: (2, 2) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (0, 3) maximum statistic, n_perm: 200
testing candidate: (3, 4) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
selected candidates: [(0, 3)]
 -- significant

---------------------------- (4) final statistics
selected variables: [(2, 4), (2, 1), (2, 3), (2, 5), (0, 3)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 1 selected sources
final source samples: [(0, 3)]
final target samples: [(2, 4), (2, 1), (2, 3), (2, 5)]



####### analysing target with index 3 from list [0, 1, 2, 3, 4]

Target: 3 - testing sources [0, 1, 2, 4]

---------------------------- (1) include target candidates
candidate set: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 3) maximum statistic, n_perm: 200
testing candidate: (3, 5) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 5) maximum statistic, n_perm: 200
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
selected candidates: [(4, 5), (1, 1), (1, 2)]
testing candidate: (1, 2) minimum statistic, n_perm: 500
 -- significant

---------------------------- (4) final statistics
selected variables: [(3, 4), (3, 1), (3, 2), (3, 3), (4, 5), (1, 1), (1, 2)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 3 selected sources
final source samples: [(4, 5), (1, 1), (1, 2)]
final target samples: [(3, 4), (3, 1), (3, 2), (3, 3)]



####### analysing target with index 4 from list [0, 1, 2, 3, 4]

Target: 4 - testing sources [0, 1, 2, 3]

---------------------------- (1) include target candidates
candidate set: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 1) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (4, 4) maximum statistic, n_perm: 200
testing candidate: (4, 2) maximum statistic, n_perm: 200
testing candidate: (4, 3) maximum statistic, n_perm: 200
testing candidate: (4, 5) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (0, 3) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
selected candidates: [(3, 1)]
 -- significant

---------------------------- (4) final statistics
selected variables: [(4, 1), (4, 4), (4, 2), (4, 3), (4, 5), (3, 1)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 1 selected sources
final source samples: [(3, 1)]
final target samples: [(4, 1), (4, 4), (4, 2), (4, 3), (4, 5)]


	0 -> 2, max_te_lag: 3
	1 -> 3, max_te_lag: 1
	3 -> 1, max_te_lag: 2
	3 -> 4, max_te_lag: 1
	4 -> 1, max_te_lag: 5
	4 -> 3, max_te_lag: 5
../_images/IDTxl_3_20.png

理论介绍

为从数据中得到有用的结果,我们需要对算法的理论有一个初步的了解

为什么使用多变量转移熵呢?

转移熵最早由Schreiber在2000年提出,它是一种度量两个随机过程之间信息流动的有向估计方法。然而系统中往往是包含多个变量的,此时二元转移熵可能会产生一些错误的结果:

  1. Spurious or redundant interactions

  2. Synergistic interactions

于是,我们需要一种可以考虑多个变量之间相互作用的方法。

IDTxl实现了Lizier在2012年提出的一个贪婪迭代算法,有如下好处:

  1. The algorithm infers all relevant sources of a target by iteratively including variables from a source’s past that maximise a conditionsl mutual information criterion.

  2. IDTxl builds the set of parent sources for each target node in the network. This iterative conditioning is designed to both remove redundancies and capture synergistic interactions in building each parent set, thus addressing the two aforementioned shortcomings of bivariate analysis.

  3. THe inclusion of source variables requires repeated statistical testing of each investigated past variable in each iteration. IDTxl handles this multiple testing, in particular the family-wise error rate(FWER), by implementing a so-called maximum statistic test.

  4. Further statistical tests are aimed at pruning the selected parent set and providing control of the FMER slao at the network level.

  5. At the same time, IDTcl automates the optimization of parameters necessary for the estimation of mTE.

前四步就是2012文章里的算法所说,最后一个在哪里,这个很重要!!

关于更详细的介绍,可以去读2012年那篇文章,或者查看 Theoretical Introduction

两个主要应用

  1. Network inference

  2. Node dynamics

网络推断的几种方法

Multivariate Transfer Entropy / Granger causality

# %load ./IDTxl/demos/demo_multivariate_te.py
# Import classes
from idtxl.multivariate_te import MultivariateTE
from idtxl.data import Data
from idtxl.visualise_graph import plot_network
import matplotlib.pyplot as plt

# a) Generate test data
data = Data()
data.generate_mute_data(n_samples=1000, n_replications=5)

# b) Initialise analysis object and define settings
network_analysis = MultivariateTE()
settings = {'cmi_estimator': 'JidtGaussianCMI',
            'max_lag_sources': 5,
            'min_lag_sources': 1}

# c) Run analysis
results = network_analysis.analyse_network(settings=settings, data=data)

# d) Plot inferred network to console and via matplotlib
results.print_edge_list(weights='max_te_lag', fdr=False)
plot_network(results=results, weights='max_te_lag', fdr=False)
plt.show()
Adding data with properties: 5 processes, 1000 samples, 5 replications
overwriting existing data

####### analysing target with index 0 from list [0, 1, 2, 3, 4]

Target: 0 - testing sources [1, 2, 3, 4]

---------------------------- (1) include target candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (0, 1) maximum statistic, n_perm: 200
testing candidate: (0, 2) maximum statistic, n_perm: 200
testing candidate: (0, 5) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) include source candidates
candidate set: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (2, 2) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
no sources selected, nothing to prune ...

---------------------------- (4) final statistics
no sources selected ...
final source samples: []
final target samples: [(0, 4), (0, 1), (0, 2)]



####### analysing target with index 1 from list [0, 1, 2, 3, 4]

Target: 1 - testing sources [0, 2, 3, 4]

---------------------------- (1) include target candidates
candidate set: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (1, 5) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (3, 3) maximum statistic, n_perm: 200
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (0, 1) maximum statistic, n_perm: 200
testing candidate: (4, 5) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
selected candidates: [(3, 3), (3, 2), (3, 1), (0, 1)]
testing candidate: (0, 1) minimum statistic, n_perm: 500
 -- significant

---------------------------- (4) final statistics
selected variables: [(1, 4), (1, 1), (1, 5), (1, 2), (1, 3), (3, 3), (3, 2), (3, 1), (0, 1)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 4 selected sources
final source samples: [(3, 3), (3, 2), (3, 1), (0, 1)]
final target samples: [(1, 4), (1, 1), (1, 5), (1, 2), (1, 3)]



####### analysing target with index 2 from list [0, 1, 2, 3, 4]

Target: 2 - testing sources [0, 1, 3, 4]

---------------------------- (1) include target candidates
candidate set: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (2, 1) maximum statistic, n_perm: 200
testing candidate: (2, 3) maximum statistic, n_perm: 200
testing candidate: (2, 5) maximum statistic, n_perm: 200
testing candidate: (2, 2) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (0, 3) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
selected candidates: [(0, 3)]
 -- significant

---------------------------- (4) final statistics
selected variables: [(2, 4), (2, 1), (2, 3), (2, 5), (2, 2), (0, 3)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 1 selected sources
final source samples: [(0, 3)]
final target samples: [(2, 4), (2, 1), (2, 3), (2, 5), (2, 2)]



####### analysing target with index 3 from list [0, 1, 2, 3, 4]

Target: 3 - testing sources [0, 1, 2, 4]

---------------------------- (1) include target candidates
candidate set: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 1) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (3, 4) maximum statistic, n_perm: 200
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 3) maximum statistic, n_perm: 200
testing candidate: (3, 5) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (4, 1) maximum statistic, n_perm: 200
testing candidate: (0, 1) maximum statistic, n_perm: 200
testing candidate: (4, 3) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
selected candidates: [(1, 1), (1, 3), (1, 2), (4, 1), (0, 1)]
testing candidate: (0, 1) minimum statistic, n_perm: 500
 -- significant

---------------------------- (4) final statistics
selected variables: [(3, 1), (3, 4), (3, 2), (3, 3), (3, 5), (1, 1), (1, 3), (1, 2), (4, 1), (0, 1)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 5 selected sources
final source samples: [(1, 1), (1, 3), (1, 2), (4, 1), (0, 1)]
final target samples: [(3, 1), (3, 4), (3, 2), (3, 3), (3, 5)]



####### analysing target with index 4 from list [0, 1, 2, 3, 4]

Target: 4 - testing sources [0, 1, 2, 3]

---------------------------- (1) include target candidates
candidate set: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 1) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (4, 4) maximum statistic, n_perm: 200
testing candidate: (4, 2) maximum statistic, n_perm: 200
testing candidate: (4, 3) maximum statistic, n_perm: 200
testing candidate: (4, 5) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (3, 3) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
selected candidates: [(3, 1)]
 -- significant

---------------------------- (4) final statistics
selected variables: [(4, 1), (4, 4), (4, 2), (4, 3), (4, 5), (3, 1)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 1 selected sources
final source samples: [(3, 1)]
final target samples: [(4, 1), (4, 4), (4, 2), (4, 3), (4, 5)]


	0 -> 1, max_te_lag: 1
	0 -> 2, max_te_lag: 3
	0 -> 3, max_te_lag: 1
	1 -> 3, max_te_lag: 1
	3 -> 1, max_te_lag: 2
	3 -> 4, max_te_lag: 1
	4 -> 3, max_te_lag: 1
../_images/IDTxl_12_37.png
# a) Generate test data
data = Data()
data.generate_mute_data(n_samples=1000, n_replications=5)
Adding data with properties: 5 processes, 1000 samples, 5 replications
overwriting existing data
data.data.shape
(5, 1000, 5)
# b) Initialise analysis object and define settings
network_analysis = MultivariateTE()
settings = {'cmi_estimator': 'JidtGaussianCMI',
            'max_lag_sources': 16,
            'min_lag_sources': 1}

# c) Run analysis
results = network_analysis.analyse_single_target(settings=settings,
                                                 data=data,
                                                 target=0,
                                                 sources=[1, 3])
Target: 0 - testing sources [1, 3]

---------------------------- (1) include target candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (0, 6), (0, 7), (0, 8), (0, 9), (0, 10), (0, 11), (0, 12), (0, 13), (0, 14), (0, 15), (0, 16)]
testing candidate: (0, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (0, 1) maximum statistic, n_perm: 200
testing candidate: (0, 2) maximum statistic, n_perm: 200
testing candidate: (0, 5) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) include source candidates
candidate set: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (1, 7), (1, 8), (1, 9), (1, 10), (1, 11), (1, 12), (1, 13), (1, 14), (1, 15), (1, 16), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (3, 6), (3, 7), (3, 8), (3, 9), (3, 10), (3, 11), (3, 12), (3, 13), (3, 14), (3, 15), (3, 16)]
testing candidate: (3, 7) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune source candidate
no sources selected, nothing to prune ...

---------------------------- (4) final statistics
no sources selected ...
final source samples: []
final target samples: [(0, 4), (0, 1), (0, 2)]
results.print_edge_list(weights='max_te_lag', fdr=False)
plot_network(results=results, weights='max_te_lag', fdr=False)
plt.show()
No significant links found in the network.
../_images/IDTxl_16_1.png

Bivariate TE / Granger causality

# %load ./IDTxl/demos/demo_bivariate_te.py
# Import classes
from idtxl.bivariate_te import BivariateTE
from idtxl.data import Data
from idtxl.visualise_graph import plot_network
import matplotlib.pyplot as plt

# a) Generate test data
data = Data()
data.generate_mute_data(n_samples=1000, n_replications=5)

# b) Initialise analysis object and define settings
network_analysis = BivariateTE()
settings = {'cmi_estimator': 'JidtGaussianCMI',
            'max_lag_sources': 5,
            'min_lag_sources': 1}

# c) Run analysis
results = network_analysis.analyse_network(settings=settings, data=data)

# d) Plot inferred network to console and via matplotlib
results.print_edge_list(weights='max_te_lag', fdr=False)
plot_network(results=results, weights='max_te_lag', fdr=False)
plt.show()
Adding data with properties: 5 processes, 1000 samples, 5 replications
overwriting existing data

####### analysing target with index 0 from list [0, 1, 2, 3, 4]

Target: 0 - testing sources [1, 2, 3, 4]

---------------------------- (1) include target candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (0, 1) maximum statistic, n_perm: 200
testing candidate: (0, 2) maximum statistic, n_perm: 200
testing candidate: (0, 3) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) include source candidates
candidate set current source: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 4) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune candidates
no sources selected, nothing to prune ...

---------------------------- (4) final statistics
no sources selected ...
final source samples: []
final target samples: [(0, 4), (0, 1), (0, 2)]



####### analysing target with index 1 from list [0, 1, 2, 3, 4]

Target: 1 - testing sources [0, 2, 3, 4]

---------------------------- (1) include target candidates
candidate set: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (1, 5) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set current source: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 2) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 3) maximum statistic, n_perm: 200
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (3, 4) maximum statistic, n_perm: 200
testing candidate: (3, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 2) maximum statistic, n_perm: 200
testing candidate: (4, 1) maximum statistic, n_perm: 200
testing candidate: (4, 3) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune candidates
selected vars sources [(3, 2), (3, 3), (3, 4), (3, 1), (4, 3), (4, 4)]
selected candidates current source: [(3, 3), (3, 2), (3, 1), (3, 4)]
testing candidate: (3, 4) minimum statistic, n_perm: 500
 -- significant
selected vars sources [(3, 2), (3, 3), (3, 4), (3, 1), (4, 3), (4, 4)]
selected candidates current source: [(4, 2), (4, 1)]
testing candidate: (4, 1) minimum statistic, n_perm: 500
 -- significant

---------------------------- (4) final statistics
selected variables: [(1, 4), (1, 1), (1, 5), (1, 2), (1, 3), (3, 3), (3, 2), (3, 1), (3, 4), (4, 2), (4, 1)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 6 selected sources
final source samples: [(3, 3), (3, 2), (3, 1), (3, 4), (4, 2), (4, 1)]
final target samples: [(1, 4), (1, 1), (1, 5), (1, 2), (1, 3)]



####### analysing target with index 2 from list [0, 1, 2, 3, 4]

Target: 2 - testing sources [0, 1, 3, 4]

---------------------------- (1) include target candidates
candidate set: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (2, 1) maximum statistic, n_perm: 200
testing candidate: (2, 3) maximum statistic, n_perm: 200
testing candidate: (2, 5) maximum statistic, n_perm: 200
testing candidate: (2, 2) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set current source: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 3) maximum statistic, n_perm: 200
testing candidate: (0, 1) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 2) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 2) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 1) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune candidates
selected vars sources [(0, 2)]
selected candidates current source: [(0, 3)]
 -- significant

---------------------------- (4) final statistics
selected variables: [(2, 4), (2, 1), (2, 3), (2, 5), (2, 2), (0, 3)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 1 selected sources
final source samples: [(0, 3)]
final target samples: [(2, 4), (2, 1), (2, 3), (2, 5), (2, 2)]



####### analysing target with index 3 from list [0, 1, 2, 3, 4]

Target: 3 - testing sources [0, 1, 2, 4]

---------------------------- (1) include target candidates
candidate set: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (3, 5) maximum statistic, n_perm: 200
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 3) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set current source: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (1, 4) maximum statistic, n_perm: 200
testing candidate: (1, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 1) maximum statistic, n_perm: 200
testing candidate: (4, 4) maximum statistic, n_perm: 200
testing candidate: (4, 2) maximum statistic, n_perm: 200
testing candidate: (4, 3) maximum statistic, n_perm: 200
testing candidate: (4, 5) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune candidates
selected vars sources [(1, 4), (1, 2), (1, 3), (1, 1), (4, 4), (4, 1), (4, 3), (4, 2)]
selected candidates current source: [(1, 1), (1, 3), (1, 2), (1, 4)]
testing candidate: (1, 4) minimum statistic, n_perm: 500
 -- significant
selected vars sources [(1, 4), (1, 2), (1, 3), (1, 1), (4, 4), (4, 1), (4, 3), (4, 2)]
selected candidates current source: [(4, 1), (4, 4), (4, 2), (4, 3)]
testing candidate: (4, 3) minimum statistic, n_perm: 500
 -- significant

---------------------------- (4) final statistics
selected variables: [(3, 4), (3, 1), (3, 5), (3, 2), (3, 3), (1, 1), (1, 3), (1, 2), (1, 4), (4, 1), (4, 4), (4, 2), (4, 3)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 8 selected sources
final source samples: [(1, 1), (1, 3), (1, 2), (1, 4), (4, 1), (4, 4), (4, 2), (4, 3)]
final target samples: [(3, 4), (3, 1), (3, 5), (3, 2), (3, 3)]



####### analysing target with index 4 from list [0, 1, 2, 3, 4]

Target: 4 - testing sources [0, 1, 2, 3]

---------------------------- (1) include target candidates
candidate set: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 1) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (4, 4) maximum statistic, n_perm: 200
testing candidate: (4, 5) maximum statistic, n_perm: 200
testing candidate: (4, 2) maximum statistic, n_perm: 200
testing candidate: (4, 3) maximum statistic, n_perm: 200

---------------------------- (2) include source candidates
candidate set current source: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 3) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (1, 4) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200
testing candidate: (1, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 3) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (3, 5) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (3) prune candidates
selected vars sources [(1, 4), (1, 3), (1, 1), (1, 2), (3, 4)]
selected candidates current source: [(1, 1), (1, 2), (1, 4), (1, 3)]
testing candidate: (1, 3) minimum statistic, n_perm: 500
 -- significant
selected vars sources [(1, 4), (1, 3), (1, 1), (1, 2), (3, 4)]
selected candidates current source: [(3, 1)]
 -- significant

---------------------------- (4) final statistics
selected variables: [(4, 1), (4, 4), (4, 5), (4, 2), (4, 3), (1, 1), (1, 2), (1, 4), (1, 3), (3, 1)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 5 selected sources
final source samples: [(1, 1), (1, 2), (1, 4), (1, 3), (3, 1)]
final target samples: [(4, 1), (4, 4), (4, 5), (4, 2), (4, 3)]


	0 -> 2, max_te_lag: 3
	1 -> 3, max_te_lag: 1
	1 -> 4, max_te_lag: 1
	3 -> 1, max_te_lag: 3
	3 -> 4, max_te_lag: 1
	4 -> 1, max_te_lag: 2
	4 -> 3, max_te_lag: 1
../_images/IDTxl_18_54.png

Multivariate mutual information

# %load ./IDTxl/demos/demo_multivariate_mi.py
# Import classes
from idtxl.multivariate_mi import MultivariateMI
from idtxl.data import Data
from idtxl.visualise_graph import plot_network
import matplotlib.pyplot as plt

# a) Generate test data
data = Data()
data.generate_mute_data(n_samples=1000, n_replications=5)

# b) Initialise analysis object and define settings
network_analysis = MultivariateMI()
settings = {'cmi_estimator': 'JidtGaussianCMI',
            'max_lag_sources': 5,
            'min_lag_sources': 1}

# c) Run analysis
results = network_analysis.analyse_network(settings=settings, data=data)

# d) Plot inferred network to console and via matplotlib
results.print_edge_list(weights='max_te_lag', fdr=False)
plot_network(results=results, weights='max_te_lag', fdr=False)
plt.show()
Adding data with properties: 5 processes, 1000 samples, 5 replications
overwriting existing data

####### analysing target with index 0 from list [0, 1, 2, 3, 4]

Target: 0 - testing sources [1, 2, 3, 4]

---------------------------- (1) include source candidates
candidate set: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (2, 1) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (2, 5) maximum statistic, n_perm: 200
testing candidate: (2, 2) maximum statistic, n_perm: 200
testing candidate: (2, 4) maximum statistic, n_perm: 200
testing candidate: (4, 4) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) prune source candidate
selected candidates: [(2, 1), (2, 5), (2, 2), (2, 4)]
testing candidate: (2, 4) minimum statistic, n_perm: 500
 -- significant

---------------------------- (3) final statistics
selected variables: [(2, 1), (2, 5), (2, 2), (2, 4)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 4 selected sources
final source samples: [(2, 1), (2, 5), (2, 2), (2, 4)]

####### analysing target with index 1 from list [0, 1, 2, 3, 4]

Target: 1 - testing sources [0, 2, 3, 4]

---------------------------- (1) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (3, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (3, 5) maximum statistic, n_perm: 200
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 3) maximum statistic, n_perm: 200
testing candidate: (4, 4) maximum statistic, n_perm: 200
testing candidate: (4, 2) maximum statistic, n_perm: 200
testing candidate: (4, 3) maximum statistic, n_perm: 200
testing candidate: (2, 4) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) prune source candidate
selected candidates: [(3, 4), (3, 1), (3, 5), (3, 2), (3, 3), (4, 4), (4, 2), (4, 3)]
testing candidate: (3, 4) minimum statistic, n_perm: 500
 -- significant

---------------------------- (3) final statistics
selected variables: [(3, 4), (3, 1), (3, 5), (3, 2), (3, 3), (4, 4), (4, 2), (4, 3)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 8 selected sources
final source samples: [(3, 4), (3, 1), (3, 5), (3, 2), (3, 3), (4, 4), (4, 2), (4, 3)]

####### analysing target with index 2 from list [0, 1, 2, 3, 4]

Target: 2 - testing sources [0, 1, 3, 4]

---------------------------- (1) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (0, 3) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (1, 5) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) prune source candidate
selected candidates: [(0, 3)]
 -- significant

---------------------------- (3) final statistics
selected variables: [(0, 3)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 1 selected sources
final source samples: [(0, 3)]

####### analysing target with index 3 from list [0, 1, 2, 3, 4]

Target: 3 - testing sources [0, 1, 2, 4]

---------------------------- (1) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (1, 1) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (1, 4) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200
testing candidate: (4, 1) maximum statistic, n_perm: 200
testing candidate: (4, 3) maximum statistic, n_perm: 200
testing candidate: (4, 2) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) prune source candidate
selected candidates: [(1, 1), (1, 4), (1, 2), (1, 3), (4, 1), (4, 3)]
testing candidate: (4, 3) minimum statistic, n_perm: 500
 -- significant

---------------------------- (3) final statistics
selected variables: [(1, 1), (1, 4), (1, 2), (1, 3), (4, 1), (4, 3)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 6 selected sources
final source samples: [(1, 1), (1, 4), (1, 2), (1, 3), (4, 1), (4, 3)]

####### analysing target with index 4 from list [0, 1, 2, 3, 4]

Target: 4 - testing sources [0, 1, 2, 3]

---------------------------- (1) include source candidates
candidate set: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 1) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 3) maximum statistic, n_perm: 200
testing candidate: (3, 5) maximum statistic, n_perm: 200
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (0, 3) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) prune source candidate
selected candidates: [(3, 1), (3, 2), (3, 3), (3, 5), (1, 1)]
testing candidate: (1, 1) minimum statistic, n_perm: 500
 -- significant

---------------------------- (3) final statistics
selected variables: [(3, 1), (3, 2), (3, 3), (3, 5), (1, 1)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 5 selected sources
final source samples: [(3, 1), (3, 2), (3, 3), (3, 5), (1, 1)]
	0 -> 2, max_te_lag: 3
	1 -> 3, max_te_lag: 1
	1 -> 4, max_te_lag: 1
	2 -> 0, max_te_lag: 1
	3 -> 1, max_te_lag: 1
	3 -> 4, max_te_lag: 1
	4 -> 1, max_te_lag: 4
	4 -> 3, max_te_lag: 1
../_images/IDTxl_20_44.png

Bivariate mutual information

# %load ./IDTxl/demos/demo_bivariate_mi.py
# Import classes
from idtxl.bivariate_mi import BivariateMI
from idtxl.data import Data
from idtxl.visualise_graph import plot_network
import matplotlib.pyplot as plt

# a) Generate test data
data = Data()
data.generate_mute_data(n_samples=1000, n_replications=5)

# b) Initialise analysis object and define settings
network_analysis = BivariateMI()
settings = {'cmi_estimator': 'JidtGaussianCMI',
            'max_lag_sources': 5,
            'min_lag_sources': 1}

# c) Run analysis
results = network_analysis.analyse_network(settings=settings, data=data)

# d) Plot inferred network to console and via matplotlib
results.print_edge_list(weights='max_te_lag', fdr=False)
plot_network(results=results, weights='max_te_lag', fdr=False)
plt.show()
Adding data with properties: 5 processes, 1000 samples, 5 replications
overwriting existing data
####### analysing target 0 of [0, 1, 2, 3, 4]

Target: 0 - testing sources [1, 2, 3, 4]

---------------------------- (1) include source candidates
candidate set current source: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 4) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 1) maximum statistic, n_perm: 200
testing candidate: (2, 5) maximum statistic, n_perm: 200
testing candidate: (2, 2) maximum statistic, n_perm: 200
testing candidate: (2, 4) maximum statistic, n_perm: 200
testing candidate: (2, 3) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 4) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 2) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) prune candidates
selected vars sources [(2, 4), (2, 0), (2, 3), (2, 1)]
selected candidates current source: [(2, 1), (2, 5), (2, 2), (2, 4)]
testing candidate: (2, 4) minimum statistic, n_perm: 500
 -- significant

---------------------------- (3) final statistics
selected variables: [(2, 1), (2, 5), (2, 2), (2, 4)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 4 selected sources
final source samples: [(2, 1), (2, 5), (2, 2), (2, 4)]
final target samples: []


####### analysing target 1 of [0, 1, 2, 3, 4]

Target: 1 - testing sources [0, 2, 3, 4]
---------------------------- (1) include source candidates
candidate set current source: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 5) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 2) maximum statistic, n_perm: 200
testing candidate: (2, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 4) maximum statistic, n_perm: 200
testing candidate: (3, 5) maximum statistic, n_perm: 200
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 3) maximum statistic, n_perm: 200
candidate set current source: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 2) maximum statistic, n_perm: 200
testing candidate: (4, 1) maximum statistic, n_perm: 200
testing candidate: (4, 3) maximum statistic, n_perm: 200
testing candidate: (4, 4) maximum statistic, n_perm: 200
testing candidate: (4, 5) maximum statistic, n_perm: 200

---------------------------- (2) prune candidates
selected vars sources [(2, 3), (3, 1), (3, 0), (3, 4), (3, 3), (3, 2), (4, 3), (4, 4), (4, 2), (4, 1), (4, 0)]
selected candidates current source: [(2, 2)]
 -- significant
selected vars sources [(2, 3), (3, 1), (3, 0), (3, 4), (3, 3), (3, 2), (4, 3), (4, 4), (4, 2), (4, 1), (4, 0)]
selected candidates current source: [(3, 4), (3, 5), (3, 1), (3, 2), (3, 3)]
testing candidate: (3, 5) minimum statistic, n_perm: 500
 -- significant
selected vars sources [(2, 3), (3, 1), (3, 0), (3, 4), (3, 3), (3, 2), (4, 3), (4, 4), (4, 2), (4, 1), (4, 0)]
selected candidates current source: [(4, 2), (4, 1), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 5) minimum statistic, n_perm: 500
 -- significant

---------------------------- (3) final statistics
selected variables: [(2, 2), (3, 4), (3, 5), (3, 1), (3, 2), (3, 3), (4, 2), (4, 1), (4, 3), (4, 4), (4, 5)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 11 selected sources
final source samples: [(2, 2), (3, 4), (3, 5), (3, 1), (3, 2), (3, 3), (4, 2), (4, 1), (4, 3), (4, 4), (4, 5)]
final target samples: []


####### analysing target 2 of [0, 1, 2, 3, 4]

Target: 2 - testing sources [0, 1, 3, 4]

---------------------------- (1) include source candidates
candidate set current source: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 3) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (0, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 2) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 1) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 5) maximum statistic, n_perm: 200
testing candidate: (4, 2) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) prune candidates
selected vars sources [(0, 2), (4, 0)]
selected candidates current source: [(0, 3)]
 -- significant
selected vars sources [(0, 2), (4, 0)]
selected candidates current source: [(4, 5)]
 -- significant

---------------------------- (3) final statistics
selected variables: [(0, 3), (4, 5)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 2 selected sources
final source samples: [(0, 3), (4, 5)]
final target samples: []


####### analysing target 3 of [0, 1, 2, 3, 4]

Target: 3 - testing sources [0, 1, 2, 4]

---------------------------- (1) include source candidates
candidate set current source: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 5) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (1, 4) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200
testing candidate: (1, 5) maximum statistic, n_perm: 200
candidate set current source: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 2) maximum statistic, n_perm: 200
testing candidate: (2, 5) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(4, 1), (4, 2), (4, 3), (4, 4), (4, 5)]
testing candidate: (4, 3) maximum statistic, n_perm: 200
testing candidate: (4, 4) maximum statistic, n_perm: 200
testing candidate: (4, 1) maximum statistic, n_perm: 200
 -- not significant

---------------------------- (2) prune candidates
selected vars sources [(1, 4), (1, 1), (1, 3), (1, 2), (1, 0), (2, 3), (4, 2), (4, 1)]
selected candidates current source: [(1, 1), (1, 4), (1, 2), (1, 3), (1, 5)]
testing candidate: (1, 4) minimum statistic, n_perm: 500
 -- significant
selected vars sources [(1, 4), (1, 1), (1, 3), (1, 2), (1, 0), (2, 3), (4, 2), (4, 1)]
selected candidates current source: [(2, 2)]
 -- significant
selected vars sources [(1, 4), (1, 1), (1, 3), (1, 2), (1, 0), (2, 3), (4, 2), (4, 1)]
selected candidates current source: [(4, 3), (4, 4)]
testing candidate: (4, 4) minimum statistic, n_perm: 500
 -- significant

---------------------------- (3) final statistics
selected variables: [(1, 1), (1, 4), (1, 2), (1, 3), (1, 5), (2, 2), (4, 3), (4, 4)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 8 selected sources
final source samples: [(1, 1), (1, 4), (1, 2), (1, 3), (1, 5), (2, 2), (4, 3), (4, 4)]
final target samples: []


####### analysing target 4 of [0, 1, 2, 3, 4]

Target: 4 - testing sources [0, 1, 2, 3]

---------------------------- (1) include source candidates
candidate set current source: [(0, 1), (0, 2), (0, 3), (0, 4), (0, 5)]
testing candidate: (0, 5) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(1, 1), (1, 2), (1, 3), (1, 4), (1, 5)]
testing candidate: (1, 1) maximum statistic, n_perm: 200
testing candidate: (1, 2) maximum statistic, n_perm: 200
testing candidate: (1, 3) maximum statistic, n_perm: 200
testing candidate: (1, 5) maximum statistic, n_perm: 200
testing candidate: (1, 4) maximum statistic, n_perm: 200
candidate set current source: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]
testing candidate: (2, 3) maximum statistic, n_perm: 200
testing candidate: (2, 4) maximum statistic, n_perm: 200
 -- not significant
candidate set current source: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 1) maximum statistic, n_perm: 200
testing candidate: (3, 2) maximum statistic, n_perm: 200
testing candidate: (3, 3) maximum statistic, n_perm: 200
testing candidate: (3, 4) maximum statistic, n_perm: 200
testing candidate: (3, 5) maximum statistic, n_perm: 200

---------------------------- (2) prune candidates
selected vars sources [(1, 4), (1, 3), (1, 2), (1, 0), (1, 1), (2, 2), (3, 4), (3, 3), (3, 2), (3, 1), (3, 0)]
selected candidates current source: [(1, 1), (1, 2), (1, 3), (1, 5), (1, 4)]
testing candidate: (1, 4) minimum statistic, n_perm: 500
 -- significant
selected vars sources [(1, 4), (1, 3), (1, 2), (1, 0), (1, 1), (2, 2), (3, 4), (3, 3), (3, 2), (3, 1), (3, 0)]
selected candidates current source: [(2, 3)]
 -- significant
selected vars sources [(1, 4), (1, 3), (1, 2), (1, 0), (1, 1), (2, 2), (3, 4), (3, 3), (3, 2), (3, 1), (3, 0)]
selected candidates current source: [(3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
testing candidate: (3, 5) minimum statistic, n_perm: 500
 -- significant

---------------------------- (3) final statistics
selected variables: [(1, 1), (1, 2), (1, 3), (1, 5), (1, 4), (2, 3), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
omnibus test, n_perm: 500
 -- significant

sequential maximum statistic, n_perm: 500, testing 11 selected sources
final source samples: [(1, 1), (1, 2), (1, 3), (1, 5), (1, 4), (2, 3), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5)]
final target samples: []


	0 -> 2, max_te_lag: 3
	1 -> 3, max_te_lag: 1
	1 -> 4, max_te_lag: 1
	2 -> 0, max_te_lag: 1
	2 -> 1, max_te_lag: 2
	2 -> 3, max_te_lag: 2
	2 -> 4, max_te_lag: 3
	3 -> 1, max_te_lag: 1
	3 -> 4, max_te_lag: 1
	4 -> 1, max_te_lag: 3
	4 -> 2, max_te_lag: 5
	4 -> 3, max_te_lag: 3
../_images/IDTxl_22_39.png

网络推断实现细节

数据处理

IDTxl uses its own class to handle data

from idtxl.data import Data
import numpy as np

The Data class holds up to 3D data, where

  1. one dimension represents processes,

  2. one dimension represents samples,

  3. one dimension represents replications.

For example, in a neuroscience setting, processes would correspond to EEG channels, samples to time steps, and replications to trials (repetitions of the same experiment).

Initialise Data Object

To import your own data set, create a Data object passing a 1D, 2D, or 3D numpy array. To specify which axis of the numpy array represents which dimension of the data, pass a one to three-letter string as dim_order, where ‘p’ stands for processes, ‘s’ for samples, and ‘r’ for replications.

** Example: 3D numpy array **

# Initialise a data object holding data with 50000 samples,
# 10 processes, and 5 replications.
mydata = np.arange(50000).reshape((1000, 10, 5))
data = Data(mydata, dim_order='spr')
Adding data with properties: 10 processes, 1000 samples, 5 replications
overwriting existing data

** Example: 2D numpy array **

# Initialise a data object holding data with 5 processes and
# 5000 samples
mydata = np.arange(5000).reshape((5, 1000))
dat = Data(mydata, dim_order='ps')
Adding data with properties: 5 processes, 1000 samples, 1 replications
overwriting existing data
mydata.shape
(5, 1000)
dat.data.shape
(5, 1000, 1)

** Example: 1D numpy array **

# Initialise a data object holding data with 1 process and
# 1000 samples
mydata = np.arange(1000)
dat = Data(mydata, dim_order='s')
Adding data with properties: 1 processes, 1000 samples, 1 replications
overwriting existing data

Example: pandas data frame

import pandas as pd
df = pd.DataFrame(
    {'col1': np.random.rand(100), 'col2': np.random.rand(100)})
data = Data(df, dim_order='sp')
# Adding data with properties: 2 processes, 100 samples, 1 replications
# overwriting existing data
Adding data with properties: 2 processes, 100 samples, 1 replications
overwriting existing data
data.data.shape
(2, 100, 1)

IDTxl also provides the functionality to generate synthetic test data

设置模型参数

选择合适的CMI 参数

CMI估计方法的选取很重要,对不同的数据类型应该选择不同的参数,主要是在setting里面设置cmi_estimator这个参数。

主要数据类型:

  1. discrete data

  2. jointly Gaussian continuous data

  3. non-linear continuous data

对于非线性的连续数据有两种估计选择:

  1. JidtKraskovCMI

  2. OpenCLKraskovCMI

import idtxl.estimators_jidt
help(idtxl.estimators_jidt)
Help on module idtxl.estimators_jidt in idtxl:

NAME
    idtxl.estimators_jidt - Provide JIDT estimators.

CLASSES
    idtxl.estimator.Estimator(builtins.object)
        JidtEstimator
            JidtDiscrete
                JidtDiscreteAIS
                JidtDiscreteCMI
                JidtDiscreteMI
                JidtDiscreteTE
            JidtGaussian
                JidtGaussianAIS
                JidtGaussianCMI
                JidtGaussianMI
                JidtGaussianTE
            JidtKraskov
                JidtKraskovAIS
                JidtKraskovCMI
                JidtKraskovMI
                JidtKraskovTE
    
    class JidtDiscrete(JidtEstimator)
     |  JidtDiscrete(settings)
     |  
     |  Abstract class for implementation of discrete JIDT-estimators.
     |  
     |  Abstract class for implementation of plug-in JIDT-estimators for discrete
     |  data. Child classes implement estimators for mutual information (MI),
     |  conditional mutual information (CMI), actice information storage (AIS), and
     |  transfer entropy (TE). See parent class for references.
     |  
     |  Set common estimation parameters for discrete JIDT-estimators. For usage of
     |  these estimators see documentation for the child classes.
     |  
     |  Args:
     |      settings : dict [optional]
     |          set estimator parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - discretise_method : str [optional] - if and how to discretise
     |            incoming continuous data, can be 'max_ent' for maximum entropy
     |            binning, 'equal' for equal size bins, and 'none' if no binning is
     |            required (default='none')
     |  
     |  Note:
     |      Discrete JIDT estimators require the data's alphabet size for
     |      instantiation. Hence, opposed to the Kraskov and Gaussian estimators,
     |      the JAVA class is added to the object instance, while for Kraskov/
     |      Gaussian estimators an instance of that class is added (because for the
     |      latter, objects can be instantiated independent of data properties).
     |  
     |  Method resolution order:
     |      JidtDiscrete
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings)
     |      Set default estimator settings.
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Return estimate of the analytical surrogate distribution.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true.
     |      
     |      Args:
     |          n_perms : int [optional]
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  get_analytic_distribution(self, **data)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset({'estimate', 'get_analytic_distributio...
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate(self, **vars)
     |      Estimate measure for a single data set.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each variable in vars should be a numpy array of realisations, where
     |      the first axis is assumed to represent realisations over samples and
     |      replications, while the second axis represents the variable dimension
     |      ([(n_samples * n_replications) x variable dimension]).
     |      
     |      For parallel estimators, the first axis of each variable is assumed to
     |      represent realisations for multiple chunks of realisations that are
     |      then handled in parallel. Each array has size
     |      [(n_samples * n_replications * n_chunks) x variable dimension]. The
     |      number of chunks has to be passed as an additional parameter (see for
     |      example the OpenCLKraskov() estimator).
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          vars: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          float
     |              estimated value
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtDiscreteAIS(JidtDiscrete)
     |  JidtDiscreteAIS(settings)
     |  
     |  Calculate AIS with JIDT's discrete-variable implementation.
     |  
     |  Calculate the active information storage (AIS) for one process. Call JIDT
     |  via jpype and use the discrete estimator. See parent class for references.
     |  
     |  Args:
     |      settings : dict
     |          set estimator parameters:
     |  
     |          - history : int - number of samples in the target's past used as
     |            embedding (>= 0)
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - discretise_method : str [optional] - if and how to discretise
     |            incoming continuous data, can be 'max_ent' for maximum entropy
     |            binning, 'equal' for equal size bins, and 'none' if no binning is
     |            required (default='none')
     |          - n_discrete_bins : int [optional] - number of discrete bins/
     |            levels or the base of each dimension of the discrete variables
     |            (default=2). If set, this parameter overwrites/sets alph. (>= 2)
     |          - alph : int [optional] - number of discrete bins/levels for var1
     |            (default=2 , or the value set for n_discrete_bins). (>= 2)
     |  
     |  Method resolution order:
     |      JidtDiscreteAIS
     |      JidtDiscrete
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings)
     |      Set default estimator settings.
     |  
     |  estimate(self, process, return_calc=False)
     |      Estimate active information storage.
     |      
     |      Args:
     |          process : numpy array
     |              realisations as either a 2D numpy array where array dimensions
     |              represent [realisations x variable dimension] or a 1D array
     |              representing [realisations], array type can be float (requires
     |              discretisation) or int
     |          return_calc : boolean
     |              return the calculator used here as well as the numeric
     |              calculated value(s)
     |      
     |      Returns:
     |          float | numpy array
     |              average AIS over all samples or local AIS for individual
     |              samples if 'local_values'=True
     |          Java object
     |              JIDT calculator that was used here. Only returned if
     |              return_calc was set.
     |      
     |      Raises:
     |          ex.JidtOutOfMemoryError
     |              Raised when JIDT object cannot be instantiated due to mem error
     |  
     |  get_analytic_distribution(self, process)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          process : numpy array
     |              realisations as either a 2D numpy array where array dimensions
     |              represent [realisations x variable dimension] or a 1D array
     |              representing [realisations], array type can be float (requires
     |              discretisation) or int
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtDiscrete:
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Return estimate of the analytical surrogate distribution.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true.
     |      
     |      Args:
     |          n_perms : int [optional]
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtDiscreteCMI(JidtDiscrete)
     |  JidtDiscreteCMI(settings=None)
     |  
     |  Calculate CMI with JIDT's implementation for discrete variables.
     |  
     |  Calculate the conditional mutual information between two variables given
     |  the third. Call JIDT via jpype and use the discrete estimator. See parent
     |  class for references.
     |  
     |  Args:
     |      settings : dict [optional]
     |          sets estimation parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - discretise_method : str [optional] - if and how to discretise
     |            incoming continuous data, can be 'max_ent' for maximum entropy
     |            binning, 'equal' for equal size bins, and 'none' if no binning is
     |            required (default='none')
     |          - n_discrete_bins : int [optional] - number of discrete bins/
     |            levels or the base of each dimension of the discrete variables
     |            (default=2). If set, this parameter overwrites/sets alph1, alph2
     |            and alphc
     |          - alph1 : int [optional] - number of discrete bins/levels for var1
     |            (default=2, or the value set for n_discrete_bins)
     |          - alph2 : int [optional] - number of discrete bins/levels for var2
     |            (default=2, or the value set for n_discrete_bins)
     |          - alphc : int [optional] - number of discrete bins/levels for
     |            conditional (default=2, or the value set for n_discrete_bins)
     |  
     |  Method resolution order:
     |      JidtDiscreteCMI
     |      JidtDiscrete
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Set default estimator settings.
     |  
     |  estimate(self, var1, var2, conditional=None, return_calc=False)
     |      Estimate conditional mutual information.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations], array type can be
     |              float (requires discretisation) or int
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |          conditional : numpy array [optional]
     |              realisations of the conditioning variable (similar to var), if
     |              no conditional is provided, return MI between var1 and var2
     |          return_calc : boolean
     |              return the calculator used here as well as the numeric
     |              calculated value(s)
     |      
     |      Returns:
     |          float | numpy array
     |              average CMI over all samples or local CMI for individual
     |              samples if 'local_values'=True
     |          Java object
     |              JIDT calculator that was used here. Only returned if
     |              return_calc was set.
     |      
     |      Raises:
     |          ex.JidtOutOfMemoryError
     |              Raised when JIDT object cannot be instantiated due to mem error
     |  
     |  get_analytic_distribution(self, var1, var2, conditional=None)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations], array type can be
     |              float (requires discretisation) or int
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |          conditional : numpy array [optional]
     |              realisations of the conditioning variable (similar to var), if
     |              no conditional is provided, return MI between var1 and var2
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtDiscrete:
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Return estimate of the analytical surrogate distribution.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true.
     |      
     |      Args:
     |          n_perms : int [optional]
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtDiscreteMI(JidtDiscrete)
     |  JidtDiscreteMI(settings=None)
     |  
     |  Calculate MI with JIDT's discrete-variable implementation.
     |  
     |  Calculate the mutual information (MI) between two variables. Call JIDT via
     |  jpype and use the discrete estimator. See parent class for references.
     |  
     |  Args:
     |      settings : dict [optional]
     |          sets estimation parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - discretise_method : str [optional] - if and how to discretise
     |            incoming continuous data, can be 'max_ent' for maximum entropy
     |            binning, 'equal' for equal size bins, and 'none' if no binning is
     |            required (default='none')
     |          - n_discrete_bins : int [optional] - number of discrete bins/
     |            levels or the base of each dimension of the discrete variables
     |            (default=2). If set, this parameter overwrites/sets alph1 and
     |            alph2
     |          - alph1 : int [optional] - number of discrete bins/levels for var1
     |            (default=2, or the value set for n_discrete_bins)
     |          - alph2 : int [optional] - number of discrete bins/levels for var2
     |            (default=2, or the value set for n_discrete_bins)
     |          - lag_mi : int [optional] - time difference in samples to calculate
     |            the lagged MI between processes (default=0)
     |  
     |  Method resolution order:
     |      JidtDiscreteMI
     |      JidtDiscrete
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Set default estimator settings.
     |  
     |  estimate(self, var1, var2, return_calc=False)
     |      Estimate mutual information.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations], array type can be
     |              float (requires discretisation) or int
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |          return_calc : boolean
     |              return the calculator used here as well as the numeric
     |              calculated value(s)
     |      
     |      Returns:
     |          float | numpy array
     |              average MI over all samples or local MI for individual
     |              samples if 'local_values'=True
     |          Java object
     |              JIDT calculator that was used here. Only returned if
     |              return_calc was set.
     |      
     |      Raises:
     |          ex.JidtOutOfMemoryError
     |              Raised when JIDT object cannot be instantiated due to mem error
     |  
     |  get_analytic_distribution(self, var1, var2)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations], array type can be
     |              float (requires discretisation) or int
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtDiscrete:
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Return estimate of the analytical surrogate distribution.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true.
     |      
     |      Args:
     |          n_perms : int [optional]
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtDiscreteTE(JidtDiscrete)
     |  JidtDiscreteTE(settings)
     |  
     |  Calculate TE with JIDT's implementation for discrete variables.
     |  
     |  Calculate the transfer entropy between two time series processes.
     |  Call JIDT via jpype and use the discrete estimator. Transfer entropy is
     |  defined as the conditional mutual information between the source's past
     |  state and the target's current value, conditional on the target's past.
     |  See parent class for references.
     |  
     |  Args:
     |      settings : dict
     |          sets estimation parameters:
     |  
     |          - history_target : int - number of samples in the target's past
     |            used as embedding. (>= 0)
     |          - history_source  : int [optional] - number of samples in the
     |            source's past used as embedding (default=same as the target
     |            history). (>= 1)
     |          - tau_source : int [optional] - source's embedding delay
     |            (default=1). (>= 1)
     |          - tau_target : int [optional] - target's embedding delay
     |            (default=1). (>= 1)
     |          - source_target_delay : int [optional] - information transfer delay
     |            between source and target (default=1) (>= 0)
     |          - discretise_method : str [optional] - if and how to discretise
     |            incoming continuous data, can be 'max_ent' for maximum entropy
     |            binning, 'equal' for equal size bins, and 'none' if no binning is
     |            required (default='none')
     |          - n_discrete_bins : int [optional] - number of discrete bins/
     |            levels or the base of each dimension of the discrete variables
     |            (default=2). If set, this parameter overwrites/sets alph1 and
     |            alph2. (>= 2)
     |          - alph1 : int [optional] - number of discrete bins/levels for
     |            source (default=2, or the value set for n_discrete_bins). (>= 2)
     |          - alph2 : int [optional] - number of discrete bins/levels for
     |            target (default=2, or the value set for n_discrete_bins). (>= 2)
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |  
     |  Method resolution order:
     |      JidtDiscreteTE
     |      JidtDiscrete
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings)
     |      Set default estimator settings.
     |  
     |  estimate(self, source, target, return_calc=False)
     |      Estimate transfer entropy from a source to a target variable.
     |      
     |      Args:
     |          source : numpy array
     |              realisations of source variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations], array type can be
     |              float (requires discretisation) or int
     |          target : numpy array
     |              realisations of target variable (similar to var1)
     |          return_calc : boolean
     |              return the calculator used here as well as the numeric
     |              calculated value(s)
     |      
     |      Returns:
     |          float | numpy array
     |              average TE over all samples or local TE for individual
     |              samples if 'local_values'=True
     |          Java object
     |              JIDT calculator that was used here. Only returned if
     |              return_calc was set.
     |      
     |      Raises:
     |          ex.JidtOutOfMemoryError
     |              Raised when JIDT object cannot be instantiated due to mem error
     |  
     |  get_analytic_distribution(self, source, target)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          source : numpy array
     |              realisations of source variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations], array type can be
     |              float (requires discretisation) or int
     |          target : numpy array
     |              realisations of target variable (similar to var1)
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtDiscrete:
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Return estimate of the analytical surrogate distribution.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true.
     |      
     |      Args:
     |          n_perms : int [optional]
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtEstimator(idtxl.estimator.Estimator)
     |  JidtEstimator(settings=None)
     |  
     |  Abstract class for implementation of JIDT estimators.
     |  
     |  Abstract class for implementation of JIDT estimators, child classes
     |  implement estimators for mutual information (MI), conditional mutual
     |  information (CMI), active information storage (AIS), transfer entropy (TE)
     |  using the Kraskov-Grassberger-Stoegbauer estimator for continuous data,
     |  plug-in estimators for discrete data, and Gaussian estimators for
     |  continuous Gaussian data.
     |  
     |  References:
     |  
     |  - Lizier, Joseph T. (2014). JIDT: an information-theoretic toolkit for
     |    studying the dynamics of complex systems. Front Robot AI, 1(11).
     |  - Kraskov, A., Stoegbauer, H., & Grassberger, P. (2004). Estimating mutual
     |    information. Phys Rev E, 69(6), 066138.
     |  - Lizier, Joseph T., Mikhail Prokopenko, and Albert Y. Zomaya. (2012).
     |    Local measures of information storage in complex distributed computation.
     |    Inform Sci, 208, 39-54.
     |  - Schreiber, T. (2000). Measuring information transfer. Phys Rev Lett,
     |    85(2), 461.
     |  
     |  Set common estimation parameters for JIDT estimators. For usage of these
     |  estimators see documentation for the child classes.
     |  
     |  Args:
     |      settings : dict [optional]
     |          set estimator parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |  
     |  Method resolution order:
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Set default estimator settings.
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset({'estimate', 'is_analytic_null_estimat...
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate(self, **vars)
     |      Estimate measure for a single data set.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each variable in vars should be a numpy array of realisations, where
     |      the first axis is assumed to represent realisations over samples and
     |      replications, while the second axis represents the variable dimension
     |      ([(n_samples * n_replications) x variable dimension]).
     |      
     |      For parallel estimators, the first axis of each variable is assumed to
     |      represent realisations for multiple chunks of realisations that are
     |      then handled in parallel. Each array has size
     |      [(n_samples * n_replications * n_chunks) x variable dimension]. The
     |      number of chunks has to be passed as an additional parameter (see for
     |      example the OpenCLKraskov() estimator).
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          vars: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          float
     |              estimated value
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtGaussian(JidtEstimator)
     |  JidtGaussian(CalcClass, settings)
     |  
     |  Abstract class for implementation of JIDT Gaussian-estimators.
     |  
     |  Abstract class for implementation of JIDT Gaussian-estimators, child
     |  classes implement estimators for mutual information (MI), conditional
     |  mutual information (CMI), actice information storage (AIS), transfer
     |  entropy (TE) using JIDT's Gaussian estimator for continuous data. See
     |  parent class for references.
     |  
     |  Set common estimation parameters for JIDT Kraskov-estimators. For usage of
     |  these estimators see documentation for the child classes.
     |  
     |  Args:
     |      CalcClass : JAVA class
     |          JAVA class returned by jpype.JPackage
     |      settings : dict [optional]
     |          set estimator parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |  
     |  Method resolution order:
     |      JidtGaussian
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, CalcClass, settings)
     |      Set default estimator settings.
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Estimate the surrogate distribution analytically.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true
     |      
     |      Args:
     |          n_perms : int
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per estimate_parallel for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  get_analytic_distribution(self, **data)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset({'estimate'})
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate(self, **vars)
     |      Estimate measure for a single data set.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each variable in vars should be a numpy array of realisations, where
     |      the first axis is assumed to represent realisations over samples and
     |      replications, while the second axis represents the variable dimension
     |      ([(n_samples * n_replications) x variable dimension]).
     |      
     |      For parallel estimators, the first axis of each variable is assumed to
     |      represent realisations for multiple chunks of realisations that are
     |      then handled in parallel. Each array has size
     |      [(n_samples * n_replications * n_chunks) x variable dimension]. The
     |      number of chunks has to be passed as an additional parameter (see for
     |      example the OpenCLKraskov() estimator).
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          vars: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          float
     |              estimated value
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtGaussianAIS(JidtGaussian)
     |  JidtGaussianAIS(settings)
     |  
     |  Calculate active information storage with JIDT's Gaussian implementation.
     |  
     |  Calculate active information storage (AIS) for some process using JIDT's
     |  implementation of the Gaussian estimator. AIS is defined as the
     |  mutual information between the processes' past state and current value.
     |  
     |  The past state needs to be defined in the settings dictionary, where a past
     |  state is defined as a uniform embedding with parameters history and tau.
     |  The history describes the number of samples taken from a processes' past,
     |  tau describes the embedding delay, i.e., the spacing between every two
     |  samples from the processes' past.
     |  
     |  See parent class for references.
     |  
     |  Args:
     |      settings : dict
     |          sets estimation parameters:
     |  
     |          - history : int - number of samples in the processes' past used as
     |            embedding
     |          - tau : int [optional] - the processes' embedding delay (default=1)
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |  
     |  Note:
     |      Some technical details: JIDT normalises over realisations, IDTxl
     |      normalises over raw data once, outside the AIS estimator to save
     |      computation time. The Theiler window ignores trial boundaries. The
     |      AIS estimator does add noise to the data as a default. To make analysis
     |      runs replicable set noise_level to 0.
     |  
     |  Method resolution order:
     |      JidtGaussianAIS
     |      JidtGaussian
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings)
     |      Set default estimator settings.
     |  
     |  estimate(self, process)
     |      Estimate active information storage.
     |      
     |      Args:
     |          process : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations]
     |      
     |      Returns:
     |          float | numpy array
     |              average AIS over all samples or local AIS for individual
     |              samples if 'local_values'=True
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtGaussian:
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Estimate the surrogate distribution analytically.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true
     |      
     |      Args:
     |          n_perms : int
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per estimate_parallel for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  get_analytic_distribution(self, **data)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtGaussianCMI(JidtGaussian)
     |  JidtGaussianCMI(settings=None)
     |  
     |  Calculate conditional mutual infor with JIDT's Gaussian implementation.
     |  
     |  Computes the differential conditional mutual information of two
     |  multivariate sets of observations, conditioned on another, assuming that
     |  the probability distribution function for these observations is a
     |  multivariate Gaussian distribution.
     |  Call JIDT via jpype and use
     |  ConditionalMutualInfoCalculatorMultiVariateGaussian estimator.
     |  If no conditional is given (is None), the function returns the mutual
     |  information between var1 and var2.
     |  
     |  See parent class for references.
     |  
     |  Args:
     |      settings : dict [optional]
     |          sets estimation parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |  
     |  Note:
     |      Some technical details: JIDT normalises over realisations, IDTxl
     |      normalises over raw data once, outside the CMI estimator to save
     |      computation time. The Theiler window ignores trial boundaries. The
     |      CMI estimator does add noise to the data as a default. To make analysis
     |      runs replicable set noise_level to 0.
     |  
     |  Method resolution order:
     |      JidtGaussianCMI
     |      JidtGaussian
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Set default estimator settings.
     |  
     |  estimate(self, var1, var2, conditional=None)
     |      Estimate conditional mutual information.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations]
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |          conditional : numpy array [optional]
     |              realisations of the conditioning variable (similar to var), if
     |              no conditional is provided, return MI between var1 and var2
     |      
     |      Returns:
     |          float | numpy array
     |              average CMI over all samples or local CMI for individual
     |              samples if 'local_values'=True
     |  
     |  get_analytic_distribution(self, var1, var2, conditional=None)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations]
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |          conditional : numpy array [optional]
     |              realisations of the conditioning variable (similar to var), if
     |              no conditional is provided, return MI between var1 and var2
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtGaussian:
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Estimate the surrogate distribution analytically.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true
     |      
     |      Args:
     |          n_perms : int
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per estimate_parallel for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtGaussianMI(JidtGaussian)
     |  JidtGaussianMI(settings=None)
     |  
     |  Calculate mutual information with JIDT's Gaussian implementation.
     |  
     |  Calculate the mutual information between two variables. Call JIDT via jpype
     |  and use the Gaussian estimator. See parent class for references.
     |  
     |  Args:
     |      settings : dict [optional]
     |          sets estimation parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - lag_mi : int [optional] - time difference in samples to calculate
     |            the lagged MI between processes (default=0)
     |  
     |  Note:
     |      Some technical details: JIDT normalises over realisations, IDTxl
     |      normalises over raw data once, outside the MI estimator to save
     |      computation time. The Theiler window ignores trial boundaries. The
     |      MI estimator does add noise to the data as a default. To make analysis
     |      runs replicable set noise_level to 0.
     |  
     |  Method resolution order:
     |      JidtGaussianMI
     |      JidtGaussian
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Set default estimator settings.
     |  
     |  estimate(self, var1, var2)
     |      Estimate mutual information.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations]
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |      
     |      Returns:
     |          float | numpy array
     |              average MI over all samples or local MI for individual
     |              samples if 'local_values'=True
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtGaussian:
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Estimate the surrogate distribution analytically.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true
     |      
     |      Args:
     |          n_perms : int
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per estimate_parallel for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  get_analytic_distribution(self, **data)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtGaussianTE(JidtGaussian)
     |  JidtGaussianTE(settings)
     |  
     |  Calculate transfer entropy with JIDT's Gaussian implementation.
     |  
     |  Calculate transfer entropy between a source and a target variable using
     |  JIDT's implementation of the Gaussian estimator. Transfer entropy is
     |  defined as the conditional mutual information between the source's past
     |  state and the target's current value, conditional on the target's past.
     |  
     |  Past states need to be defined in the settings dictionary, where a past
     |  state is defined as a uniform embedding with parameters history and tau.
     |  The history describes the number of samples taken from a variable's past,
     |  tau descrices the embedding delay, i.e., the spacing between every two
     |  samples from the processes' past.
     |  
     |  See parent class for references.
     |  
     |  Args:
     |      settings : dict
     |          sets estimation parameters:
     |  
     |          - history_target : int - number of samples in the target's past
     |            used as embedding
     |          - history_source  : int [optional] - number of samples in the
     |            source's past used as embedding (default=same as the target
     |            history)
     |          - tau_source : int [optional] - source's embedding delay
     |            (default=1)
     |          - tau_target : int [optional] - target's embedding delay
     |            (default=1)
     |          - source_target_delay : int [optional] - information transfer delay
     |            between source and target (default=1)
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |  
     |  Note:
     |      Some technical details: JIDT normalises over realisations, IDTxl
     |      normalises over raw data once, outside the CMI estimator to save
     |      computation time. The Theiler window ignores trial boundaries. The
     |      CMI estimator does add noise to the data as a default. To make analysis
     |      runs replicable set noise_level to 0.
     |  
     |  Method resolution order:
     |      JidtGaussianTE
     |      JidtGaussian
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings)
     |      Set default estimator settings.
     |  
     |  estimate(self, source, target)
     |      Estimate transfer entropy from a source to a target variable.
     |      
     |      Args:
     |          source : numpy array
     |              realisations of source variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations]
     |          var2 : numpy array
     |              realisations of target variable (similar to var1)
     |      
     |      Returns:
     |          float | numpy array
     |              average TE over all samples or local TE for individual
     |              samples if 'local_values'=True
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtGaussian:
     |  
     |  estimate_surrogates_analytic(self, n_perm=200, **data)
     |      Estimate the surrogate distribution analytically.
     |      
     |      This method must be implemented because this class'
     |      is_analytic_null_estimator() method returns true
     |      
     |      Args:
     |          n_perms : int
     |              number of permutations (default=200)
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per estimate_parallel for this estimator.
     |      
     |      Returns:
     |          float | numpy array
     |              n_perm surrogates of the average MI/CMI/TE over all samples
     |              under the null hypothesis of no relationship between var1 and
     |              var2 (in the context of conditional)
     |  
     |  get_analytic_distribution(self, **data)
     |      Return a JIDT AnalyticNullDistribution object.
     |      
     |      Required so that our estimate_surrogates_analytic method can use the
     |      common_estimate_surrogates_analytic() method, where data is formatted
     |      as per the estimate method for this estimator.
     |      
     |      Args:
     |          data : numpy arrays
     |              realisations of random variables required for the calculation
     |              (varies between estimators, e.g. 2 variables for MI, 3 for
     |              CMI). Formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          Java object
     |              JIDT calculator that was used here
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtKraskov(JidtEstimator)
     |  JidtKraskov(CalcClass, settings=None)
     |  
     |  Abstract class for implementation of JIDT Kraskov-estimators.
     |  
     |  Abstract class for implementation of JIDT Kraskov-estimators, child classes
     |  implement estimators for mutual information (MI), conditional mutual
     |  information (CMI), actice information storage (AIS), transfer entropy (TE)
     |  using the Kraskov-Grassberger-Stoegbauer estimator for continuous data.
     |  See parent class for references.
     |  
     |  Set common estimation parameters for JIDT Kraskov-estimators. For usage of
     |  these estimators see documentation for the child classes.
     |  
     |  Args:
     |      CalcClass : JAVA class
     |          JAVA class returned by jpype.JPackage
     |      settings : dict [optional]
     |          set estimator parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - kraskov_k : int [optional] - no. nearest neighbours for KNN
     |            search (default=4)
     |          - normalise : bool [optional] - z-standardise data (default=False)
     |          - theiler_t : int [optional] - no. next temporal neighbours ignored
     |            in KNN and range searches (default=0)
     |          - noise_level : float [optional] - random noise added to the data
     |            (default=1e-8)
     |          - num_threads : int | str [optional] - number of threads used for
     |            estimation (default='USE_ALL', note that this uses *all*
     |            available threads on the current machine)
     |          - algorithm_num : int [optional] - which Kraskov algorithm (1 or 2)
     |            to use (default=1). Only applied at this method for TE and AIS
     |            (is already applied for MI/CMI). Note that default algorithm of 1
     |            here is different to the default ALG_NUM argument for the JIDT
     |            AIS KSG estimator.
     |  
     |  Method resolution order:
     |      JidtKraskov
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, CalcClass, settings=None)
     |      Set default estimator settings.
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset({'estimate'})
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate(self, **vars)
     |      Estimate measure for a single data set.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each variable in vars should be a numpy array of realisations, where
     |      the first axis is assumed to represent realisations over samples and
     |      replications, while the second axis represents the variable dimension
     |      ([(n_samples * n_replications) x variable dimension]).
     |      
     |      For parallel estimators, the first axis of each variable is assumed to
     |      represent realisations for multiple chunks of realisations that are
     |      then handled in parallel. Each array has size
     |      [(n_samples * n_replications * n_chunks) x variable dimension]. The
     |      number of chunks has to be passed as an additional parameter (see for
     |      example the OpenCLKraskov() estimator).
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          vars: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          float
     |              estimated value
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtKraskovAIS(JidtKraskov)
     |  JidtKraskovAIS(settings)
     |  
     |  Calculate active information storage with JIDT's Kraskov implementation.
     |  
     |  Calculate active information storage (AIS) for some process using JIDT's
     |  implementation of the Kraskov type 1 estimator. AIS is defined as the
     |  mutual information between the processes' past state and current value.
     |  
     |  The past state needs to be defined in the settings dictionary, where a past
     |  state is defined as a uniform embedding with parameters history and tau.
     |  The history describes the number of samples taken from a processes' past,
     |  tau describes the embedding delay, i.e., the spacing between every two
     |  samples from the processes' past.
     |  
     |  See parent class for references.
     |  
     |  Args:
     |      settings : dict
     |          sets estimation parameters:
     |  
     |          - history : int - number of samples in the processes' past used as
     |            embedding
     |          - tau : int [optional] - the processes' embedding delay (default=1)
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - kraskov_k : int [optional] - no. nearest neighbours for KNN
     |            search (default=4)
     |          - normalise : bool [optional] - z-standardise data (default=False)
     |          - theiler_t : int [optional] - no. next temporal neighbours ignored
     |            in KNN and range searches (default=0)
     |          - noise_level : float [optional] - random noise added to the data
     |            (default=1e-8)
     |          - num_threads : int | str [optional] - number of threads used for
     |            estimation (default='USE_ALL', note that this uses *all*
     |            available threads on the current machine)
     |          - algorithm_num : int [optional] - which Kraskov algorithm (1 or 2)
     |            to use (default=1)
     |  Note:
     |      Some technical details: JIDT normalises over realisations, IDTxl
     |      normalises over raw data once, outside the AIS estimator to save
     |      computation time. The Theiler window ignores trial boundaries. The
     |      AIS estimator does add noise to the data as a default. To make analysis
     |      runs replicable set noise_level to 0.
     |  
     |  Method resolution order:
     |      JidtKraskovAIS
     |      JidtKraskov
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings)
     |      Set default estimator settings.
     |  
     |  estimate(self, process)
     |      Estimate active information storage.
     |      
     |      Args:
     |          process : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations]
     |      
     |      Returns:
     |          float | numpy array
     |              average AIS over all samples or local AIS for individual
     |              samples if 'local_values'=True
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtKraskov:
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtKraskovCMI(JidtKraskov)
     |  JidtKraskovCMI(settings=None)
     |  
     |  Calculate conditional mutual inform with JIDT's Kraskov implementation.
     |  
     |  Calculate the conditional mutual information (CMI) between three variables.
     |  Call JIDT via jpype and use the Kraskov 1 estimator. If no conditional is
     |  given (is None), the function returns the mutual information between var1
     |  and var2. See parent class for references.
     |  
     |  Args:
     |      settings : dict [optional]
     |          set estimator parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - kraskov_k : int [optional] - no. nearest neighbours for KNN
     |            search (default=4)
     |          - normalise : bool [optional] - z-standardise data (default=False)
     |          - theiler_t : int [optional] - no. next temporal neighbours ignored
     |            in KNN and range searches (default=0)
     |          - noise_level : float [optional] - random noise added to the data
     |            (default=1e-8)
     |          - num_threads : int | str [optional] - number of threads used for
     |            estimation (default='USE_ALL', note that this uses *all*
     |            available threads on the current machine)
     |          - algorithm_num : int [optional] - which Kraskov algorithm (1 or 2)
     |            to use (default=1)
     |  Note:
     |      Some technical details: JIDT normalises over realisations, IDTxl
     |      normalises over raw data once, outside the CMI estimator to save
     |      computation time. The Theiler window ignores trial boundaries. The
     |      CMI estimator does add noise to the data as a default. To make analysis
     |      runs replicable set noise_level to 0.
     |  
     |  Method resolution order:
     |      JidtKraskovCMI
     |      JidtKraskov
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Set default estimator settings.
     |  
     |  estimate(self, var1, var2, conditional=None)
     |      Estimate conditional mutual information.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations]
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |          conditional : numpy array [optional]
     |              realisations of the conditioning variable (similar to var), if
     |              no conditional is provided, return MI between var1 and var2
     |      
     |      Returns:
     |          float | numpy array
     |              average CMI over all samples or local CMI for individual
     |              samples if 'local_values'=True
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtKraskov:
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtKraskovMI(JidtKraskov)
     |  JidtKraskovMI(settings=None)
     |  
     |  Calculate mutual information with JIDT's Kraskov implementation.
     |  
     |  Calculate the mutual information between two variables. Call JIDT via jpype
     |  and use the Kraskov 1 estimator. See parent class for references.
     |  
     |  Args:
     |      settings : dict [optional]
     |          sets estimation parameters:
     |  
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - kraskov_k : int [optional] - no. nearest neighbours for KNN
     |            search (default=4)
     |          - normalise : bool [optional] - z-standardise data (default=False)
     |          - theiler_t : int [optional] - no. next temporal neighbours ignored
     |            in KNN and range searches (default=0)
     |          - noise_level : float [optional] - random noise added to the data
     |            (default=1e-8)
     |          - num_threads : int | str [optional] - number of threads used for
     |            estimation (default='USE_ALL', note that this uses *all*
     |            available threads on the current machine)
     |          - algorithm_num : int [optional] - which Kraskov algorithm (1 or 2)
     |            to use (default=1)
     |          - lag_mi : int [optional] - time difference in samples to calculate
     |            the lagged MI between processes (default=0)
     |  
     |  Note:
     |      Some technical details: JIDT normalises over realisations, IDTxl
     |      normalises over raw data once, outside the MI estimator to save
     |      computation time. The Theiler window ignores trial boundaries. The
     |      MI estimator does add noise to the data as a default. To make analysis
     |      runs replicable set noise_level to 0.
     |  
     |  Method resolution order:
     |      JidtKraskovMI
     |      JidtKraskov
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Set default estimator settings.
     |  
     |  estimate(self, var1, var2)
     |      Estimate mutual information.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations]
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |      
     |      Returns:
     |          float | numpy array
     |              average MI over all samples or local MI for individual
     |              samples if 'local_values'=True
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtKraskov:
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class JidtKraskovTE(JidtKraskov)
     |  JidtKraskovTE(settings)
     |  
     |  Calculate transfer entropy with JIDT's Kraskov implementation.
     |  
     |  Calculate transfer entropy between a source and a target variable using
     |  JIDT's implementation of the Kraskov type 1 estimator. Transfer entropy is
     |  defined as the conditional mutual information between the source's past
     |  state and the target's current value, conditional on the target's past.
     |  
     |  Past states need to be defined in the settings dictionary, where a past
     |  state is defined as a uniform embedding with parameters history and tau.
     |  The history describes the number of samples taken from a variable's past,
     |  tau descrices the embedding delay, i.e., the spacing between every two
     |  samples from the processes' past.
     |  
     |  See parent class for references.
     |  
     |  Args:
     |      settings : dict
     |          sets estimation parameters:
     |  
     |          - history_target : int - number of samples in the target's past
     |            used as embedding
     |          - history_source  : int [optional] - number of samples in the
     |            source's past used as embedding (default=same as the target
     |            history)
     |          - tau_source : int [optional] - source's embedding delay
     |            (default=1)
     |          - tau_target : int [optional] - target's embedding delay
     |            (default=1)
     |          - source_target_delay : int [optional] - information transfer delay
     |            between source and target (default=1)
     |          - debug : bool [optional] - return debug information when calling
     |            JIDT (default=False)
     |          - local_values : bool [optional] - return local TE instead of
     |            average TE (default=False)
     |          - algorithm_num : int [optional] - which Kraskov algorithm (1 or 2)
     |            to use (default=1)
     |  
     |  Note:
     |      Some technical details: JIDT normalises over realisations, IDTxl
     |      normalises over raw data once, outside the CMI estimator to save
     |      computation time. The Theiler window ignores trial boundaries. The
     |      CMI estimator does add noise to the data as a default. To make analysis
     |      runs replicable set noise_level to 0.
     |  
     |  Method resolution order:
     |      JidtKraskovTE
     |      JidtKraskov
     |      JidtEstimator
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings)
     |      Set default estimator settings.
     |  
     |  estimate(self, source, target)
     |      Estimate transfer entropy from a source to a target variable.
     |      
     |      Args:
     |          source : numpy array
     |              realisations of source variable, either a 2D numpy array where
     |              array dimensions represent [realisations x variable dimension]
     |              or a 1D array representing [realisations]
     |          var2 : numpy array
     |              realisations of target variable (similar to var1)
     |      
     |      Returns:
     |          float | numpy array
     |              average TE over all samples or local TE for individual
     |              samples if 'local_values'=True
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtKraskov:
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from JidtEstimator:
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)

FUNCTIONS
    common_estimate_surrogates_analytic(estimator, n_perm=200, **data)
        Estimate the surrogate distribution analytically for JidtEstimator.
        
        Estimate the surrogate distribution analytically for a JidtEstimator
        which is_analytic_null_estimator(), by sampling estimates at random
        p-values in the analytic distribution.
        
        Args:
            estimator : a JidtEstimator object, which returns True to a call to
                its is_analytic_null_estimator() method
            n_perms : int
                number of permutations (default=200)
            data : numpy arrays
                realisations of random variables required for the calculation
                (varies between estimators, e.g. 2 variables for MI, 3 for CMI)
        
        Returns:
            float | numpy array
                n_perm surrogates of the average MI/CMI/TE over all samples
                under the null hypothesis of no relationship between var1 and
                var2 (in the context of conditional)

FILE
    /home/fsf/2020/complex_system/information_theory/githubs/IDTxl/idtxl/estimators_jidt.py

Different CMI estimators are available for discrete data, jointly Gaussian continuous data, and non-linear continuous data. For non-linear continuous data, two alternatives are available:

  1. A multithreaded Java-implementation (JidtKraskovCMI)

  2. An OpenCL-implementation (OpenCLKraskovCMI, to be run on GPUs).

settings = {'cmi_estimator': 'JidtKraskovCMI'}
import idtxl.estimators_opencl
help(idtxl.estimators_opencl)
Help on module idtxl.estimators_opencl in idtxl:

NAME
    idtxl.estimators_opencl

CLASSES
    idtxl.estimator.Estimator(builtins.object)
        OpenCLKraskov
            OpenCLKraskovCMI
            OpenCLKraskovMI
    
    class OpenCLKraskov(idtxl.estimator.Estimator)
     |  OpenCLKraskov(settings=None)
     |  
     |  Abstract class for implementation of OpenCL estimators.
     |  
     |  Abstract class for implementation of OpenCL estimators, child classes
     |  implement estimators for mutual information (MI) and conditional mutual
     |  information (CMI) using the Kraskov-Grassberger-Stoegbauer estimator for
     |  continuous data.
     |  
     |  References:
     |  
     |  - Kraskov, A., Stoegbauer, H., & Grassberger, P. (2004). Estimating mutual
     |    information. Phys Rev E, 69(6), 066138.
     |  - Lizier, Joseph T., Mikhail Prokopenko, and Albert Y. Zomaya. (2012).
     |    Local measures of information storage in complex distributed computation.
     |    Inform Sci, 208, 39-54.
     |  - Schreiber, T. (2000). Measuring information transfer. Phys Rev Lett,
     |    85(2), 461.
     |  
     |  Estimators can be used to perform multiple, independent searches in
     |  parallel. Each of these parallel searches is called a 'chunk'. To search
     |  multiple chunks, provide point sets as 2D arrays, where the first
     |  dimension represents samples or points, and the second dimension
     |  represents the points' dimensions. Concatenate chunk data in the first
     |  dimension and pass the number of chunks to the estimators. Chunks must be
     |  of equal size.
     |  
     |  Set common estimation parameters for OpenCL estimators. For usage of these
     |  estimators see documentation for the child classes.
     |  
     |  Args:
     |      settings : dict [optional]
     |          set estimator parameters:
     |  
     |          - gpuid : int [optional] - device ID used for estimation (if more
     |            than one device is available on the current platform) (default=0)
     |          - kraskov_k : int [optional] - no. nearest neighbours for KNN
     |            search (default=4)
     |          - normalise : bool [optional] - z-standardise data (default=False)
     |          - theiler_t : int [optional] - no. next temporal neighbours ignored
     |            in KNN and range searches (default=0)
     |          - noise_level : float [optional] - random noise added to the data
     |            (default=1e-8)
     |          - debug : bool [optional] - calculate intermediate results, i.e.
     |            neighbour counts from range searches and KNN distances, print
     |            debug output to console (default=False)
     |          - return_counts : bool [optional] - return intermediate results,
     |            i.e. neighbour counts from range searches and KNN distances
     |            (default=False)
     |  
     |  Method resolution order:
     |      OpenCLKraskov
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Initialize self.  See help(type(self)) for accurate signature.
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset({'estimate'})
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate(self, **vars)
     |      Estimate measure for a single data set.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each variable in vars should be a numpy array of realisations, where
     |      the first axis is assumed to represent realisations over samples and
     |      replications, while the second axis represents the variable dimension
     |      ([(n_samples * n_replications) x variable dimension]).
     |      
     |      For parallel estimators, the first axis of each variable is assumed to
     |      represent realisations for multiple chunks of realisations that are
     |      then handled in parallel. Each array has size
     |      [(n_samples * n_replications * n_chunks) x variable dimension]. The
     |      number of chunks has to be passed as an additional parameter (see for
     |      example the OpenCLKraskov() estimator).
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          vars: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          float
     |              estimated value
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class OpenCLKraskovCMI(OpenCLKraskov)
     |  OpenCLKraskovCMI(settings=None)
     |  
     |  Calculate conditional mutual inform with OpenCL Kraskov implementation.
     |  
     |  Calculate the conditional mutual information (CMI) between three variables
     |  using OpenCL GPU-code. If no conditional is given (is None), the function
     |  returns the mutual information between var1 and var2. See parent class for
     |  references.
     |  
     |  Args:
     |      settings : dict [optional]
     |          set estimator parameters:
     |  
     |          - gpuid : int [optional] - device ID used for estimation (if more
     |            than one device is available on the current platform) (default=0)
     |          - kraskov_k : int [optional] - no. nearest neighbours for KNN
     |            search (default=4)
     |          - normalise : bool [optional] - z-standardise data (default=False)
     |          - theiler_t : int [optional] - no. next temporal neighbours ignored
     |            in KNN and range searches (default=0)
     |          - noise_level : float [optional] - random noise added to the data
     |            (default=1e-8)
     |          - debug : bool [optional] - return intermediate results, i.e.
     |            neighbour counts from range searches and KNN distances
     |            (default=False)
     |          - return_counts : bool [optional] - return intermediate results,
     |            i.e. neighbour counts from range searches and KNN distances
     |            (default=False)
     |  
     |  Method resolution order:
     |      OpenCLKraskovCMI
     |      OpenCLKraskov
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Initialize self.  See help(type(self)) for accurate signature.
     |  
     |  estimate(self, var1, var2, conditional=None, n_chunks=1)
     |      Estimate conditional mutual information.
     |      
     |      If conditional is None, the mutual information between var1 and var2 is
     |      calculated.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [(realisations * n_chunks) x
     |              variable dimension] or a 1D array representing [realisations],
     |              array type should be int32
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |          conditional : numpy array
     |              realisations of conditioning variable (similar to var1)
     |          n_chunks : int
     |              number of data chunks, no. data points has to be the same for
     |              each chunk
     |      
     |      Returns:
     |          float | numpy array
     |              average CMI over all samples or local CMI for individual
     |              samples if 'local_values'=True
     |          numpy arrays
     |              distances and neighborhood counts for var1 and var2 if
     |              debug=True and return_counts=True
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from OpenCLKraskov:
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class OpenCLKraskovMI(OpenCLKraskov)
     |  OpenCLKraskovMI(settings=None)
     |  
     |  Calculate mutual information with OpenCL Kraskov implementation.
     |  
     |  Calculate the mutual information (MI) between two variables using OpenCL
     |  GPU-code. See parent class for references.
     |  
     |  Args:
     |      settings : dict [optional]
     |          set estimator parameters:
     |  
     |          - gpuid : int [optional] - device ID used for estimation (if more
     |            than one device is available on the current platform) (default=0)
     |          - kraskov_k : int [optional] - no. nearest neighbours for KNN
     |            search (default=4)
     |          - normalise : bool [optional] - z-standardise data (default=False)
     |          - theiler_t : int [optional] - no. next temporal neighbours ignored
     |            in KNN and range searches (default=0)
     |          - noise_level : float [optional] - random noise added to the data
     |            (default=1e-8)
     |          - debug : bool [optional] - return intermediate results, i.e.
     |            neighbour counts from range searches and KNN distances
     |            (default=False)
     |          - return_counts : bool [optional] - return intermediate results,
     |            i.e. neighbour counts from range searches and KNN distances
     |            (default=False)
     |          - lag_mi : int [optional] - time difference in samples to calculate
     |            the lagged MI between processes (default=0)
     |  
     |  Method resolution order:
     |      OpenCLKraskovMI
     |      OpenCLKraskov
     |      idtxl.estimator.Estimator
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, settings=None)
     |      Initialize self.  See help(type(self)) for accurate signature.
     |  
     |  estimate(self, var1, var2, n_chunks=1)
     |      Estimate mutual information.
     |      
     |      Args:
     |          var1 : numpy array
     |              realisations of first variable, either a 2D numpy array where
     |              array dimensions represent [(realisations * n_chunks) x
     |              variable dimension] or a 1D array representing [realisations],
     |              array type should be int32
     |          var2 : numpy array
     |              realisations of the second variable (similar to var1)
     |          n_chunks : int
     |              number of data chunks, no. data points has to be the same for
     |              each chunk
     |      
     |      Returns:
     |          float | numpy array
     |              average MI over all samples or local MI for individual
     |              samples if 'local_values'=True
     |          numpy arrays
     |              distances and neighborhood counts for var1 and var2 if
     |              debug=True and return_counts=True
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  __abstractmethods__ = frozenset()
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from OpenCLKraskov:
     |  
     |  is_analytic_null_estimator(self)
     |      Indicate if estimator supports analytic surrogates.
     |      
     |      Return true if the estimator implements estimate_surrogates_analytic()
     |      where data is formatted as per the estimate method for this estimator.
     |      
     |      Returns:
     |          bool
     |  
     |  is_parallel(self)
     |      Indicate if estimator supports parallel estimation over chunks.
     |      
     |      Return true if the supports parallel estimation over chunks, where a
     |      chunk is one independent data set.
     |      
     |      Returns:
     |          bool
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from idtxl.estimator.Estimator:
     |  
     |  estimate_parallel(self, n_chunks=1, re_use=None, **data)
     |      Estimate measure for multiple data sets (chunks).
     |      
     |      Test if the estimator used provides parallel capabilities; if so,
     |      estimate measure for multiple data sets ('chunks') in parallel.
     |      Otherwise, iterate over individual chunks.
     |      
     |      The number of variables in data depends on the measure to be estimated,
     |      e.g., 2 for mutual information and 3 for a conditional mutual
     |      information.
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations over samples and
     |      replications and chunks, while the second axis represents the variable
     |      dimension ([(samples * replications) * chunks x variable dimension]).
     |      
     |      Each entry in data should be a numpy array with realisations, where the
     |      first axis is assumed to represent realisations (over chunks), while
     |      the second axis is the variable dimension.
     |      
     |      Each numpy array with realisations can hold either the realisations for
     |      multiple chunks or can hold the realisation for a single chunk, which
     |      gets replicated for parallel estimation and gets re-used for iterative
     |      estimation, in order to save memory. The variables for re-use are
     |      provided in re-use as list of dictionary keys indicating entries in
     |      data for re-use.
     |      
     |      Args:
     |          self : Estimator class instance
     |              estimator
     |          n_chunks : int [optional]
     |              number of data chunks (default=1)
     |          re_use : list of keys [optional}
     |              realisatins to be re-used (default=None)
     |          data: numpy arrays
     |              realisations of random variables
     |      
     |      Returns:
     |          numpy array
     |              estimated values for each chunk
     |      
     |      Raises:
     |          ex.AlgorithmExhaustedError
     |              Raised from self.estimate() when calculation cannot be made
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from idtxl.estimator.Estimator:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)

DATA
    digamma = <ufunc 'psi'>

FILE
    /home/fsf/2020/complex_system/information_theory/githubs/IDTxl/idtxl/estimators_opencl.py

设置Source and Target lags

由于计算资源有限,我们不可能考虑所有的时间延迟,所以可以设置一个延迟计算范围,甚至可以设置延迟计算间隔,仅仅计算有效的延迟,合理利用先验知识,具体就是在setting里面设置如下参数:

max_lag_sources, min_lag_sources, max_lag_target, min_lag_target, tau_sources, tau_target

看情况待定吧,一般要记得设置 max_lag_sources,否则就会计算很多无用的东西。

settings = {'max_lag_sources': 16,
            'min_lag_sources': 6}

For transfer entropy estimation, additionally, a maximum lag for the target can be set. If the lag is not specified, it will be set to be equal to the maximum source lag.

The minimum target lag is always set to 1 sample, to ensure so-called self-prediction optimality (Wibral, 2013, PLOS 8(2)). This is further discussed in the Wiki’s Theoretical Introduction.

settings = {'max_lag_sources': 16,
            'min_lag_sources': 6,
            'max_lag_target': 9}

重要性统计检验

The stats module provides the implementation of several statistical significance tests.

在网格推断中主要有五个重要性统计检验:

  1. Maximum test

  2. Minimum test

  3. Omnibus test

  4. Sequential maximum test

  5. FDR correction

关于它们的详细介绍和完整参数设置,请查看源码或说明文档:Documentationstats section

参数设置还是在setting里面,主要就是设置打乱次数和一个水平指标,这个指标代表什么含义呢?需要去查一下

settings = {'n_perm_max_stat': 200,
            'alpha_max_stat': 0.05,
            'permute_in_time': False}

Conditioning on specific variables

这个是为了对某些特定的变量做一个条件测试,设置方法如下所示,一般我们不会特别指明

In order to enforce the inclusion of specific variables in the condition set, add the add_conditionals key in the analysis settings dictionary and provide the list of variables to include. Each variable is defined as as tuple (process index, lag), e.g. (0,3) corresponds to the variable in process 0 with a lag of 3 samples with respect to the target.

settings = {'add_conditionals': [(0,3), (1,2)]}

By setting add_conditionals to faes, the algorithm adds the current value of all sources to the conditional set. This is meant to correct for instantaneous mixing in the source.

settings = {'add_conditionals': 'faes'}

分析过程

参数设置好了,我们就可以开始进行分析了

单个目标

主要就是使用analyse_single_target这个函数,可以设置targetsources

In order to restrict the analysis to a specific target process in the system, run the analyse_single_target method and provide target argument in addition to the required data and settings.

By default, when a target process is specified(e.g. process 0), all the other processes will be considered as potential sources.

data = Data()
data.generate_mute_data(n_samples=1000, n_replications=5)
Adding data with properties: 5 processes, 1000 samples, 5 replications
overwriting existing data
settings = {'cmi_estimator': 'JidtKraskovCMI',
            'max_lag_sources': 5,
            'min_lag_sources':1,
            'permute_in_time': False,
            'add_conditionals': [(0,3), (1,2)]}

add_conditionals 这个参数不能随便进行设置,刚才设置成fae的时候网络分析会报错

results = network_analysis.analyse_single_target(settings=settings,
                                                 data=data,
                                                 target=0,
                                                 sources=[1, 3])
Target: 0 - testing sources [1, 3]
Adding the following variables to the conditioning set: [(0, 3), (1, 2)].

---------------------------- (1) include source candidates
candidate set current source: [(1, 3), (1, 5), (1, 4), (1, 1)]
testing candidate: (1, 3) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200

这个例子计算起来好慢啊

这是什么错误呢?

问题好像是出现在add_conditionals上面

全网络分析

To analyse the whole network, use the analyse_network method and provide the required data and setting arguments.

results = network_analysis.analyse_network(settings=settings,
                                           data=data)
####### analysing target 0 of [0, 1, 2, 3, 4]

Target: 0 - testing sources [1, 2, 3, 4]
Adding the following variables to the conditioning set: [(0, 3), (1, 2)].

---------------------------- (1) include source candidates
candidate set current source: [(1, 3), (1, 5), (1, 4), (1, 1)]
testing candidate: (1, 3) 
WARNING: Number of replications is not sufficient to generate the desired number of surrogates. Permuting samples in time instead.
maximum statistic, n_perm: 200
testing candidate: (1, 4) maximum statistic, n_perm: 200

THe algorithm will loop over all processes in the network and first analyse each of them separately as a target. The results will finally be combined in a Results object. By default, an FDR correction will be performed when combining the results form different target1995.

In order to skip the FDR correction, refer to the tutorial on Statistical Significance Tests.

除此之外,我们还可以限制target variables 和source variables

The result class

IDTxl所有的计算结果保存在result当中,比如ActiveInformationStorage()MUltivaraiteTE(),还有单目标及全网络分析的结果以及参数设置属性列表也都在这里,直接查看属性即可

一些初步的应用:

  1. 使用result.settings查看参数设置

  2. 使用results.print_edge_list(weights='max_te_lag', fdr=False)打印出重要的信息流动和延迟

  3. 使用get_adjacency_matrix()可以查看关联关系矩阵

  4. 可以查看对某个单独目标的贡献

  5. 可以查看数据的性质

  6. 可以用来可视化

还有一些高级功能,如combine_resultsPartialInformationDecomposition等等,可以查看Documentation以及相关的论文,目前还用不到

可视化网络分析

下面我们来考虑下一个话题:Node dynamics !

节点动力学分析

Active Information Storage

这个例子其实就是对于单个的随机过程来进行分析的,就是看看\(X_n^{(k)}\)\(X_{n+1}\)的贡献,需要提前设置所要考察的最大的k值。

# %load ./IDTxl/demos/demo_active_information_storage.py
# Import classes
from idtxl.active_information_storage import ActiveInformationStorage
from idtxl.data import Data

# a) Generate test data
data = Data()
data.generate_mute_data(n_samples=1000, n_replications=5)

# b) Initialise analysis object and define settings
network_analysis = ActiveInformationStorage()
settings = {'cmi_estimator':  'JidtGaussianCMI',
            'max_lag': 5}

# c) Run analysis
results = network_analysis.analyse_network(settings=settings, data=data)

# d) Plot list of processes with significant AIS to console
print(results.get_significant_processes(fdr=False))

Partial Information Decomposition

下面这个是一个异或问题的例子,输入是两个变量,输出是一个变量

# %load ./IDTxl/demos/demo_partial_information_decomposition.py
# Import classes
import numpy as np
from idtxl.partial_information_decomposition import (
                                        PartialInformationDecomposition)
from idtxl.data import Data

# a) Generate test data
n = 100
alph = 2
x = np.random.randint(0, alph, n)
y = np.random.randint(0, alph, n)
z = np.logical_xor(x, y).astype(int)
data = Data(np.vstack((x, y, z)), 'ps', normalise=False)

# b) Initialise analysis object and define settings for both PID estimators
pid = PartialInformationDecomposition()
settings_tartu = {'pid_estimator': 'TartuPID', 'lags_pid': [0, 0]}
settings_sydney = {
    'alph_s1': alph,
    'alph_s2': alph,
    'alph_t': alph,
    'max_unsuc_swaps_row_parm': 60,
    'num_reps': 63,
    'max_iters': 1000,
    'pid_estimator': 'SydneyPID',
    'lags_pid': [0, 0]}

# c) Run Tartu estimator
results_tartu = pid.analyse_single_target(
    settings=settings_tartu, data=data, target=2, sources=[0, 1])

# d) Run Sydney estimator
pid = PartialInformationDecomposition()
results_sydney = pid.analyse_single_target(
    settings=settings_sydney, data=data, target=2, sources=[0, 1])

# e) Print results to console
print('\nLogical XOR')
print('Estimator            Sydney\t\tTartu\t\tExpected\n')
print('Uni s1               {0:.4f}\t\t{1:.4f}\t\t{2:.2f}'.format(
    results_sydney.get_single_target(2)['unq_s1'],
    results_tartu.get_single_target(2)['unq_s1'],
    0))
print('Uni s2               {0:.4f}\t\t{1:.4f}\t\t{2:.2f}'.format(
    results_sydney.get_single_target(2)['unq_s2'],
    results_tartu.get_single_target(2)['unq_s2'],
    0))
print('Shared s1_s2         {0:.4f}\t\t{1:.4f}\t\t{2:.2f}'.format(
    results_sydney.get_single_target(2)['shd_s1_s2'],
    results_tartu.get_single_target(2)['shd_s1_s2'],
    0))
print('Synergy s1_s2        {0:.4f}\t\t{1:.4f}\t\t{2:.2f}'.format(
    results_sydney.get_single_target(2)['syn_s1_s2'],
    results_tartu.get_single_target(2)['syn_s1_s2'],
    1))

The result is actually interesting.

可以看到结果还是很不错的,0.9705已经很接近于期望值1喽。

这个要去读一下相关的文献了:

  1. Non negative information decomposition

  2. Local information measure

实现细节

这部分内容的理论基础是:Local information Measure:具体可以查看2008,2010,2014年的那几篇文献,我们先来看看程序是什么样子~

具体同网路推断差不多:数据处理,设置模型参数,选择合适的估计方法,进行重要性统计检验,结果分析等等,具体参考Documentation

注意事项

  1. 参数太多,设置起来要小心