Multivariate Models, PCA, ICA and ICS

Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Multivariate Models, PCA, ICA and ICS
Klaus Nordhausen1
Hannu Oja1
1
2
Esa Ollila2
Tampere School of Public Health
University of Tampere
Department of Mathematical Science
University of Oulu
3
Department of Statistics
Rutgers - The State University of New Jersey
David E. Tyler3
R
Preliminaries
Multivariate Models
Whitening
Outline
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
R
PCA
ICA
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Important Matrices and Decompositions
• A p × p permutation matrix P is obtained by permuting the rows
and/or columns of the identity matrix Ip .
• The sign-change matrix J is a diagonal matrix with diagonal
elements ±1.
• The p × p matrix O is an orthogonal matrix if OT O = OOT = Ip .
• P and J are orthogonal matrices.
• D is a diagonal matrix.
• Let A be p × p matrix. The singular value decomposition (SVD)
of A is
A = O∗ DOT .
• Let A be p × p matrix. The eigenvalue decomposition of A is
A = ODOT .
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Affine transformation
Let x be a p-vector. An affine transformation has then the form
y = Ax + b,
where A is a full rank p × p matrix with SVD A = O∗ DOT and b is a
p − vector .
This transformation can be interpreted as a change in coordinate
system. The original system of x is first rotated / reflected using OT
then componentwise rescaled using D, again rotated / reflected using
O∗ and finally the origin of the system is shifted by b.
In statistics methods and estimates are preferred that do not depend
on the underlying coordinate system.
• A method like for example a hypothesis test is called affine
invariant if it does not depend on the coordinate system, i.e. the
p-value does not change when the coordinate system is
changed.
• Estimates are called affine equivariant if they follow a change in
the coordinate system in an appropriate way.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Location and Scatter Functionals
Let x be a p-variate random vector with cdf F .
A p-vector valued functional T = T(F ) = T(x) is called a location
functional if it is affine equivariant in the sense that
T(Ax + b) = AT(x) + b
for all full-rank p × p-matrices A and for all p-vectors b.
A p × p matrix S = S(F ) = S(x) is a scatter functional if it is affine
equivariant in the sense that
S(Ax + b) = AS(x)AT
for all full-rank p × p-matrices A and for all p-vectors b.
The corresponding sample versions are called location statistic and
scatter statistic.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Special Scatter Functionals
A symmetrized scatter functional version of a scatter functional S can
be obtained by using two independent copies x1 and x2 of x
Ssym (x) := S(x1 − x2 ).
A shape matrix is only affine equivariant in the sense that
S(Ax + b) ∝ AS(x)AT .
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Independence property
A scatter functional S has the independence property if S(x) is a
diagonal matrix for all x with independent margins.
Note that in general scatter functionals do not have the independence
property. Only the covariance matrix, the matrix of fourth moments
and symmetrized scatter matrices have this property.
If, however, x has independent and at least p − 1 symmetric
components all scatter matrices will be diagonal matrices.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
M-estimates of Location and Scatter
There are a lot of techniques to construct location and scatter
estimates. In this talk we will mainly use M-estimates.
M-estimators for Location (T) and Scatter (S) satisfy the following two
implicit equations:
−1
T = T(x) = (E(w1 (r )))
E (w1 (r )x)
and
S = S(x) = E w2 (r )(x − T)(x − T)T
for some suitably chosen weight functions w1 (r ) and w2 (r ). The
scalar r is the Mahalanobis distance between x and T, that is,
r = ||x − T||S .
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Examples of Scatter Matrices
The most common location and scatter statistics are the vector of
means E(x) and the regular covariance matrix COV(x).
Using those two and compute a so called 1 step M-estimator one
yields as a scatter the matrix of fourth moments.
COV4 (x) =
1
E(r 2 (x − E(x))(x − E(x))T )
p+2
Another famous M-estimator is Tyler’s shape matrix .
(x − T(x))(x − T(x))T
STyl (x) = pE
||x − T(x)||2S
The symmetrized version of Tyler’s shape matrix is known as
Dümbgens’s shape matrix.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Location - Scatter Model
All models discussed in this talk can be derived from the location
scatter model
x = Ω + µ,
where x = (x1 , . . . , xp )T is a p-variate random vector, = (1 , . . . , p )T
is a p-variate random vector standardized in a way explained later, Ω
a full rank p × p mixing matrix and µ a p-variate location vector. The
quantity Σ = ΩΩT is the scatter matrix parameter. For further
analysis the assumptions imposed on are crucial. Note however,
that the standardized vector is actually not observed, only x is
directly measureable. The vector is rather a mental construction
than something with a physical meaning. Yet, in some cases, can
have an interpretation and it might even be the goal of the analysis to
recover it when only x is observed. Either way, one of the first
challenges one faces in practical data analysis is to evaluate which
assumptions on can be justified for the data at hand.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Symmetric Models
A1: Multivariate normal model. has a standard
multivariate normal distribution N(0, Ip ).
A2: Elliptic model. has a spherical distribution around the
origin, i.e. O ∼ for all orthogonal p × p matrices O.
A3: Exchangeable sign-symmetric model. In this model is
symmetric around the origin in the sense that PJ ∼ for all permutation matrices P and sign change
matrices J.
A4: Sign-symmetric model. is symmetric around the origin
in the sense that J ∼ for all sign change matrices J.
A5: Central symmetric model. is symmetric around the
origin in the sense that − ∼ .
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Asymmetric Models
B1: Finite mixtures of elliptical distributions with
proportional scatter matrices. For a fixed k is
Pk
= i=1 pi (i + µi ), where p = (p1 , . . . , pk ) is
Pk
Multin(1, π) distributed with 0 ≤ πi ≤ 1 and i=1 πi = 1
and i ’s are all independent and follow A2.
B2: Skew-elliptical model. = sgn(∗p+1 − α − β∗p )∗ ,
where (∗ T , ∗p+1 )T satisfies A2, but with dimension
p + 1, and α, β ∈ R are constants.
B3: Independent component model. The components of are independent with E(i ) = 0 and Var (i ) = 1.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Relationships Between the Models
The symmetric models A1-A5 are ranked from the strongest
symmetry assumption to the weakest one, which means
A1 ⊂ A2 ⊂ A3 ⊂ A4 ⊂ A5.
The symmetric models can be seen as border cases of the
asymmetric models and we have the following relationships:
• A1 = A2 ∩ B3
• A2 ⊂ B1 and A2 ⊂ B2
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Uniqueness of Model parameters
• The location parameter µ is well-defined in models A1-A5. In
that case µ is the center of symmetry.
• The scatter parameter Σ is only well defined in models A1 and
B3. It is the covariance matrix.
• The mixing matrix Ω is not well defined in any of the models.
Further assumptions are needed in most models in order to have
well-defined parameters Ω and Σ.
Relation between location and scatter functionals in the different
models:
• Location statistics estimate in A1-A5 the same population
quantity, the center of symmetry.
• Scatter statistics measure in A1-A3 the same population
quantity. They measure the shape of data, i.e. S(x) ∝ Σ.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Data Transformations
In practical data analysis data is often transformed to different
coordinates. The transformation can be model based or not and may
be motivated by different reasons. Transformations we will have a
closer look at are:
• Whitening
• Principal components.
• Independent components.
• Invariant components.
Other transformations are for example:
• Canonical correlations.
• Factor analysis.
Most transformations are based usually on E(x) and COV(x).
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Whitening
The whitening transformation centers the variables and jointly
uncorrelates and scales them to have unit variances. The
transformation is
y = COV−1/2 (x)(x − E(x)),
where then
E(y) = 0
and COV(y) = Ip .
Properties:
• In A1 it makes the marginal variables independent.
• In A2 y will be spherical.
• It does usually not recover .
• The resulting coordinate system is not affine invariant. It only
holds that
1
1
COV− 2 (Ax + b)(Ax + b − E(Ax + b)) = O COV− 2 (x)(x − E(x)).
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Principal Components
The principal components also uncorrelates marginals. It does
however not scale them to have unit variances but chooses the new
variables successively in such a way that they are linear combination
of the original variables which have maximal variation under the
condition of being orthogonal with the previous ones.
The transformation is based on the eigenvalue decomposition of the
regular covariance COV(x) = ODOT . The principal components are
y = OT x,
with COV(y) = D.
Variations of PCA:
• centered PCA. The variables are first centered, i.e. x ← x − E(x).
• scaled PCA. Each variable is scaled to have unit variance, i.e.
xi ← xi /σi . Where σi2 is the variance of xi .
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Properties and Purpose of PCA
PCA has the properties:
• not affine invariant.
• not model based, only assumes existence of the first two
moments.
• lot of nice geometrical properties, especially in A1.
Main purposes:
• dimension reduction.
• in connection with other multivariate techniques. For example in
regression to avoid multicollinearity problems or in clustering.
• outlier detection.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
R
Comparison Whitening vs. PCA
In the following example 70 observations are sampled from a
bivariate normal distribution.
Original data
Whitenend data
5
5
4
4
4
3
●
●
● ●●
●
●
● ●
●
● ●
● ●●
●
● ●
●
●●●
●
●
●
●
●●●● ● ●●
●●
●
● ●
●
●
●
●
●
●● ●
●
●
●●
●●
●
●●
●
−5
−4
−3
●
−2●
●
−1
●
−1
1
2
●
●
●
●
●
●
2
2
●
●
●
●
●
4
5
−5
−4
−3
●
●
●
−2
● ●●
●
●
●
●
●
●● ● ●
●
● ●
●● ●
● ●●
●
●
● ●●●● ●
●●●
●
●●
● ●
● ●●
● ●
●
●
●
3
3
●
●
1
●
3
●
2
●
Principal components
5
1
●
−1
1
−1●
● ●
●
●
●
●
●●
●
●
●
●
●
●
2
●
3
4
5
−5
−4
●
−3●
●
●●
●
●
1
●
●●
●
●
●
●
●●
●
●
● ● ●●
●
●● ●
●
●● ●
●
●
● ●●
●
●
●●●
● ●
● ● ●●
●
●● ● ● ●
●● ●
●
●
●
●
●
●
●
−2
−1
1
−1
●
−2
−2 ●
−2
−3
−3
−3
−4
−4
−4
−5
−5
−5
2
●
●
●
●
3
●
4
5
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Robust Whitening and Robust PCA
The mean vector and the covariance matrix are not very robust and
therefore the transformations based on them can be dominated by
only a few influential observations.
A common method to robustify classical methods is to replace the
mean vector and the covariance matrix by more robust robust location
and scatter functionals. This of course leads only to the same method
if the functionals estimate the same population quantity. Which
basically means this is possible only in models A1-A3.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Independent Component Analysis
Independent component analysis (ICA) can be seen as a model
based transformation.
Let assume for simplicity that µ = 0. Then the IC model B3 reduces
to
x = Ω,
where has independent components.
The model is however ill defined since also
x = (ΩPDJ)(JD−1 PT ) = Ω∗ ∗
where ∗ has independent components.
Therefore the components can be rescaled, permuted and their sign
changed. To fix the scale a common assumption is E() = 0 and
COV() = Ip .
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Estimation of the Mixing Matrix
In ICA the main goal is to estimate the mixing matrix Ω respectively
an unmixing matrix Λ. This is possible up to sign and permutation if at
most one component are gaussian.
In some cases then the up to sign and order recovered underlying
independent components z = Λx have physical meaning.
Application areas of ICA of are:
• signal processing.
• medical image analysis.
• dimension reduction.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Common Structure of Most ICA Algorithms
Most ICA algorithms consist of the following two steps.
Step 1: Whitening. x is whitened so that E(x) = 0 and
COV(x) = Ip . Then
x = Oz∗
with an orthogonal matrix O and independent
components z∗ which have E(z∗ ) = 0 and
COV(z∗ ) = Ip .
Step 2: Rotation. Find an orthogonal matrix O whose columns
maximize (minimize) a chosen criterion g(oT x).
Common criteria are measures of marginal
nongaussianity (negentropy, moments) and likelihood
functions.
Well known algorithms are fastICA, FOBI, JADE, PearsonICA, . . .
R
Preliminaries
Multivariate Models
Whitening
PCA
Picture example for ICA
ICA
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
R
Multiple functionals for transformation
So far the transformations involved mainly one location and / or one
scatter functional. Recently is was suggested to use two location
functionals and / or two scatter functionals for this purpose.
Useful to recall in this context is that skewness can be interpreted as
the difference of two different location functionals and kurtosis as the
ratio of two different scatter functionals.
For example the classical univariate measures:
β1 (x) =
E3 (x) − E(x)
E((x − E(x))3 )
= p
var (x)3/2
var (x)
β2 (x) =
E((x − E(x))4 )
var4 (x)
=
var (x)2
var (x)
with
E3 (x) = E
with var4 (x) = E
x − E(x)
var (x)
x − E(x)
var (x)
x
(x − E(x))2
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Two Scatter Matrices for ICS
Two different scatter matrices S1 = S1 (x) and S2 = S2 (x) can be
used to find an invariant coordinate system (ICS) as follows:
Starting with S1 and S2 , define a p × p transformation matrix
B = B(x) and a diagonal matrix D = D(x) by
T
T
S−1
1 S2 B = B D
that is, B gives the eigenvectors of S−1
1 S2 . The following result can
then be shown to hold.
The transformation x → z = B(x)x yields an invariant coordinate
system in the sense that
B(Ax)(Ax) = JB(x)x
for some p × p sign change matrix J.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Interpretation of the ICS Transformation
This transformation can be seen as a double decorrelation. First x is
whitened using S1 and on the whitened data a PCA is performed with
respect to S2 .
Basically this can also be seen as a way to compare two scatter
matrices. If the PCA step finds still an interesting direction after the
whitening step then the two scatter matrices measure different
population quantities.
The diagonal elements of D can be seen as ratios of two different
scatter functionals. Hence the values can be seen as kurtosis
measures and therefore the invariant coordinates are according to
their kurtosis measures with respect to S1 and S2 .
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
An extension of ICS
The sign of the column vectors of B, respectively of the invariant
coordinates z is not fixed. One option is here to use two pairs (T1 , S1 )
and (T2 , S2 ) and fix the sign then such that
The the a way to construct the invariant coordinate system is to
• center x using T1 , i.e. x ← x − T1 (x)
• estimate B and the invariant coordinate z
• fix the signs of z so that T2 (z) ≥ 0.
In that case T2 (z) ≥ 0 can be seen as the difference of two location
measures and therefore as a measure of skewness.
If the D has distinct diagonal entries and T2 (z) > 0 the transformation
is well defined.
Denote this version of ICS from now on as centered ICS.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Applications of ICS
ICS is useful in the following context:
• Descriptive statistics and model selection (centered ICS).
• Outlier detection and clustering.
• Dimension reduction.
• ICA, if S1 and S2 have the independence property and the IC
model holds then B is an estimate of the unmixing matrix.
• In the transformation-retransformation framework of multivariate
nonparametric methods.
Some of the applications will be now discussed in more detail.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
On the choice of S1 and S2
As shown previously there are a lot of possibilities for S1 and S2 to
choose from. Since all the scatter matrices have different properties,
these can yield different invariant coordinate systems.
Unfortunately, there are so far no theoretic results about the optimal
choice. This is still an open research question.
Some comments are however already possible:
• For two given scatter matrices, the order has no effect
• Depending on the application in mind the scatter matrices should
fulfill some further conditions.
• Practise showed so far that for most data sets different choices
yield only minor differences.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Descriptive Statistics based on Centered ICS
Using the two pairs (T1 , S1 ) and (T2 , S2 ) then a data set can be
described via
• The location: T1 (x)
• The scatter: S1 (x)
• Measure of skewness: T2 (z)
• Measure of Kurtosis: S2 (z)
The last two measures can then be used to construct tests for
multinormality or ellipticity (see for example Kankainen et al. 2007).
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Moment Based Descriptive Statistics using ICS
An interesting choice for model selection is T1 = E(x), S1 = COV(x),
T2 = p1 E(||x − E(x)||2COV x), the vector of third moments, S2 the matrix
of fourth moments.
Denote D(k ) as the set of all positive definite diagonal matrices with
at most k distinct diagonal entries, then the values of s and D in the
eight models are:
Model
A1
A2
A3
A4
A5
B1
B2
B3
Skewness s
0
0
0
0
0
≥0
∝ ep or ∝ e1
≥0
Kurtosis D
Ip
D(1)
D(1)
D(p)
D(p)
D(k )
D(2)
D(p)
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Finding outliers, structure and clusters
In general, that the coordinates are ordered according to their
kurtosis is a useful feature. One can assume that interesting
directions are ones with extreme measures of kurtosis.
Outliers for example are usually shown in the first coordinate.
If the original data is coming from an elliptical distribution (A2), S1
and S2 measure the same population quantity and therefore the
values of D in that case should be all the same. For non-elliptical
distributions however they measure different quantities and therefore
the coordinates can reveal "hidden" structures.
In the case of x coming from an mixture of elliptical distributions (B1),
ICS estimates heuristically spoken to Fisher’s linear discriminant
space. (Without knowing the class labels!)
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
R
Finding the outliers
The modified wood gravity data is a classical data set for outlier
detection methods. It has 6 measurements on 20 observations
containing four outliers. Here S1 is a M-estimator based on t1 and S2
based on t2 .
●
●
●
●●●
0.50
●
●
●
●●
● ●
●
●
●
●
●
●
●
●●
●
●●
●
●●
0.60
●
●
●
●
● ●
●● ●
●
●
●
●
●
●
●
●●
●
13
●
● ●●
●
● ●
●
● ●
●
●
● ●
●●
●
●●
●
●● ●
●
●
●
●
● ● ●
0.55
●
● ●
● ●
●●
●
● ●
● ● ●
● ●
●●
●
●
●
● ●●
●●
●● ● ●
●
●
●
x5
●
●
●
●
●
●
● ●
●●
●
●
●
●
● ●
●●
●
●●
●
●
●
●
● ●●
●
●● ●
●
●
● ●●
●
●
●
●
●
y
●
●●
●
0.85
11
9
●
0.95
●
●●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●●
● ●
●
●●
●●
●
● ●●
●
●
●
●
IC.3
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
● ●
●●
● ● ●●
●●● ●
●
● ●
● ●
● ● ●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
● ●
●
●
●●
●●
●●
●
● ●
●●● ● ●
● ● ●●
●
●
●
●
●
●
●
●
●●
●
IC.4
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●● ● ●
● ●
●
●
●●
●
●
●●●
● ●● ●
●
● ● ●
●
● ●●
●
●●
●
●●●
●
●
● ●
−28
−24
●
●
●
●● ●
●
●
●
● ●
●
●
●
●
●●
● ●●
●●
●
●
●
● ●●
IC.5
14
16
●●
●
●
●
18
●
● ●● ● ●●●●
● ●●
●
●
●
●
●●
−30
−24
●
●●
●●
●
●●
●
● ●
●
●
●●
●
●●●
●
●●●●
●●
●
●●
●
●
●
●
●
●
●
●
● ●● ●
●●
●
●
●● ●
−28
●
●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
● ●●
●
●●
● ●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●●
●
●●
● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●
●
●
0.0
●
●
●
●
●●
● ●
●
● ●● ●
●●
●
●●
●
●
●
●
●
●
●
●
●
IC.2
●
● ●
●
●
●●
●
●
● ●
●
● ●
●
x4
●
●
●
●
●
●
●●
●
●●
●
●● ●
●
−1.5
●
●
●
●
●
● ●●●●
●
●●
● ●● ●
●
−28
●
● ●
●
●
●● ●●
●
●● ● ●
●
●●
●
●
●
●
●●
0.40
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
14 16 18
●
●● ●
●
●
●● ●
● ● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
● ●
●
●●
●●
●
●
●
●
● ●●● ●
● ● ●
●
● ●
●●●●
●●
●
●
●
●
●
●
●●
●
●
0.45
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
● ●
●
● ●
●●
●
●
●
●●
●
●
● ●
●●
●
●●
●
●
●● ●
●●
●
●
3
●
●
●
●
●
1
●
●
●
●
●●
●
● ●
● ●
IC.6
●
●
−28
●
●
●●
●
●
●
●
● ●
●
● ●●
●
●
●
●●
●
● ● ●
●
●
●
●
● ●
●●
3
●
●●
●
●●
●●
●
●
●
●
●
x3
●
●●
●●
●
●
●
●●
1
●
●
●
●
● ●
● ●
●
−1
●
●
●
●
IC.1
−1
●
● ●
●●
●
●
●
13
0.0
0.45
●●●
●●
●
● ●
●
●
●
●
●●
●
● ●
●
●
●●
●
●
● ●
●
●
●
●
● ● ●
●
●●
●●
●
●●●
● ●
●
●
●●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
11
●
●●
●
●
●●
● ●● ● ●
●
●
●●
● ●
●
● ●●
●
●
9
●
−1.5
0.60
●
●
●
●
●
0.55
●
●
●
0.50
●
● ●
●
● ●
●●
● ●● ●
●
●
●●
●●
x2
●
● ●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
0.40
0.11
●●
●
●● ●
● ●
●●
0.40
●●
●
●
●●
●
●
●
●
● ● ●
●
●
●●
●
●
●●
●●
●
●
0.95
0.15
●
●
●
●
0.40
●
●
● ● ●
●
● ● ●
●
●
● ●
●
●
●
●●
●●
●
●
0.85
●●
●
●● ●
●
● ●● ●
●
●
x1
Invariant coordinates
0.45 0.55
●
●●
●
●
0.45 0.60
0.15
●
−30
Original data
0.11
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
R
Finding the structure I
Original data
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●● ●
●
●
●
●
●●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
● ●●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ● ●● ●
●●
●
V2
●
●
●
●●
●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
● ●
●
●
●
●
●●
●
●● ●●
●
●●●●
●
●●
● ●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
● ●●
●
●●
●
●
●
●
●●●
●● ●
●●●●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●●●
●●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●●● ●
●
●●●
●
●
●
●
●● ●●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
● ●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●● ●● ●
●
● ●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●●
●●
●
●●
●●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●●
●
V3
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●●
●●
●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●● ●
● ●
●
●●●
●●
●
●
●●
●
●●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●●●
●
● ●●●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●●
●
●●●
●●●
●●
●
●
●●●●
●
●●
●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
● ●●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
● ●●●
●
●●
●
●●
●●● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●●
●
●
●
●●●●
●
V4
● ●●
●
● ●
●
● ●●
●
●
●
●
●
●
●● ●●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
● ●●
●●●
●
●
●
●●●●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
● ●●
●
● ●
●
●
●●
●●
● ●●●
●
●
●
●
●●
●●
● ●●
●●●
●
●
●
●
●
●●
●
●
●
●●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●●
●
● ●●●●●
●
●
● ●
●
●
●●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●●
●● ●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●●
●●●
●
●
●
●
●●
●
●
●●●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●● ●
●
●
●●
●● ●
●
●
●●
●
●● ●
●●
●
● ●●●
● ●
●●
●●
●
●●
●
●
●●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
V5
●
●●
●
●
●
●
●
●●
●●●
●●
●●
●
●
●
●
●
●●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●●●●●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●●
●
●
●●●
●●
●
●
●● ●
●
●●● ●
●
●
● ●●
●
●
●●
●●●●
●●
●
●●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●●●
●
●●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●●●
● ●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●●
●●
●●
●
●●
●●●
●●
● ● ●
●
●
●
●
●●
●
●●●●
●●
●●
●●● ●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●●
● ●●
●
●
● ● ●●● ●
●●
●
●
●
●●
●
●●
●●
●●●
●
●●
●
●● ●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●●
●●●
●● ●●●
●
●
●
●
●●●
●●
●
●● ●●
●●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●
●●●●
●
●●●● ● ●
●
●
●
● ●●
●
●●
●●●
●
●
V6
●
●
−5
0
5
−3
0
2
−6
−2
2
5
0 5
●●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
● ●●
● ●● ●●
●●
●
−5 0
−10
● ●
●
●
●● ●
●
●
●
●●
●
●●●
● ●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
● ●●●
● ●●
●
2
4
0
0
●●●
●●
●●
●
●●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●●●●
●
●
●●
●
●
●
●
●●
●●
● ●●
●
−3
−4
●●●
●●
●●
●●
●
●●
●
●●
●
●
●
●●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●● ●●
●
●●
●
●
●
●●
● ●
●
●●
●
2
10
0
−10
4
0
−4
10
0
−10
10
●●●
●
● ●● ●●
●●
●●●
●
●●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
● ●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●●●●
● ●● ●●
●
●
V1
The following data set is
simulated and has 400
observations for 6
variables. It looks like an
elliptical data set.
0
−6 −2
−10
Multivariate Models
Whitening
PCA
ICA
ICS
Finding the structure II
PCA
0.5
●
●●
●●
●
●●
● ●
●
●●●
●
●
●● ●
●
●
●
●●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●●●
● ●
●●●
●
●
●
● ● ●
● ●
●
● ●●
●
●● ●
●●●●
●●●
●
●
●
●
●
●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●●
●●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●●
●
●●
●
●●
●●
●●
●
●●●
● ●
●●
●●
●●
●
●
● ●●
●●
●
●
● ●
●
●●
●
● ●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●●
●
●●
●
●●●
●●
●
●
●●
●
● ●●
●● ●
●
●
●●●
●
●
●
● ● ●●
●●
●
●●●
●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●●
●
●
●● ●
●●●
●
●●
●
●●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●●
●●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
● ●●●
●
●
●
●
● ●
Comp.3
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●● ●●●
●
●
●
●
●●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●● ●
●●
● ●
●
●●
● ●●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●
●
●●●
●
●
● ●
●● ●●
●●
● ●●
●
●●
●
●●●
●
●
●●
●●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
● ●
●●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●●●●●
●
●
●
●
● ●●
●
● ●
●●
● ● ●● ●
●●
●●● ●●
●
●●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
●●
Comp.4
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●●●
●
●●
● ●●
●
●●
● ●
●
●
●●
●
●●
●● ●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
● ●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●●
●
●●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●● ● ●
●●
●
●
●●
●●● ●●●
●●
●●
●
●
●● ●●
●
●
●
●
● ●
●
●●
●
●
●●● ●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●●
●●
●●
●●
●
●●
● ●●● ●
●
● ●●
●●
●●
●●
●●
●
● ●
●●● ●●
●●
●
●●
●
●●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●●
●
●●●●
●
●
●
● ●● ●●●
●● ●●●
● ●
●
●
●●●
●
●
●
●
●
●●
●●●●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
● ●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●●
●●
●
●●
●
●
●●
●●
●●
●● ●
●●
● ●
Comp.5
●● ●
●●
●
●●●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
●● ● ●
●
●●
●●●●
●
●
●
●● ●●●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●● ●
●●
●●
●
●
●
●
●
●
●●
●● ●● ● ●
●
●
●
●●●●●●
●●●
●● ●
●
●●●●●
●●
●●
●●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●●
●●●
●●
●
●
●
●
●●
●●
●
●
●
●
●●●
● ●●● ●
●
●
●●●●●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
● ●
●
●
● ●●●●●
●
●
●
●
●● ●
●
●
●●
●●
●
●
●●●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●
●●●●
●
●
●
●●● ●
●●
●
●●●
●●
●
●●
●
●● ●
●
●
●
● ●●● ●●
● ● ●●
●●
●
●
● ●●● ●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●●●●●
●●
●
●
●
●●
●● ●
●●
●
●●
●● ●●
●
●●
●
●
●
●
●
●
●
Comp.6
−10 0
10
−6 −2
2
6
−2
0
2
5
Comp.2
−10
● ●
●
●
●
●
●●
●
●
●
●●● ●
●
●
●
●●
●
●
●
●
●●●●● ●
●● ●
●
●●
●●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●
●●●●
●●
●●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
● ●●
●●
● ●
●●
4
Comp.1
●
●●
● ●
●
●●
● ●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
● ●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●●
●
●
●
●●●
●
●
●
●
●●●●
●
●●●●
●●
●
●● ●●
15
0.5
●
● ●
●
●● ● ●
● ●●●
●
●
●
●
●
●●●
●●
●
●
●●
●
●●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
0
−0.5
●
● ●●
● ●●
●
●●
●
●
●
● ●●
●
●
●
●
●
●●●●
●
●●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●●
●
●●
● ●●●
●●
●●
●
●
● ●●●
●
●●
●
●
●●●
●●●
●
●●●
●
●
●
●●
−6
2 4
●
● ●●
●●
● ●●
●
●
●
●
●
●
●●
●
●
●●
●
● ●●●
●●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●●
●
● ● ● ●
2
−2
●
●
●●
●●
●
●
●●
●
●●●
●
●● ●
●●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●●
●
●●
●●●
●
●●
●●
●●
●
●●
●● ●
●
●
●
●
● ●● ●
0
5
−2
−2
2 4
−5
5
−5 0
−0.5
Preliminaries
R
Multivariate Models
Whitening
PCA
ICA
ICS
Finding the structure III
ICS
● ● ●
●
●
●●●
●
●●
●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
● ●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●
●
●● ●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
● ●
−3
0
2
4
●
●
●
●
● ●● ●●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●● ●
●● ●●
●●
●●
●
●●●● ●
●
●● ● ●
●
● ●●
●●
●
●●●●●
●
●●●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●●●●●
●
●
●
●
●●●
●●
●
●●●
●
●
●
● ●●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
● ●● ●
●
●●●
●●
●●●
●●
●
● ●●
●●
●●
●
●
●
●●●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●●
●
●●●
●● ●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●●
●●
●●
●●●●
●●●
●
●●
●● ●
●●
● ●
●
●
●
●
●
●
●●●●●
● ●
●
●
●●
●●
●
●
●
●●
●●●●●
●
●●
●
●
●●
●
●
●
●●●
●
●●
●
●
●●
●
●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●● ●●
●●
●
●●
●●●
●
●● ● ●●
●●●●
●
● ● ●
●●● ●
●
●●●
●
●
●●●●
●●
●
●
●
●
●
●
● ●●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●●
●
●
● ●●●
●●
●
●
●
●
●
●
●
●
●● ●● ●
●
IC.4
●
●
●
●
●●
●●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
● ●●
●●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●●
●
●
●●
●
●●
●
●
●
●
●●
●●
●●
●
●●
●●
●
●●●●
●
●●
●
●
●
●
●
●● ●
●
●
●●●● ●
●
● ● ●
●●●
● ●●
●●
●
●●● ●●
●
●
●●●
●
●●
●
●●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●●
● ●●
●
●●● ●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
● ●
●
●
●●
●
●
●●●
●●●●
●
●●●
●●
●●●
●
●●●
●● ●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●●
●
●
●
●● ●●
● ● ●
●
IC.5
●
●
●● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
● ●
●
●
●●
● ●●
●
●●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●
●
●
● ●●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
● ●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●● ●●●
●
●●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
● ●
●
●●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●●●●●●
●
●
●
●
●
●
●● ●
●
●●●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
IC.6
−3
0
2
−2
0
2
2
IC.3
●
●
●●● ●
● ●
●●
●
●●
●●
●
● ●●
●
●●
●
●●
●
●
●
●● ●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●● ●
●●
●
●●
●●
●●●
●
● ●
●
●
●●●● ●
● ●
● ●
●
0
●
●
●●●
●● ●
● ●
●●
●
● ●●
●●
●
●●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
● ●●
●
●
●●
●●
●●
●
●
2
●●
●●
●●
●
●●
●
●
●
●●●●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●● ●
●
●●
●
●
●
●●
4
0 1 2
●
●
●
● ●● ●● ●●
●
●
●●
●
●
●● ●
●
●●●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●●
●
● ●
●●●● ●● ● ●●
●●
●
0
−2
●
●
●
●● ● ●
●●●
●●
●
●● ●●
●●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
● ●
●
●●
●●
●
●
●●●
●
●
● ●● ●
−3
2
−3
●
0
●
● ●
●● ●
● ● ●
●
● ●●
●●●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●● ●
● ●
●
●●●
●●●
●
● ●●
●● ●
●●
●
●●
2
2
0
−2
IC.2
−2
●
●●●
●
●
●
●
●
●
●
●●●
●
●●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●●
2
●
●
● ● ●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
● ●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●●
●●
●
●● ● ●
●
●●● ●
●●
●
●
● ●
● ●
●●
●●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
0
2
0
●
●
●
●●
●●● ●
●
●●
●
●●
●
●
●
●
●
●
●● ●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
−4
−1
2
IC.1
−1
−2
−4
−2
Preliminaries
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
R
Clustering and dimension reduction
10
−1.5
●
1.0
0.0
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●●●●
●
●● ●
●
● ●
●●
●●
●
●
●
●
●
●
● ●●
●●●
●●
●● ●
●
●●
● ●● ● ●
●●●
●●
● ●
●●
●
●●
●
●●●●
●●
●
●
●
● ●●
●●
●●
●
●
●
●● ● ● ●
●
●
● ●● ●●● ●
●
●
● ● ●●
●
●●●
●
●●
● ●●
●
●
●●
●
● ●
●
●●
●
●● ●●
●
●
●
●
IC.2
●
●●
●
●● ● ●
● ●● ●
●
● ●● ●
● ●
● ●
●●
● ●●●
●
●
● ● ●●●
●
●● ●
●
●●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●●●
●
●●
● ●●●●
●
●
●●●
● ●●●
●
●●
●
●
●●● ●●
●●● ●
● ●
●
●
●●●
●●
●●●
●●●
●●●●
●
●
●
●
●●● ● ●● ●
●
●●●
● ●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
● ●
●● ●●
●
●
●
●
●●● ●
●
●● ●
●
● ●●●●
●●● ● ●● ● ● ●●●
● ● ●
●
●●
●
●●● ●
●
●
● ●
●
●
●
● ●●
●
●●●
●
●
● ●
●●●● ● ● ●●● ●
●
●
●
●
●
●
●
●
●●●● ●
●●
●● ● ● ● ●
●●
●
●
●
●
● ●●
●
●
● ●●● ●●
● ●●
●
●●
●●
● ● ●●
●●
● ●
● ●● ●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●●●
●
●●●
●●●●
●●
●● ●
●
●●
●
●
● ●
●●●
●●
●●
●
●●
●
●●
●
●●
●
●
●
●●●●●
●●
●●●
● ●
●● ●●●●●
● ●●●
●●●
●
●
●●
●
●
●
● ●● ●
●● ●●●●
●● ●
●
●
●
●●●
●●● ●
●●●●● ●
●
●
●● ● ● ● ●
●
●
5
6
7
IC.3
●
9
1.0
●
●●
●● ● ● ●
●
● ●●
● ● ●●
●●
● ● ●●
●●●
●
●●
●
●● ●
●●●
●
●
●
●● ●
● ●●●●
●●
● ●●
●
●
●
●
●
●
●
●●
●●●
●
● ●
●●
● ●●●
●
●●●
●●●
●
●●
●
●●
●●
●● ●●
●●
●
●
●
●● ●
●●
●●●●
●●
●●●
●●● ●●●
●
● ● ●●●
●
●
● ●
●
● ●●● ●●
●
● ●
●
● ●●
● ●
●
●● ●
● ●●
●●● ●
● ●
●●
●●●
●
●● ●●
●
● ●●
●●
●
●●
●
●
● ●●●●
●
●
●
●● ● ● ●
●
●
●
4
5
6
7
9
7
8
●
● ● ●
●● ●
●
●
●
●●
●
●
● ●
●●
●
●
●●
●
●
●
●●
●
●●
●●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●● ●●
● ● ●
● ●●
●●
●● ● ●●●●●
●●
●●
●
●● ●●
●
●
● ●●
●
●
●●
●
3
2.0
●●●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●●
●●
●
●
●
●● ●●
●●●●
●
●
●●
●●●
●
●
●●
●
●●
●
●●●● ●
●●
●●
●
● ●●●●
● ●
●● ● ●
●●
●
●●
●●
●
● ●
●
●●●
●●●●
●
●
● ●●
● ●
● ●
●●●● ● ●
● ●
●● ●●
●
● ● ●●
● ●
●
8
●
●●
●
●●
●●
●●●●●
●●●
●
●●
● ● ●●●
●
● ● ●●
●
●● ●●
●● ●
●
●
●
● ●●●
●
●● ●
●
● ●●● ●●
●●●
●
●
●● ●
●
●
●
●●●
●● ●
● ●
●●
●
●
●●●
●●
●
●
●
●●
● ●
●
●
●
●
●
4
●
●
●
● ●● ●
● ●●● ●
●●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●● ●
●
●
●●●
●●
●
●●
●
●
●
●
● ●
● ● ●● ●●●
●
●
●
●
● ●● ●
●
●● ● ●
●
●
● ●●
●● ●●
● ●●● ●
●● ●● ●●
●● ●
●
●
●● ●
●
●●●● ●●
●
● ● ●
● ●
●●● ●●
● ●
●
● ● ●●●● ●● ●
● ●●
●
●
●
●
●
●
●
●
●●●
● ●●
●●
●●
●●
●●●●
● ●
●●
●●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
2.0
5
6
7
8
9
10
●
−1.5
To illustrate clustering via
an ICS we use the Iris
data set.
The different species are
colored differently and we
can see, that the last
component is enough for
doing the clustering.
0.0
●●
●●●
●
●
●
●
●●
● ●●
●●
●
●
●●
●
● ● ● ●
●
●
●●
●
●● ●● ●●
●
●
●
●
●
●●
● ●●●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
● ●● ●●●●
●● ●
●●
●
●● ●
●
●●
●
●
6
IC.1
●
● ●●
●
●
●
●
●
●●
●
●
● ●●●●●
●●
●●
● ●●●
●
●●● ●
● ●●
● ● ●●●
● ●
● ●●
●● ● ● ●
●●● ●
●
●● ●
● ●
●●
●
●
●●
●
●● ● ●
●● ●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
●
●
●●●●
●●●●
●
●● ● ●
●
●
●
●
●
●
●
●
●● ●
●●●●●
●
●
●
● ● ●
5
●
●
●
4
●
IC.4
● ●
●
●●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●●● ●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
● ●
●
●●
7
9
6
8
●● ●
●
●
●
●
●
●
●● ●
● ●
● ● ●●
●
● ●● ●●●● ●●● ●●
●●
●●
●●
●
●
●
●
● ●●●
●●
●
●
●
● ●●
●
●●
●●●●
●
●
●● ●●
●●
● ●●
●
●
●
●
●●●● ●
●
●●●
●
●●
●
●
●● ●●●
●●
●●●
●
●●
● ●●
●● ● ● ●●
●●●
● ●● ●● ●●
● ●
●
● ●
● ● ●
●●
● ●
●
5
7
4
6
●
3
5
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
ICS and ICA
When the IC model holds then the ICS transformation recovers the
independent components upto scale, sign and permutation if
• S1 and S2 have the independence property.
• the diagonal elements of D are all distinct.
In the case of S1 = COV and S2 = COV4 this corresponds to the
FOBI algorithm and the condition of D having distinct elements means
the independent components must have all different kurtosis values.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Implementation in R
The two scatter transformation for ICS / ICA is implemented in R in
the package ICS. The main function ics can take the name of two
functions for S1 and S2 or two in advance computed scatter matrices.
The R package offers then several functions to work with an ics
object and offers also several scatter matrices and two tests of
multinormality.
Basically everything shown in this talk can be easily done in R using
the methods of base R and the package ICS.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
R
Australian Athletes Data, a More Detailed Example
Using R
We have now a closer look at the Australian Athletes data set using
PCA and ICS on the four variables:
• Body Mass Index
• Body Fat
• Sum of Skin Folds
• Lean Body Mass
>
>
>
>
+
>
library(DAAG)
library(ICS)
data(ais)
data.ana <- data.frame(BMI=ais$bmi, Bfat=ais$pcBfat,
ssf=ais$ssf, LBM=ais$lbm)
pairs(data.ana)
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Australian Athletes Data, a More Detailed Example
Using R
> pairs(data.ana)
35
25
15
100
80
●
●
●
●
●
●
●
●
● ●
●
●●
●
●●●
●●
●●
●
●
●●●●●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
●●
●●
●●●●
●● ●
● ●
●
20
25
●
30
35
●
●
●
●●●
●
●●
●
●
●●
●●
●●
●
● ●●
●
●●
●
●●
●●
●
●
●●
●
●●
● ●●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
● ●●
●
●●
●●
●
●●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
● ● ●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●● ● ●
●●
●
●●
● ●●
●
●
●
●●
●
●
● ●●●
●●
● ●●
●
●●
●
● ●
●
●
●
●
●●
●
●●
●
●●
● ●
● ●
●
●●
●
●
●● ●
●
●●
●●●●
● ● ● ●● ●
●●
●
●●● ●
●●●
●●
●●
●
ssf
●
●
●
●
● ●
● ●
●●● ●
● ●
●
●
● ●
●
●●
●● ● ●
●●
●
●●● ●
●
●
● ●●
●
●
●●
●
●●●
●
●●●
●
●
● ●● ●
●● ●
●
●● ● ●●
●● ●
●
● ●
●●
●
●
●●
●
● ●
●
●●
●
●●●●●●● ●●
● ●●
●●●
●
●
●●
●
●
●●
●
●●
●
●●
●●●
●
●
● ●
●
●
●
●●
●
●
●●
●●
●
●●
●●
●●●●
●●
●
●● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●
●●
● ●●
●
● ●●
●
●
●
●
●●●
●
●●
●●●
● ●●
●
●
●
●●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●●
●●
● ●● ●
●
● ● ● ●●
●
●●
●●●●●
● ●● ●
●
●●
●
●
●
●
●
● ●●●
●●
●●
●
●● ●●●
●
●
●
●
●
●
●
●
●
●● ● ●
●●●●
● ●●
●
●●●●●●
●
●●●
●
●
●
●
●
●
●
●●●●
● ● ● ●●
●
●●●
●●
●
●
●
50
100 150 200
35
30
●
●
●
● ●
● ●●● ●
●
●
●
●
●
●●
● ●●
●
● ●
●●●●
●
●
●●
●
●●
●●
●●●● ●
●
●
●
●
●● ● ●
●
●●●
●●● ●
●
●
●
●●
● ●● ● ●
●●●●●
●
●
● ●
●●● ●
●●
●
●●●
●
●
●
●
●
●
●
●● ● ● ● ●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●●●
●
● ●●
●
●
●●
●
●●●
●●
●
●
●
●
●
●●●
● ●●
●●
●
●
●●
●●
●
●●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●● ●
● ●
●
●●
●●
●● ●
●
●
●●●●
●●●●
●●●● ●
●●● ●
●
● ●
●●●●● ●
●
●● ●
●
●
●
●
●●
●●
●●
●●
●●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●●
●●
●
●
●● ●
●
●●
●● ●●●
●●●
●
●●
●●
●
● ●
●●
●
● ●●
25
● ● ●
●
●
●●
●
● ●●● ● ●●
●●●
●● ●
●
● ●●
●●●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●●
●●● ●●
●
●
●●
●
●
●
●
●●
●● ●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
● ●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●● ● ●●●
●
●●●
●● ●
● ●●
●
●
●
●●
●
●
●● ●
●
● ●● ●
●
●●●
●
●
100
●
●
Bfat
80
●
●
●●●
●
●●
●
●
●●
●
●●●
●●
●
●●●
●●
●
●●
●
●
●
●
●
●●
●●
●●●
●●
●
●
●
●●
●
●●
●
●● ●
●
●
●
●●
●
●
●
●●●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
60
●
60
●
●
●
●
●●
●
● ●● ●
●
●
●
●
● ●●
●
●
●
●
● ●
●● ●
●
●
●
●
●
●
●●
● ●
●●●●
●●●●●
●
●
●
●
● ●
●●
●
●
●● ●●
●
●●
●
●
●
●
● ●
●●
●●
●●
●
●
●
● ●
●●
●●●●● ●● ●
●●● ●
●
●
● ● ●●●● ●
●
● ●
●
●●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
● ●●●●●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●●
●
●●
●●
●●● ●
●
●
●
●
●● ●●
● ●
●●
●
●
●
●●●●
●● ●
●●
●●
● ●
●●
●
● ●
● ●
●●
●●●
●
●●
●
●● ●●
●
●
● ●● ●● ●
●
●●
●
●●● ●
●●
●●●
●●
●●●●
●●
●●
●
●
●● ●●
●
●
●●●
●
●●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
● ●●
●
●●
●
40
40
●
●
20
35
●
● ●●
● ●
●●
● ● ● ●●
● ●
●
● ●
● ●●
●
●●
●●●
●●●●●
●
●●●
●
●●
●
● ● ●●●● ●
●
●
●●
●●
●
●
●
●
●
●● ●●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●●●●●
●●
●
● ●● ●
●
●
●●
●
●
●
●
●
●
●● ● ●●● ●
●
●●
●● ●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●●● ●●
●● ●
● ●
●
●
●
●
●●●
●
●●
● ●●
●
● ●
●
●
●
●
●●
●
●
● ● ●
● ●
● ●
●
●● ●
BMI
5
25
●
LBM
100 150 200
15
●
●
50
5
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Australian Athletes Data II
Performing the PCA
> pca.AS<-princomp(data.ana,score=TRUE)
> pairs(pca.AS$scores)
●
●
● ●● ● ●
●
●● ●
●●● ●
●
●
●
●
● ● ●●
●●
● ●●
●●
●● ●
●
●●
●
●
●●
● ●●●
●●
●
●●
● ●●
●●●
●
●
●● ●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●●
● ●●● ●●●
● ● ●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●● ●● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●●●
●●●
●
●
●
●●
●● ●●
●
●
● ●
●●
●
● ●
●●
●
● ●●
●●
● ●
●
●
●
●
4
●
●
2
0
−2
●
●
●
●●
●
● ● ● ●●
●●●
●● ●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●●●
●
●●
●● ●
●
●
●
● ●●●●●
●●●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●●
●●●●●
●
●
● ●●
●
●●
●
● ● ●●
●
●
●
●
● ●●●
●●●
●
●
●
●
●
● ●
●●●●●
● ●●
●
●
●
●
●● ● ● ●
●●
●
●● ●
●●
●● ●
●●
●●
●
● ●●
−100
0
●
●
●
●
●
●
● ●
●
●●● ● ●
●
●●●●●●
●●
● ●
●
●●
●●●
●
●●
●●● ●●●
●●
●
●●
●
●
●●
●●
●
●●● ●
●
●●●●
●
●●●
●
●
●
●●●
●●
●
●●
●●
●
●
●●
●●
●●
●●
●
●●●
●
●
●●
●●
●
●
●●
●
●●●●●
●
●●
●
●
●
●● ●
● ●●●
● ●●
●●
●
●
●●
● ●
●
● ●●● ●●
●
●
● ●
●
●●●●
●
●● ●
● ●
●
●
●
●
●
●●
●
●
●
●●
●●●
● ●
●●●● ●
●●
0
●
●
●●
●
●
●●
●● ●● ● ●●●
● ●
●●
●
●●
●
●●
●●
●
●
●
●
●●
●●●
●
● ●●●●●
●
● ●●●
●
●
●●
●●
●
●
●●
●● ●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●●
●●
● ●●●
●●●
●
●●
●
●
●●●
●
●●●●
●●
●
●
● ●
●● ● ●
●
●
●
●
● ●●●
● ●●●●●●●
●
●
● ●
●
●● ●●●
●●
●●
●●
●●
● ●●
●
●
●
●●
●
●
●●
●●
●
●
● ●●
● ●
●
●● ● ●
●● ●
●
● ● ●
●
●
●
●
●
●
●
●
●● ●● ●
●
●
●●●●
● ●●● ●●
● ● ●●●●●●●●●
●●
●
●
●
●
●
● ●
●●●●
●●
● ●●●●
●
● ●
●●
●●●●
●
●
● ●
●
● ●
●●●
●●
●●●
●●●●●
●
●●
●●●●
●
●
● ●●
●
● ●●●●
● ●
●●
●
●●
●
●●
●
●
●
●
●●●●
●
●●
●●
● ●●
●
●● ●
●
●
●● ●●
●
●●●
●●●●
●●
●●
●●
●
●
● ● ●●
●
●●●
● ●●
● ● ● ●●
●●
●●●●●●●
●
●
●
●
●
Comp.3
●
●
●
●●
● ●●
●
● ●●● ●●
●●
●●
●●● ●●
●
●
●●●
●●
●●●
●
● ● ● ●●●●●●
●●●
●●
● ●● ●
● ●
●●
●●
●
●●
●
●
●●
●
●● ●
●
●● ●
●
●
●● ● ●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●●
●●
●
●●
●●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●●
●
●
●●
●
●
●
● ●●
●
●
●●
●●
●
●
● ●
●
●●
●
●
●● ●●
●●
●
●
●●● ●●
●
●
●
●● ● ● ●
●
●●
●
● ● ●
● ● ●
●
●
●
●
●
●
●
●
●●●
●● ● ●
●● ●
●
●
●● ● ●
● ●●
●●
●● ● ●
●●
●●
●●
●●●●●
●
●
●
●
●
●●●●●
●●
●●
●
●
●●
●●● ●
●●
●●●
●
●
●●
●●
●●
●●
●●●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●●●
● ● ●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●●● ●
● ●●
●
●● ●●
●
●
●
●
●
● ●● ●
● ●● ●●●●●
●
●●●
●●
● ●●●
●●
●
●
● ●● ●
●●
●
Comp.4
●
●
●
●
●
●
●
● ● ●● ● ●
●
●
●
●●●●
●
● ● ●
●●
● ●●
●●● ●●
●
●●●● ●●●
●
● ● ●●
●
●●
● ●● ●
●
● ●● ●
●●
●●●
●
●●● ●
●
● ●●
●
●
●●
●
●●
●●
●
● ●● ●
●
●
●●
●
●
●●●●
●
● ●
●●
●
●●●
●●
●
●●
●
●●
●●
●
●
●●●
●
●
●
●●
●
●
●● ●
●
●●●
●
●●
●●
●
●
●●●
●
●
●●●
●
●
●
● ●
● ●
●●
● ●●● ●● ●●
●
●●●●●●●
●●●
●
●
● ● ● ●
● ●
●●
●
4
●●
●
−4
−2
0
2
4
4
●
Comp.2
2
2
−40
0
20
●
●
● ●
●
●● ●
●● ●●●●
●●
● ●
●●
●●
●
●
●
●
●
●●●
●
●
●● ●
●
●●
●●
●
●
●● ●
●
●●
●
●
● ●●
●
●●
●●
●●
●
●
●●
●
●●
●●●
●●
●
●
●
●
●●●
●● ●●●
●
●
●
●●
●●●
● ●●
●
●
●
●
●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
● ● ● ●●
●●
●
●●
●
●
●●
●●
●
●
● ●● ●●
●●
●●
●●
●●
● ● ●●
●
●● ●
●● ●●●
●
●
●
●
0
●
●
●●
●
●
●●
●●
● ●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●
●●●
●
●●
●●
●●●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●●
● ●●●
●
●
●●●
●●
●
●
●●
●
●
●
● ●
●●●●
●● ●
● ●
●
●
●●
●
●
●
●●
●●
●
●
●● ● ●●
● ● ● ●●
●
●●●● ● ●●●
● ●●
●●
●
●
●
●
● ●● ●
●
●
●
●●●
●
●●
●
● ●
● ●
● ●
●
●
●
●
●
0
Comp.1
−2
●
●
●
●
●
●
●
●●
●
●●●●●
●●●● ●●
●
●
●
●●
●
●●●●●●
●●
● ●●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●● ● ●● ●
●●●
●
●
● ● ●●
● ●
●
●
● ●●
●● ●
●
●● ●
●●
●
●●
● ● ●●●●
●●●●
●
● ● ●● ● ●●
●
●
●
●● ●
● ●●● ●
●
●
●
●
● ●● ●● ●
●
●
●
●
●
●
●
●●●●● ● ●
●
●● ●
●● ●● ●
●
●
●
●
●
●
●
−100
20
−2
0
●
●●●
●
●● ●
● ● ●●
●●
●
●
●●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●●● ●
●●
●●
●
●
●
●
● ●
●●●
●
●●
●
●
●
●●
●
●
●●●●●
●●
●
●
●●
●
●●●● ●
●
●●
●
●●
●●● ●●●
●
●
●●
●● ●
●
●
●●●
●●
●●
●
● ●●●●
●●
●
●
● ●●
● ●●●
●
●●●●
●● ● ●●
●
●
●
●
●●●
●
●
●●
● ●●●
●
●●
●
● ●
●●
●
● ●●●
● ●●
●
●
● ●
●
●
●
●
●
●
●
−4
−40
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Australian Athletes Data III
> pairs(pca.AS$scores,col=ais$sex)
20
0
−40
●
4
●
●
● ●● ● ●
●
●● ●
●●● ●
● ●
●
●
●
● ● ●●
●●
● ●●
●●
●● ●
●
●●
●
●
●●
● ●●●
●●
●
●●
● ●●
●●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
● ●●● ● ● ● ●
●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●● ●●●●
●
●
●●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
● ● ●● ●● ●
●●
●
●
●●●
●●●●
●
●
●
●●
●● ●●
●
●
●
● ●
●●
●
●● ●
● ●
● ●
●●
● ●
●
●
●
●
●
2
●
●
●●
●
● ● ● ●●
●●●
●● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●●●
●
●●
●● ●
●
●
●
●●●
●
●
●●
●
●●●●
●
●●●●●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●●
●●●●●
●
●
●
● ●●●
●
●
●
● ● ●●
●
●
●●
●
●
●
●
●
●
●
● ● ●● ●● ●●●
●
●●
●
● ●●●
●
● ●●
●
●
●
●● ●● ●
●●
●● ●
●●
●●
●
−100
0
●
●
●
●
●
●● ●● ●
●
●
●●●●
● ●●● ●●
● ● ●●●●●●●●●
●●
●
●
●
●
●
● ●
●●●●
●●
● ●●●●
●
● ●
●●
●●●●
●
●
● ●
●
● ●
●●●
●●
●●●
●●●●●
●
●●
●●●●
●
●
● ●●
●
● ●●●●
● ●
●●
●●
●●
●●
●●
●
●
●● ●
●●
●●
●●●
●
●● ●
●●
●
●
●● ●●
●
●●●
●●●●
●●
●●
●
●●
●
●
●● ●
●
●●●
● ●●
● ● ● ●●
●●
●●●●●●●
0
2
●
●
●
●
●
●
●
●
●●●
●● ● ●
●● ●
●
●
●● ● ●
● ●●
●●
●● ● ●
●
●●
●●
●●●●●
●
●
●
●
●
●
●●●●●
●●
●●
●
●
●●
●●● ●
●●
●●●
●
●
●●
●●
●●
●●
●
●●●
●
●
●
●
●●
● ●●
●● ●●
●●
●●
●●
●●●●
●
●●
●
●
● ● ●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●●●● ●
●
●●
●
●
●
●
●
●
● ● ●
●
● ●● ●
● ●● ●●●●●
●
●●● ●
●●
●●●
●●
●
●
● ●● ●
●●
●
●●
●
−2
0
●
●
●
●
●
●●
● ●●
●
● ●●● ●●
●●
●●
●●● ●●
●
●
●●●
●●
●●●
●
● ● ● ●●●●●●
●●●
●●
● ●● ●
● ●
●●
●●
●
●●
●
●
●●
●
●● ●
●
●
●
●
●
●
●● ● ●●
●
●
●● ●
●
●●
● ●●●●
●
●
●●
●
●
●
●●
●
●●
●●
●
●●●
●
●
●
●
●
●●
●●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●●
● ●
●
●●
●
●
●● ●●
●●
●
●
●●● ●●
●
●
●
●● ● ● ●
●
●●
●
● ● ●
● ● ●
●
●
●
●
Comp.3
−4
●
●
●
●
●●
●
●
●●
●● ●● ● ●●●
● ●
●●
●
●●
●
●●
●●
●
●
●
●
●●
●●●
●
● ●●●●●
●
● ●●●
●
●
●●
●●
●
●
●●
●● ●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●●
● ●●●
●●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
● ●
●● ● ●
●
●●● ●
● ●●●
● ●●●●●
●●
●●
● ●
●
●● ●●●
●
●
●
●●
●
●
●
●
●
●
●●
●● ●●●
●● ●●
● ●●
● ●
● ● ● ●
●● ● ●
● ●
●
● ● ●
●
●
●
●
●
●
●
●
●
● ●
●
●●● ● ●
●
●●●●●●
●●
● ●
●
●●
●●●
●
●●
●●● ●●●
●●
●
●●
●
●
●●
●●
●
●●● ●
●
●●●●
●
●●●
●
●
●
●●●
●●
●
●●
●●
●
●
●●
●●
●●
●●
●
●●●
●
●
●●
●●
●
●
●●
●
●●●●●
●
●●
●
●
●
●● ●
●●●●
● ●●
●●
●●
●
●
●●
●● ●
●
●
●
●
●
● ●●●
●●●●●
●
● ●
● ●
●
●
●
●
●
●●
●
●
●
●●
●●●
● ●
●●●● ●
●●
●
●
●
●
● ● ●● ● ●
●
●
●
●●●●
●
● ● ●
●●
● ●●
●●● ●●
●
●●●● ●●●
●
● ● ●●
●
●●
● ●● ●
●
● ●● ●
●●
●●●
●
●●● ●
●
● ●●
●
●
●●
●
●●
●●
●
● ●● ●
●
●
●●
●
●●●●
●
● ●
●●
●
●●
●●
●
●●
●
●●
●●
●●
●
●
●●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●● ● ●
●
●●●
●
●●●
●
●
● ●
●●
● ●●● ●● ●●
●
●●●●●●●
●●●
●
●
● ● ● ●
● ●
●●
●
●
●
● ●●
−2
0
●
●
●
●
Comp.2
4
4
Comp.4
4
●
●
2
2
●
●
● ●
●
●● ●
●● ●●●●
●●
● ●
●●
●●
●
●
●
●
●
●●●
●
●
●● ●
●
●●
●●
●
●
●● ●
●
●●
●
●
● ●●
●
●●
●●
●●
●
●
●●
●
●●
●●●
●●
●
●
●
●
●●●
●● ●●●
●
●
●
●●
●●●
● ●●
●
●
●
●
●
●
●●
●
●
●
● ●
●●
●
●●●
●
●
●● ● ●● ●
●
●●
●●
●
●
●
● ● ● ●●
●●
●
●●
●
●
●
●●
●
●
●●
● ●● ●●
●●
●●
●●
● ● ●●
●
●
●● ●
●● ●●●
●
●
●
●
0
●
●
●●
●
●
●●
●●
● ●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●
●●●
●
●●
●●
●●●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●●
● ●●●
●
●
●●●
●●
●
●
●●
●
●
●
● ●
●●●●
●● ●
● ●
●
●
●●
●
●
●
●●
●●
●
●
●● ● ●●
● ● ● ●●
●
●●●● ● ●●●
● ●●
●
● ●●
● ●
● ●●
●
●
●
●
●●●
●
●●
●
●
●
● ●
● ●
●
0
●
Comp.1
−2
●
●●
●
●
●
●
●
●
●●●●●
●
●●●● ●●
●
●
●
●●
●
●●●●●●
●●
● ●●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●● ● ●● ●
●●●
●
●
● ● ●●
● ●
●
●
● ●●
●● ●
●
●● ●
●●
●
●●
● ● ●●●●
●●●●
●
● ● ●● ● ●●
●
●
●
●● ●
●●●●● ●
●
●
●
●
●
●
●
●● ●● ● ●●
●
●
●●●●● ● ●
●
●● ●
●
●
●
● ●
●
●
●
●
●
●
●
−100
20
−2
0
●
●●●
●
●● ●
● ● ●●
●●
●
●
●●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●●● ●
●●
●●
●
●
●
●
● ●
●●●
●
●●
●
●
●
●●
●
●
●●●●●
●●
●
●
●●
●
●●●● ●
●
●●
●
●●
●●● ●●●
●
●
●●
●● ●
●
●
●●●
●●
●●
●
● ●●●●
●●
●
●
● ●●
● ●●●
●
●●●●
●● ● ●●
●
●
●
●
●●●
●
●
● ● ●●
●●●
●
● ● ● ●●●
●
● ●●●
● ●●
● ● ●●
●
●
●
●
−4
−40
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Australian Athletes Data IV
Descriptive statistics using ICS
> LOCATION <- colMeans(data.ana)
> LOCATION
BMI
Bfat
ssf
LBM
22.95589 13.50743 69.02178 64.87371
>
> SCATTER <- cov(data.ana)
> SCATTER
BMI
Bfat
ssf
LBM
BMI
8.202111
3.324882
29.94890 26.72126
Bfat 3.324882 38.313946 194.11891 -29.27451
ssf 29.948901 194.118912 1060.50092 -88.42529
LBM 26.721255 -29.274514 -88.42529 170.83006
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
R
Australian Athletes Data V
>
>
+
>
data.ana.centered <- sweep(data.ana, 2, LOCATION, "-")
ics.AS <- ics(data.ana.centered, stdKurt = FALSE,
stdB="Z")
summary(ics.AS)
ICS based on two scatter matrices
S1: cov
S2: cov4
The generalized kurtosis measures of the components are:
[1] 1.7154 1.2788 0.9353 0.7659
The Unmixing matrix is:
[,1]
[,2]
[,3]
[,4]
[1,] 0.1777 -0.4586 0.0975 -0.0527
[2,] 0.5371 0.1698 -0.0552 -0.0418
[3,] 0.3740 -0.3108 0.0256 -0.1471
[4,] 0.1142 0.4964 -0.0775 -0.0147
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Australian Athletes Data VI
>
+
>
>
>
Z <- sweep(ics.components(ics.AS),2,
sign(mean3(ics.components(ics.AS))),"*")
SKEWNESS <- mean3(Z)
SKEWNESS
IC.1
IC.2
IC.3
IC.4
0.48211688 0.09281212 0.14727342 0.16866117
> KURTOSIS <- ics.AS@gKurt
> KURTOSIS
[1] 1.7154201 1.2787838 0.9352998 0.7658729
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Australian Athletes Data VII
> pairs(Z)
0 1 2 3
−2
2
1
0
●
●
●●
● ●
●●
●●
●● ● ●
●
●●
●●
● ●
●
●●●
●
●●
●
●●● ● ●
●
●
●
●
●●
●
●●● ●
●●●●●
●
●
●●●
●
●●●●
●
●●
●●
●
●
●
●●●
●●
●●
●●●
●●
●●● ●
●
●●
●
●●●
●●●
●●● ●●
●
●●●●●
●●
●
●●●
●
●
●
●
●
●●
●●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●●
●●●
●
●
●●
●
●●
●
●●●
●
●●
●●
●● ●
● ●●●
●
●
IC.2
●
●
●
● ●●●
●●
●● ●●
●
●
● ●●
●●
●
● ●
●● ●
●●
●
● ●● ●
●●
●●
● ●
●
●
●
●●● ●●●
●
●
●●
●
●●●
●
●●
●●
●●
●●●
●
●●●
●● ●● ● ●
●
●
●
●
●
●●
●●
●
●●
●● ●
●
●●●
●
●●
● ●●●●●● ●● ●
●● ●●●● ●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●●
●
●●
●●
●
● ●●●●
●
●
●
−2
0 1 2 3 4
●
●●
●
●
●● ● ●
●● ●
●
●
● ●●
●
● ●
●●
●
● ● ● ●
●● ● ●● ●●●
●
● ●●●
●
●
●●
● ● ●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●●●
●●
●
●
●●
●●
●●
●
● ●
●
●
●●
●
●
●
●●
●
●
●
●●●●
●
●●
●●●
●●
● ●●●
●
●●
●
●
●
●
●
●
●●●
●
●●
●●●
●
●
●
●●
● ●
●●
●●●
●●
●●●● ●
●
●●● ●
●● ●●
●
●● ●●● ●●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
● ●●
● ●●
●●
● ●
●
●
● ● ●●
●
●
●
●
●
●●
●● ●
●●
●
●
●● ●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●●● ●●●
●● ● ●● ●
●
●
●
●●
●
●● ●● ●
●●
●●
●
● ●●
●
●
●
●
●
●
●●
●●
●
● ●●
● ●●
●● ●
●
●
●
● ●
●
●
●
●● ●
●●
●
●
●●
● ●●
●
●● ●
●●
●●● ● ●
●
●
● ●●●
●
●●●
●●●
●
●
●●
●
●
●●
●
● ●
● ●
●
●●
●
●●● ●
●●●
●
●
● ●●●
●
●●●●●● ● ● ●
●●●
●●●●●
●●●
●●
●●●●
●
●
●● ●
●●
●
● ●●●
●
●●
●●●●
● ●● ●●
●●●
●●●
●
●
●
●
● ●●
●
●
●●
●
●●●
●●
●
●
●● ●
●
●
●●
●
●●
●
●
●●
●●●
●
●●
●
●
●
●●
●●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
● ●●
●● ●●●
●
●
●●● ●●
●●●●
●● ● ●
●
●●●
● ●
●●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●●
●●●
●●
●●●
●
●
●●
●●● ●●
●
●● ●
●
●
●●
●
●●
●
●
●
●
●
●● ● ● ●
●
●●●
●
●
● ●
●●●
●●
●
●●●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●●● ●
●●
●
●
●
●●
●●●
●●
●●
●●
●
●●●
●●
●
●
●
●
●
● ●
●
●● ●●●● ●
● ●
●
● ●
●
●
●
●
●
●
●
●
●●
● ● ●
●
●
●
●
●●
●
●
● ●●●● ●
●
●
● ●● ●
●
●●●
●●
●●●
●●
●
●●●
●
●●
●
●
●
●
●
●●●
●●●
●
●
●●● ●
●
●
●
●
●
● ●●●
●
●
●
●
●●
●
●
●
●
●
●●
● ●●●
●●●●
●
●●●●
●●
●●
●●
●
●
●
●●●
●● ● ●
●●
● ●
●
●●●
●
●●
●
●●
●●
●
● ●●
●
●
●
● ●●
● ●
●
●
●●
●
●
●●
●
●
●●●
●● ●
●
●
●●●●● ●●
● ●●● ● ●●
●
●
●
● ● ●
●
●
●●● ●
●●
●
● ●
●●●
● ●●
●●●
●
●
●●
●●
●
●
●
●
●●●●●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●●
●●
● ●●●
●
●●●
●●
●
●●●
●●●●
●●
●
●
● ●●
●
●
●●
●●
●
●
●
●●
●●●
●●
●●
● ●
●●●●
●●
●
●●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●● ● ●
●
●●●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●●
●●●●
●●●
● ●
●● ●
●●
●
2
●
●
0 1 2 3 4
●
●
●
● ●
●
●
●●
●●
●
● ●●●
●●
● ●●
●●
●
●●
● ● ●● ●●
●●
●●●
●●●●
●
●●● ●●●●●●
●● ●
● ●●
●
●●
●
●
●
●●●●●
●
●●
●●
●●
●
● ●●
●● ●
●
●
●
●
●●
●●
●
●● ●●● ●
● ●
● ●
●●
●●
●
●●●●
●
●
●●
●●
●●●
●
●
●
●●●●●
●
●
● ●●
●
●
●● ●
●●
●
●
●●● ●
●●
● ● ●●
●
●
●●●
●
●● ● ●●
●●
●
●
●● ●
●
●
●
●●
● ●
●
●
● ●● ●
●
●
●
●
IC.3
●
●
●
●●
●
● ●●
●●
●
● ● ●●●
●
●●
●●●
● ●●
●● ● ●
●● ●
●●
●● ●
●
●
●
●●●
●●●●
●
●●● ●●●
●●
●
● ●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
● ● ●
●●● ●
●●
●●
●●
●
●
●●●●● ●
●
● ● ●● ● ●●
● ●●●
●● ●
●
●●
●
●●●
●
●●
●
●
●●
● ●●
●
●
●●
●
●●●
●●●
●●
●
●
●●
●●●
●●
●
●
●● ● ●●●
●● ●
●
●
●
● ● ●
● ●
● ●●●●
●●
●
●●
●●
●
●●
−3
−1
1 2 3
IC.4
●
1 2 3
●
●
●
●
●
●
● ●● ● ●
●● ●
●
● ●●● ●
●
●
●
●
●
●●● ●
●
●
●●
●●
●●
●●
●
●●●●
●●
●●●
●
●
●●
●●
●
●●●●
●●●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
● ●●●●●
●●●
●
●
● ●
●
●●●●●●●
●
●
● ●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●● ●
●●
● ●
●
●
●
●
●
●
●
●
●
● ●●● ●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●●●
●●
●
●
● ●
●
●
●
●
●
●●●
●●
●●●●
●
●
●
●●
●●
●●
●
●
●
●
●
●●●●●
●●
● ●
● ●
●
●●●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●●●
●●
●
●● ●
● ●
●
−2
●
1
●
●
●
−1
−2
0 1 2 3
IC.1
0
●
●
−2
●
●
●●
−3
−2
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Australian Athletes Data VIII
> pairs(Z, col=ais$sex)
0 1 2 3
−2
2
1
0
●
●
●●
● ●
●●
●●
●● ● ●
●
●●
●●
● ●
●
●●●
●
●●
●
●●● ● ●
●
●
●
●
●●
●
●●● ●
●●●●●
●
●
●●●
●
●●●●
●
●●
●●
●
●
●
●●●
●●
●●
●●●
●●
●●● ●
●
●●
●
●●●
●●●
●●● ●●
●
●●●●●
●●
●
●●●
●
●
●
●
●
●●
●●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●●
●●●
●
●
●●
●
●●
●
●●●
●
●●
●●
●● ●
● ●●●
●
●
IC.2
●
●
●
● ●●●
●●
●● ●●
●
●
● ●●
●●
●
● ●
●● ●
●●
●
● ●● ●
●●
●●
● ●
●
●
●
●●● ●●●
●
●
●●
●
●●●
●
●●
●●
●●
●●●
●
●●●
●● ●● ● ●
●
●
●
●
●
●●
●●
●
●●
●● ●
●
●●●
●
●●
● ●●●●●● ●● ●
●● ●●●● ●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●●
●
●●
●●
●
● ●●●●
●
●
●
−2
0 1 2 3 4
●
●●
●
●
●● ● ●
●● ●
●
●
● ●●
●
● ●
●●
●
● ● ● ●
●● ● ●● ●●●
●
● ●●●
●
●
●●
● ● ●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●●●
●●
●
●
●●
●●
●●
●
● ●
●
●
●●
●
●
●
●●
●
●
●
●●●●
●
●●
●●●
●●
● ●●●
●
●●
●
●
●
●
●
●
●●●
●
●●
●●●
●
●
●
●●
● ●
●●
●●●
●●
●●●● ●
●
●●● ●
●● ●●
●
●● ●●● ●●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
● ●●
● ●●
●●
● ●
●
●
● ● ●●
●
●
●
●
●
●●
●● ●
●●
●
●
●● ●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●●● ●●●
●● ● ●● ●
●
●
●
●●
●
●● ●● ●
●●
●●
●
● ●●
●
●
●
●
●
●
●●
●●
●
● ●●
● ●●
●● ●
●
●
●
● ●
●
●
●
●● ●
●●
●
●
●●
● ●●
●
●● ●
●●
●●● ● ●
●
●
● ●●●
●
●●●
●●●
●
●
●●
●
●
●●
●
● ●
● ●
●
●●
●
●●● ●
●●●
●
●
● ●●●
●
●●●●●● ● ● ●
●●●
●●●●●
●●●
●●
●●●●
●
●
●● ●
●●
●
● ●●●
●
●●
●●●●
● ●● ●●
●●●
●●●
●
●
●
●
● ●●
●
●
●●
●
●●●
●●
●
●
●● ●
●
●
●●
●
●●
●
●
●●
●●●
●
●●
●
●
●
●●
●●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
● ●●
●● ●●●
●
●
●●● ●●
●●●●
●● ● ●
●
●●●
● ●
●●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●●
●●●
●●
●●●
●
●
●●
●●● ●●
●
●● ●
●
●
●●
●
●●
●
●
●
●
●
●● ● ● ●
●
●●●
●
●
● ●
●●●
●●
●
●●●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●●● ●
●●
●
●
●
●●
●●●
●●
●●
●●
●
●●●
●●
●
●
●
●
●
● ●
●
●● ●●●● ●
● ●
●
● ●
●
●
●
●
●
●
●
●
●●
● ● ●
●
●
●
●
●●
●
●
● ●●●● ●
●
●
● ●● ●
●
●●●
●●
●●●
●●
●
●●●
●
●●
●
●
●
●
●
●●●
●●●
●
●
●●● ●
●
●
●
●
●
● ●●●
●
●
●
●
●●
●
●
●
●
●
●●
● ●●●
●●●●
●
●●●●
●●
●●
●●
●
●
●
●●●
●● ● ●
●●
● ●
●
●●●
●
●●
●
●●
●●
●
● ●●
●
●
●
● ●●
● ●
●
●
●●
●
●
●●
●
●
●●●
●● ●
●
●
●●●●● ●●
● ●●● ● ●●
●
●
●
● ● ●
●
●
●●● ●
●●
●
● ●
●●●
● ●●
●●●
●
●
●●
●●
●
●
●
●
●●●●●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●●
●●
● ●●●
●
●●●
●●
●
●●●
●●●●
●●
●
●
● ●●
●
●
●●
●●
●
●
●
●●
●●●
●●
●●
● ●
●●●●
●●
●
●●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●● ● ●
●
●●●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●●
●●●●
●●●
● ●
●● ●
●●
●
2
●
●
0 1 2 3 4
●
●
●
● ●
●
●
●●
●●
●
● ●●●
●●
● ●●
●●
●
●●
● ● ●● ●●
●●
●●●
●●●●
●
●●● ●●●●●●
●● ●
● ●●
●
●●
●
●
●
●●●●●
●
●●
●●
●●
●
● ●●
●● ●
●
●
●
●
●●
●●
●
●● ●●● ●
● ●
● ●
●●
●●
●
●●●●
●
●
●●
●●
●●●
●
●
●
●●●●●
●
●
● ●●
●
●
●● ●
●●
●
●
●●● ●
●●
● ● ●●
●
●
●●●
●
●● ● ●●
●●
●
●
●● ●
●
●
●
●●
● ●
●
●
● ●● ●
●
●
●
●
IC.3
●
●
●
●●
●
● ●●
●●
●
● ● ●●●
●
●●
●●●
● ●●
●● ● ●
●● ●
●●
●● ●
●
●
●
●●●
●●●●
●
●●● ●●●
●●
●
● ●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
● ● ●
●●● ●
●●
●●
●●
●
●
●●●●● ●
●
● ● ●● ● ●●
● ●●●
●● ●
●
●●
●
●●●
●
●●
●
●
●●
● ●●
●
●
●●
●
●●●
●●●
●●
●
●
●●
●●●
●●
●
●
●● ● ●●●
●● ●
●
●
●
● ● ●
● ●
● ●●●●
●●
●
●●
●●
●
●●
−3
−1
1 2 3
IC.4
●
1 2 3
●
●
●
●
●
●
● ●● ● ●
●● ●
●
● ●●● ●
●
●
●
●
●
●●● ●
●
●
●●
●●
●●
●●
●
●●●●
●●
●●●
●
●
●●
●●
●
●●●●
●●●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
● ●●●●●
●●●
●
●
● ●
●
●●●●●●●
●
●
● ●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●● ●
●●
● ●
●
●
●
●
●
●
●
●
●
● ●●● ●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●●●
●●
●
●
● ●
●
●
●
●
●
●●●
●●
●●●●
●
●
●
●●
●●
●●
●
●
●
●
●
●●●●●
●●
● ●
● ●
●
●●●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●●●
●●
●
●● ●
● ●
●
−2
●
1
●
●
●
−1
−2
0 1 2 3
IC.1
0
●
●
−2
●
●
●●
−3
−2
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Australian Athletes Data IX
Analyzing only the females
> data.ana.females<-data.ana[ais$sex=="f",]
> pca.AS.f<-princomp(data.ana.females,score=TRUE)
> pairs(pca.AS.f$scores)
●●
●
●
Comp.1
5
15
−3
●
●●
●● ● ●●●●
●
● ●
●●●
●● ●
●
●
● ●● ● ● ●
●● ●● ●
●
●●
●
●●
●
● ●● ●
● ●●
●●● ●●●●
●
● ● ●●●
● ●●●
● ●
● ●
●
●●
● ● ●●
●● ●
●
●
●● ●●
●
● ●
●
●
●
−15
−5
5
15
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●● ● ●
●●
●
● ●●
●● ● ●
● ●●
● ●
●
●
●●
●● ●●
●●●
●
●●
●●
●●●
●●●
●
●●
● ●●
●
●
●●
●●
●● ● ●
●
●● ●
●● ● ● ●
● ●
● ●●
●
●
● ●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
3
1
−3
−1
●
−100
●
●
●●
●
●●
●
●
●
●●●●
● ●
●●●●
●
●●
●
●●
●●
●●
●
● ●
●
●
● ●●
●
●● ●●●
●●●
● ● ●●
●
●
● ●
● ●
● ●
● ●●● ●●●
●
● ● ●●●
●● ●
● ● ●●
●
●
●
●
● ● ●●●
●
●
●
●
0
50
●
●
● ●●
●●
●
● ●●
● ●● ●
● ●
●● ●
●●
●● ● ●●
●
●
●
● ● ●●
●●●
●●● ● ●
●● ● ●
●
● ●
● ●● ●
●●●●
●●
●
●
●
●
●
●
●●● ● ● ●
● ●
●
●● ● ●
●
●
●
●
●
●
● ●
●
●
●
●
● ●●
●
● ●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
● ●
●
●
●
●
● ●●
●●
● ● ●●
●
●●
●● ●●
●●
●● ●
●● ● ●●
● ●
● ● ●●●
●● ●
●●
●
●
●
●
●
●●
●● ●● ●● ●
●●
●●
●● ●
●
● ● ●
●
●
●
●● ●
●
●
●
●
●●● ●
●
●● ●
●
● ●●●
●●
● ●●●
●●
●●
●● ●
● ●●
●
●
●●
●●●
●●
●
●
●
●
●
●
●●
● ● ● ● ● ● ●● ●
●
●
●
● ●●● ● ●
●●● ●●●
● ● ● ●●
●
●
●●
●●
●
●●
●
●●●
●
●
●
●
●
● ●●●
●
●
● ●
● ●● ●●●
●
● ●● ●●●
● ●
● ●●
●
● ●
●
● ●
● ●●
● ●●
●
●●● ●
● ● ●● ●●●
●
●●
●
●●
● ● ● ● ●● ●●
●● ● ●
●
●●
●● ●
● ● ●●
●
●
●
●
●
●
●
● ●
●
●
Comp.3
●
●
●
●
●● ●●
●●
●
●●
●
●●●●●● ● ●●
●●●●
●
●● ● ●
●●● ● ●
● ●
●
●● ●
●
●● ● ● ●●●●
●
● ●●● ●
●●
●
● ● ●
● ●● ●
●
●
●
●● ● ● ● ● ●●
●
●
●● ●●
●
●●
●
● ●●●
●
●
●
−2
0
2
●
4
Comp.4
0
●
●
●
−2
●
●
2
4
●
3
●
●
●
●
●
●● ●
●
●
● ●● ● ● ●
●
●
●●
●● ● ● ●
●
●
●●
● ● ●
●● ●
●
●●
●
●
●●
●
●●
●
●
●●●●
●● ●
●● ●●
●
●● ●
●● ●●● ●
● ●
●
●●●
● ●● ● ●
●
●●
● ●●
●
●● ●
●●
●
● ● ●
●
●
●
Comp.2
1
●
●
●●
●
●● ●
●
●
●
●● ●● ●
●
● ●●
●
●● ● ●
●● ●
●
● ● ●●
●
●
●
●●
●
● ●
●
●●●●
●●●
●
●●●
●
●●
●
●
●●●●
●● ● ●
●● ●
● ●
● ●●●
● ●●●●
●
●● ●
●
● ●
●
●
●●
●●
●
●●
●
●
●
−1
●
● ●
●●
●●
●●
● ● ●●
●
●
●
● ●●
●● ●●●
●● ●
● ●●●
●●
● ●● ●
●●
●
●
●●
● ●
●
● ● ●●
●
●
●
●
●●●
●● ●● ● ●●●
●●
●
● ●●
●● ● ● ● ●● ●● ●●
●
●
●●●
●●
●●
●●
●
●
50
−5
● ●● ● ●
●
●● ●● ●●●
●
●
● ● ●●
● ●
● ●
●●
●●
●●
●●●
●
●
●
●●● ● ●
●● ●
● ●● ●● ● ●
●
● ●●●
●
● ● ●
●●●
●
●●
●
●●
● ● ●
●
●
●
● ● ● ●● ●
●
● ●●
●
●
●
●
●
●
●
0
●
−100
−15
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Australian Athletes Data X
Analyzing only the females
> LOCATION <- colMeans(data.ana.females)
> LOCATION
BMI
Bfat
ssf
LBM
21.9892 17.8491 86.9730 54.8949
>
> SCATTER <- cov(data.ana.females)
> SCATTER
BMI
Bfat
ssf
LBM
BMI
6.969747
9.508302
60.63405 13.66024
Bfat 9.508302 29.734018 178.96012 15.33174
ssf 60.634049 178.960117 1145.86058 95.24906
LBM 13.660242 15.331741
95.24906 47.91675
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
R
Australian Athletes Data XI
> data.ana.f.centered <- sweep(data.ana.females, 2,
+ LOCATION, "-")
> ics.AS.f <- ics(data.ana.f.centered, stdKurt = FALSE,
+ stdB="Z")
> summary(ics.AS.f)
ICS based on two scatter matrices
S1: cov
S2: cov4
The generalized kurtosis measures of the components are:
[1] 1.3583 1.1320 1.0322 0.8330
The Unmixing matrix is:
[,1]
[,2]
[,3]
[,4]
[1,] 0.2996 -0.3870 0.0640 -0.0034
[2,] -0.2892 -0.0861 0.0496 -0.0496
[3,] 0.5923 0.1401 -0.0378 -0.2158
[4,] 0.0414 0.6212 -0.0863 0.0232
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Australian Athletes Data XII
>
+
>
>
>
Z.f <- sweep(ics.components(ics.AS.f),2,
sign(mean3(ics.components(ics.AS.f))),"*")
SKEWNESS <- mean3(Z.f)
SKEWNESS
IC.1
IC.2
IC.3
IC.4
0.32308843 0.16111629 0.13685658 0.06543187
> KURTOSIS <- ics.AS.f@gKurt
> KURTOSIS
[1] 1.3583203 1.1320473 1.0322094 0.8329946
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
Australian Athletes Data XIII
> pairs(Z.f)
0
1
2
3
−2 −1
0
●
1
2
●
3
●
4
−2
●
●
3
●
●
−2
0
1
2
●
●
●
●
●
●
●
●
●●
● ● ●●
● ●
●● ●
●●
●
●
●●
● ●● ●
● ●●
● ●
●
● ●
● ● ●●
●
●●●
●●●●
●●
●
●
●●●
●
●●
●●
●
●● ●
●
●●
●●
●
●
●
●●
●
●●●
● ● ● ●
● ●
● ●
●
● ● ●
●
●
●
IC.2
●
2
1
●
IC.3
●
●
●
●
● ● ●● ●●
● ● ●● ●●● ● ● ●
● ●
●
●● ● ●●●●
●
●●●●
●● ● ●
● ● ●●
●● ●
●●
● ● ● ●● ●● ●
●
● ●● ●
●
● ●●
●
● ●●●●
●●●
● ● ●●
●
●●
● ●
● ●
●●
●
●
●
●
2
●
●
●
● ●●
● ●●●
●
●
● ●● ●
● ●●
● ●●
●●
●
●
●●
● ●●●●
●
●●
●
●●●●●
●
●
●● ●
●●●●
●
●
●
●
●●● ●●
●
●
● ●●
●
● ● ●● ● ●
●
●●●
●●
●●●● ●●●● ●
● ●
●
● ●● ●
●
●
● ●
●●
●
●
●●
●●
●
● ●●●
●●
●
●
● ●
●● ● ●
● ●
●
●
● ● ● ●●
●
●
●
●
●●
●
●●
●
●●● ● ●
● ●●
●●
● ●
●
● ●●●
●
● ●
● ●●
●
●
●●
●● ●●
●
● ● ● ●●
●
●● ●
●●●
●
●
● ●
●
1
●
●
●
●
●●●●
● ● ●●
●
● ● ●●●
●
● ●
●● ●
●●
●
●● ●
●●
●
● ●
●
●
●
●● ● ●● ● ●●
●
●●●● ●
●
●
● ●●
●
●
●●●●
●
● ●
●
●●●●●
●●
● ● ●●
● ●● ●● ●
●
● ●●● ● ●
●
● ● ●
●
●
●
●
●
0
●
●
●
●
●●● ● ●● ●
●
●● ●● ●
● ●●
● ●●
●● ●
●
●●
●
●●●
●●●●
●
●●
●● ●● ●●
● ● ●●
●
●
●●
●
●● ●●●
●
● ●●
● ●●●●●●
● ●
●
●
● ●●● ●
●
●
● ● ●●
●
●
●
●
●
●
●
●
−2 −1
●
●
●
●
●
●
● ● ●●●
●
●
●●● ●●
●
●
●
●● ●●
●
●
● ●●
●●●
●● ●
●
● ●
●
● ●●● ● ●
●
● ● ●●
●
● ●● ●
●
●●●
●
●
●●● ●
●●●
●●
●●
● ●●●
●● ●
●● ● ●
●
●●● ●
−1 0
● ●●
●
● ●
●
●
●●● ●●
●
●
●
●
●
● ●●
●
●●●●
●
●● ●●
●
●●
●●
● ●
●●
●●
●●● ● ●
● ●● ●
● ●●
●●
● ●
●
● ●●
●● ●● ●
●
●
●
●
●
●
●● ● ●
●●
●
●● ●
●●
●●
● ●
● ●
●●
●
●
3
4
●
●
●
●
●●
●
●
●●
●
● ●●
●
●
●
●
●
●● ●
●●
●
●
● ● ●
●● ●●
● ●●
●● ●●● ●●
●
●●
●
●●
●●●
●●●●
●● ●
● ● ●
●
●●
●
●●
●
● ●●
●
● ●
●●
● ●●
●●
● ●
●●
●
● ● ●
●●
●
● ●
●
●
2
●
1
●
−1 0
●
●
●
● ● ● ● ●●
●
●
●● ●●
● ●
●
●
●
● ● ●●
●
●●● ●●●● ●
●
●
●●
●●●
●●●
● ●
● ●●●●●●
●● ●
●
●●
●●
●●
●
●●●
●
●● ●
●●
●
●●
●● ●
●
●● ●
●
●● ●
●
●
1 2 3
●
●
●
●
●
−3
●●
●●
● ●● ●
●
●
● ● ● ●●●
● ●●● ●
●
●
●
● ● ●●
●●
●●●●● ● ●●
●
●
●
●
● ● ●
●●
●● ●●●
●
●●● ●
●●●●●
● ●●
● ●●●
●
●
●●
●
●
●●●
●●
●
● ●●●
●
●
●
●
● ●
−1
1 2 3
IC.4
−3
● ●
IC.1
●
●
●
●● ●
●
● ●●●●●● ●●
●
●●●●●
●
●
●● ● ●
●
●●●
●●
●
●
●● ●●●
●
●●● ● ●
●●● ●
●●
● ●●
●●●
●
● ●
●●
●
●
● ● ●
●
●●
●● ●
●●
●●
● ●●
●
● ●●
● ●●
● ●
●
−1
●
ICS
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Key references I
Tyler, D. E., Critchley, F., Dümbgen, L. and Oja, H. (2008).
Exploring multivariate data via multiple scatter matrices. Journal
of the Royal Statistical Society, Series B, accepted.
Oja, H., Sirkiä, S. and Eriksson, J. (2006). Scatter matrices and
independent component analysis. Austrian Journal of Statistics,
19, 175–189.
Nordhausen, K., Oja, H. and Tyler, D. E. (2008). Two scatter
matrices for multivariate data analysis: The package ICS. Journal
of Statistical Software, accepted.
Nordhausen, K., Oja, H. and Ollila, E. (2008). Multivariate models
and their first four moments. Submitted.
R
Preliminaries
Multivariate Models
Whitening
PCA
ICA
ICS
Key references II
Nordhausen, K., Oja, H. and Ollila, E. (2008): Robust
independent component analysis based on two scatter matrices.
Austrian Journal of Statistics, 37, 91–100.
Nordhausen, K., Oja, H. and Tyler, D. E. (2006): On the efficiency
of invariant multivariate sign and rank tests. In Liski, E.P., Isotalo,
J., Niemelä, J., Puntanen, S., and Styan, G.P.H. (editors)
“Festschrift for Tarmo Pukkila on his 60th birthday”, 217–231,
University of Tampere, Tampere.
Kankainen, A., Taskinen, S. and Oja, H. (2007). Tests of
multinormality based on location vectors and scatter matrices.
Statistical Methods & Applications, 16, 357–379.
R