Multivariate Models, PCA, ICA and ICS
Transcrição
Multivariate Models, PCA, ICA and ICS
Preliminaries Multivariate Models Whitening PCA ICA ICS Multivariate Models, PCA, ICA and ICS Klaus Nordhausen1 Hannu Oja1 1 2 Esa Ollila2 Tampere School of Public Health University of Tampere Department of Mathematical Science University of Oulu 3 Department of Statistics Rutgers - The State University of New Jersey David E. Tyler3 R Preliminaries Multivariate Models Whitening Outline Preliminaries Multivariate Models Whitening PCA ICA ICS R PCA ICA ICS R Preliminaries Multivariate Models Whitening PCA ICA ICS Important Matrices and Decompositions • A p × p permutation matrix P is obtained by permuting the rows and/or columns of the identity matrix Ip . • The sign-change matrix J is a diagonal matrix with diagonal elements ±1. • The p × p matrix O is an orthogonal matrix if OT O = OOT = Ip . • P and J are orthogonal matrices. • D is a diagonal matrix. • Let A be p × p matrix. The singular value decomposition (SVD) of A is A = O∗ DOT . • Let A be p × p matrix. The eigenvalue decomposition of A is A = ODOT . R Preliminaries Multivariate Models Whitening PCA ICA ICS Affine transformation Let x be a p-vector. An affine transformation has then the form y = Ax + b, where A is a full rank p × p matrix with SVD A = O∗ DOT and b is a p − vector . This transformation can be interpreted as a change in coordinate system. The original system of x is first rotated / reflected using OT then componentwise rescaled using D, again rotated / reflected using O∗ and finally the origin of the system is shifted by b. In statistics methods and estimates are preferred that do not depend on the underlying coordinate system. • A method like for example a hypothesis test is called affine invariant if it does not depend on the coordinate system, i.e. the p-value does not change when the coordinate system is changed. • Estimates are called affine equivariant if they follow a change in the coordinate system in an appropriate way. R Preliminaries Multivariate Models Whitening PCA ICA ICS Location and Scatter Functionals Let x be a p-variate random vector with cdf F . A p-vector valued functional T = T(F ) = T(x) is called a location functional if it is affine equivariant in the sense that T(Ax + b) = AT(x) + b for all full-rank p × p-matrices A and for all p-vectors b. A p × p matrix S = S(F ) = S(x) is a scatter functional if it is affine equivariant in the sense that S(Ax + b) = AS(x)AT for all full-rank p × p-matrices A and for all p-vectors b. The corresponding sample versions are called location statistic and scatter statistic. R Preliminaries Multivariate Models Whitening PCA ICA ICS Special Scatter Functionals A symmetrized scatter functional version of a scatter functional S can be obtained by using two independent copies x1 and x2 of x Ssym (x) := S(x1 − x2 ). A shape matrix is only affine equivariant in the sense that S(Ax + b) ∝ AS(x)AT . R Preliminaries Multivariate Models Whitening PCA ICA ICS Independence property A scatter functional S has the independence property if S(x) is a diagonal matrix for all x with independent margins. Note that in general scatter functionals do not have the independence property. Only the covariance matrix, the matrix of fourth moments and symmetrized scatter matrices have this property. If, however, x has independent and at least p − 1 symmetric components all scatter matrices will be diagonal matrices. R Preliminaries Multivariate Models Whitening PCA ICA ICS M-estimates of Location and Scatter There are a lot of techniques to construct location and scatter estimates. In this talk we will mainly use M-estimates. M-estimators for Location (T) and Scatter (S) satisfy the following two implicit equations: −1 T = T(x) = (E(w1 (r ))) E (w1 (r )x) and S = S(x) = E w2 (r )(x − T)(x − T)T for some suitably chosen weight functions w1 (r ) and w2 (r ). The scalar r is the Mahalanobis distance between x and T, that is, r = ||x − T||S . R Preliminaries Multivariate Models Whitening PCA ICA ICS Examples of Scatter Matrices The most common location and scatter statistics are the vector of means E(x) and the regular covariance matrix COV(x). Using those two and compute a so called 1 step M-estimator one yields as a scatter the matrix of fourth moments. COV4 (x) = 1 E(r 2 (x − E(x))(x − E(x))T ) p+2 Another famous M-estimator is Tyler’s shape matrix . (x − T(x))(x − T(x))T STyl (x) = pE ||x − T(x)||2S The symmetrized version of Tyler’s shape matrix is known as Dümbgens’s shape matrix. R Preliminaries Multivariate Models Whitening PCA ICA ICS Location - Scatter Model All models discussed in this talk can be derived from the location scatter model x = Ω + µ, where x = (x1 , . . . , xp )T is a p-variate random vector, = (1 , . . . , p )T is a p-variate random vector standardized in a way explained later, Ω a full rank p × p mixing matrix and µ a p-variate location vector. The quantity Σ = ΩΩT is the scatter matrix parameter. For further analysis the assumptions imposed on are crucial. Note however, that the standardized vector is actually not observed, only x is directly measureable. The vector is rather a mental construction than something with a physical meaning. Yet, in some cases, can have an interpretation and it might even be the goal of the analysis to recover it when only x is observed. Either way, one of the first challenges one faces in practical data analysis is to evaluate which assumptions on can be justified for the data at hand. R Preliminaries Multivariate Models Whitening PCA ICA ICS Symmetric Models A1: Multivariate normal model. has a standard multivariate normal distribution N(0, Ip ). A2: Elliptic model. has a spherical distribution around the origin, i.e. O ∼ for all orthogonal p × p matrices O. A3: Exchangeable sign-symmetric model. In this model is symmetric around the origin in the sense that PJ ∼ for all permutation matrices P and sign change matrices J. A4: Sign-symmetric model. is symmetric around the origin in the sense that J ∼ for all sign change matrices J. A5: Central symmetric model. is symmetric around the origin in the sense that − ∼ . R Preliminaries Multivariate Models Whitening PCA ICA ICS Asymmetric Models B1: Finite mixtures of elliptical distributions with proportional scatter matrices. For a fixed k is Pk = i=1 pi (i + µi ), where p = (p1 , . . . , pk ) is Pk Multin(1, π) distributed with 0 ≤ πi ≤ 1 and i=1 πi = 1 and i ’s are all independent and follow A2. B2: Skew-elliptical model. = sgn(∗p+1 − α − β∗p )∗ , where (∗ T , ∗p+1 )T satisfies A2, but with dimension p + 1, and α, β ∈ R are constants. B3: Independent component model. The components of are independent with E(i ) = 0 and Var (i ) = 1. R Preliminaries Multivariate Models Whitening PCA ICA Relationships Between the Models The symmetric models A1-A5 are ranked from the strongest symmetry assumption to the weakest one, which means A1 ⊂ A2 ⊂ A3 ⊂ A4 ⊂ A5. The symmetric models can be seen as border cases of the asymmetric models and we have the following relationships: • A1 = A2 ∩ B3 • A2 ⊂ B1 and A2 ⊂ B2 ICS R Preliminaries Multivariate Models Whitening PCA ICA ICS Uniqueness of Model parameters • The location parameter µ is well-defined in models A1-A5. In that case µ is the center of symmetry. • The scatter parameter Σ is only well defined in models A1 and B3. It is the covariance matrix. • The mixing matrix Ω is not well defined in any of the models. Further assumptions are needed in most models in order to have well-defined parameters Ω and Σ. Relation between location and scatter functionals in the different models: • Location statistics estimate in A1-A5 the same population quantity, the center of symmetry. • Scatter statistics measure in A1-A3 the same population quantity. They measure the shape of data, i.e. S(x) ∝ Σ. R Preliminaries Multivariate Models Whitening PCA ICA ICS Data Transformations In practical data analysis data is often transformed to different coordinates. The transformation can be model based or not and may be motivated by different reasons. Transformations we will have a closer look at are: • Whitening • Principal components. • Independent components. • Invariant components. Other transformations are for example: • Canonical correlations. • Factor analysis. Most transformations are based usually on E(x) and COV(x). R Preliminaries Multivariate Models Whitening PCA ICA ICS Whitening The whitening transformation centers the variables and jointly uncorrelates and scales them to have unit variances. The transformation is y = COV−1/2 (x)(x − E(x)), where then E(y) = 0 and COV(y) = Ip . Properties: • In A1 it makes the marginal variables independent. • In A2 y will be spherical. • It does usually not recover . • The resulting coordinate system is not affine invariant. It only holds that 1 1 COV− 2 (Ax + b)(Ax + b − E(Ax + b)) = O COV− 2 (x)(x − E(x)). R Preliminaries Multivariate Models Whitening PCA ICA ICS Principal Components The principal components also uncorrelates marginals. It does however not scale them to have unit variances but chooses the new variables successively in such a way that they are linear combination of the original variables which have maximal variation under the condition of being orthogonal with the previous ones. The transformation is based on the eigenvalue decomposition of the regular covariance COV(x) = ODOT . The principal components are y = OT x, with COV(y) = D. Variations of PCA: • centered PCA. The variables are first centered, i.e. x ← x − E(x). • scaled PCA. Each variable is scaled to have unit variance, i.e. xi ← xi /σi . Where σi2 is the variance of xi . R Preliminaries Multivariate Models Whitening PCA ICA ICS Properties and Purpose of PCA PCA has the properties: • not affine invariant. • not model based, only assumes existence of the first two moments. • lot of nice geometrical properties, especially in A1. Main purposes: • dimension reduction. • in connection with other multivariate techniques. For example in regression to avoid multicollinearity problems or in clustering. • outlier detection. R Preliminaries Multivariate Models Whitening PCA ICA ICS R Comparison Whitening vs. PCA In the following example 70 observations are sampled from a bivariate normal distribution. Original data Whitenend data 5 5 4 4 4 3 ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ●●●● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ●● ● −5 −4 −3 ● −2● ● −1 ● −1 1 2 ● ● ● ● ● ● 2 2 ● ● ● ● ● 4 5 −5 −4 −3 ● ● ● −2 ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ●●● ● ●● ● ● ● ●● ● ● ● ● ● 3 3 ● ● 1 ● 3 ● 2 ● Principal components 5 1 ● −1 1 −1● ● ● ● ● ● ● ●● ● ● ● ● ● ● 2 ● 3 4 5 −5 −4 ● −3● ● ●● ● ● 1 ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●●● ● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● −2 −1 1 −1 ● −2 −2 ● −2 −3 −3 −3 −4 −4 −4 −5 −5 −5 2 ● ● ● ● 3 ● 4 5 Preliminaries Multivariate Models Whitening PCA ICA ICS Robust Whitening and Robust PCA The mean vector and the covariance matrix are not very robust and therefore the transformations based on them can be dominated by only a few influential observations. A common method to robustify classical methods is to replace the mean vector and the covariance matrix by more robust robust location and scatter functionals. This of course leads only to the same method if the functionals estimate the same population quantity. Which basically means this is possible only in models A1-A3. R Preliminaries Multivariate Models Whitening PCA ICA ICS Independent Component Analysis Independent component analysis (ICA) can be seen as a model based transformation. Let assume for simplicity that µ = 0. Then the IC model B3 reduces to x = Ω, where has independent components. The model is however ill defined since also x = (ΩPDJ)(JD−1 PT ) = Ω∗ ∗ where ∗ has independent components. Therefore the components can be rescaled, permuted and their sign changed. To fix the scale a common assumption is E() = 0 and COV() = Ip . R Preliminaries Multivariate Models Whitening PCA ICA ICS Estimation of the Mixing Matrix In ICA the main goal is to estimate the mixing matrix Ω respectively an unmixing matrix Λ. This is possible up to sign and permutation if at most one component are gaussian. In some cases then the up to sign and order recovered underlying independent components z = Λx have physical meaning. Application areas of ICA of are: • signal processing. • medical image analysis. • dimension reduction. R Preliminaries Multivariate Models Whitening PCA ICA ICS Common Structure of Most ICA Algorithms Most ICA algorithms consist of the following two steps. Step 1: Whitening. x is whitened so that E(x) = 0 and COV(x) = Ip . Then x = Oz∗ with an orthogonal matrix O and independent components z∗ which have E(z∗ ) = 0 and COV(z∗ ) = Ip . Step 2: Rotation. Find an orthogonal matrix O whose columns maximize (minimize) a chosen criterion g(oT x). Common criteria are measures of marginal nongaussianity (negentropy, moments) and likelihood functions. Well known algorithms are fastICA, FOBI, JADE, PearsonICA, . . . R Preliminaries Multivariate Models Whitening PCA Picture example for ICA ICA ICS R Preliminaries Multivariate Models Whitening PCA ICA ICS R Multiple functionals for transformation So far the transformations involved mainly one location and / or one scatter functional. Recently is was suggested to use two location functionals and / or two scatter functionals for this purpose. Useful to recall in this context is that skewness can be interpreted as the difference of two different location functionals and kurtosis as the ratio of two different scatter functionals. For example the classical univariate measures: β1 (x) = E3 (x) − E(x) E((x − E(x))3 ) = p var (x)3/2 var (x) β2 (x) = E((x − E(x))4 ) var4 (x) = var (x)2 var (x) with E3 (x) = E with var4 (x) = E x − E(x) var (x) x − E(x) var (x) x (x − E(x))2 Preliminaries Multivariate Models Whitening PCA ICA ICS Two Scatter Matrices for ICS Two different scatter matrices S1 = S1 (x) and S2 = S2 (x) can be used to find an invariant coordinate system (ICS) as follows: Starting with S1 and S2 , define a p × p transformation matrix B = B(x) and a diagonal matrix D = D(x) by T T S−1 1 S2 B = B D that is, B gives the eigenvectors of S−1 1 S2 . The following result can then be shown to hold. The transformation x → z = B(x)x yields an invariant coordinate system in the sense that B(Ax)(Ax) = JB(x)x for some p × p sign change matrix J. R Preliminaries Multivariate Models Whitening PCA ICA ICS Interpretation of the ICS Transformation This transformation can be seen as a double decorrelation. First x is whitened using S1 and on the whitened data a PCA is performed with respect to S2 . Basically this can also be seen as a way to compare two scatter matrices. If the PCA step finds still an interesting direction after the whitening step then the two scatter matrices measure different population quantities. The diagonal elements of D can be seen as ratios of two different scatter functionals. Hence the values can be seen as kurtosis measures and therefore the invariant coordinates are according to their kurtosis measures with respect to S1 and S2 . R Preliminaries Multivariate Models Whitening PCA ICA ICS An extension of ICS The sign of the column vectors of B, respectively of the invariant coordinates z is not fixed. One option is here to use two pairs (T1 , S1 ) and (T2 , S2 ) and fix the sign then such that The the a way to construct the invariant coordinate system is to • center x using T1 , i.e. x ← x − T1 (x) • estimate B and the invariant coordinate z • fix the signs of z so that T2 (z) ≥ 0. In that case T2 (z) ≥ 0 can be seen as the difference of two location measures and therefore as a measure of skewness. If the D has distinct diagonal entries and T2 (z) > 0 the transformation is well defined. Denote this version of ICS from now on as centered ICS. R Preliminaries Multivariate Models Whitening PCA ICA ICS Applications of ICS ICS is useful in the following context: • Descriptive statistics and model selection (centered ICS). • Outlier detection and clustering. • Dimension reduction. • ICA, if S1 and S2 have the independence property and the IC model holds then B is an estimate of the unmixing matrix. • In the transformation-retransformation framework of multivariate nonparametric methods. Some of the applications will be now discussed in more detail. R Preliminaries Multivariate Models Whitening PCA ICA ICS On the choice of S1 and S2 As shown previously there are a lot of possibilities for S1 and S2 to choose from. Since all the scatter matrices have different properties, these can yield different invariant coordinate systems. Unfortunately, there are so far no theoretic results about the optimal choice. This is still an open research question. Some comments are however already possible: • For two given scatter matrices, the order has no effect • Depending on the application in mind the scatter matrices should fulfill some further conditions. • Practise showed so far that for most data sets different choices yield only minor differences. R Preliminaries Multivariate Models Whitening PCA ICA ICS Descriptive Statistics based on Centered ICS Using the two pairs (T1 , S1 ) and (T2 , S2 ) then a data set can be described via • The location: T1 (x) • The scatter: S1 (x) • Measure of skewness: T2 (z) • Measure of Kurtosis: S2 (z) The last two measures can then be used to construct tests for multinormality or ellipticity (see for example Kankainen et al. 2007). R Preliminaries Multivariate Models Whitening PCA ICA ICS Moment Based Descriptive Statistics using ICS An interesting choice for model selection is T1 = E(x), S1 = COV(x), T2 = p1 E(||x − E(x)||2COV x), the vector of third moments, S2 the matrix of fourth moments. Denote D(k ) as the set of all positive definite diagonal matrices with at most k distinct diagonal entries, then the values of s and D in the eight models are: Model A1 A2 A3 A4 A5 B1 B2 B3 Skewness s 0 0 0 0 0 ≥0 ∝ ep or ∝ e1 ≥0 Kurtosis D Ip D(1) D(1) D(p) D(p) D(k ) D(2) D(p) R Preliminaries Multivariate Models Whitening PCA ICA ICS Finding outliers, structure and clusters In general, that the coordinates are ordered according to their kurtosis is a useful feature. One can assume that interesting directions are ones with extreme measures of kurtosis. Outliers for example are usually shown in the first coordinate. If the original data is coming from an elliptical distribution (A2), S1 and S2 measure the same population quantity and therefore the values of D in that case should be all the same. For non-elliptical distributions however they measure different quantities and therefore the coordinates can reveal "hidden" structures. In the case of x coming from an mixture of elliptical distributions (B1), ICS estimates heuristically spoken to Fisher’s linear discriminant space. (Without knowing the class labels!) R Preliminaries Multivariate Models Whitening PCA ICA ICS R Finding the outliers The modified wood gravity data is a classical data set for outlier detection methods. It has 6 measurements on 20 observations containing four outliers. Here S1 is a M-estimator based on t1 and S2 based on t2 . ● ● ● ●●● 0.50 ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● 0.60 ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● 13 ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● 0.55 ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ●● ● ● ● ● ● x5 ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● y ● ●● ● 0.85 11 9 ● 0.95 ● ●● ● ●● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●● ● ● ● ● IC.3 ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ●● ● ● ● ● ●● ●● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● IC.4 ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ●●● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●●● ● ● ● ● −28 −24 ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ●● IC.5 14 16 ●● ● ● ● 18 ● ● ●● ● ●●●● ● ●● ● ● ● ● ●● −30 −24 ● ●● ●● ● ●● ● ● ● ● ● ●● ● ●●● ● ●●●● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● −28 ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●●●● ● ● 0.0 ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● IC.2 ● ● ● ● ● ●● ● ● ● ● ● ● ● ● x4 ● ● ● ● ● ● ●● ● ●● ● ●● ● ● −1.5 ● ● ● ● ● ● ●●●● ● ●● ● ●● ● ● −28 ● ● ● ● ● ●● ●● ● ●● ● ● ● ●● ● ● ● ● ●● 0.40 ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● 14 16 18 ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●●●● ●● ● ● ● ● ● ● ●● ● ● 0.45 ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ●● ● ● 3 ● ● ● ● ● 1 ● ● ● ● ●● ● ● ● ● ● IC.6 ● ● −28 ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● 3 ● ●● ● ●● ●● ● ● ● ● ● x3 ● ●● ●● ● ● ● ●● 1 ● ● ● ● ● ● ● ● ● −1 ● ● ● ● IC.1 −1 ● ● ● ●● ● ● ● 13 0.0 0.45 ●●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● 11 ● ●● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● 9 ● −1.5 0.60 ● ● ● ● ● 0.55 ● ● ● 0.50 ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ●● x2 ● ● ● ● ● ● ● ●● ● ●● ●● ●● ● ● ● ● 0.40 0.11 ●● ● ●● ● ● ● ●● 0.40 ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● 0.95 0.15 ● ● ● ● 0.40 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● 0.85 ●● ● ●● ● ● ● ●● ● ● ● x1 Invariant coordinates 0.45 0.55 ● ●● ● ● 0.45 0.60 0.15 ● −30 Original data 0.11 Preliminaries Multivariate Models Whitening PCA ICA ICS R Finding the structure I Original data ● ● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● V2 ● ● ● ●● ●●● ● ●●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ●●●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ●● ● ● ● ●● ● ●● ● ● ● ● ●●● ●● ● ●●●● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●●● ●● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ●● ●●● ● ● ●●● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ●● ●● ● ●● ● ●● ● ● ● ● ●●●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ●● ●● ● ● ● ● ● ● ●● ● ● ●● ●●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ●● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ● ●● ●● ●● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● V3 ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●●●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ●●●● ● ● ● ● ● ●● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ●● ●● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ●● ● ● ●● ● ●●● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●●●●● ● ● ●●● ● ● ● ●● ●●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ●●● ● ●●● ●●● ●● ● ● ●●●● ● ●● ● ● ● ●● ● ●●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ●●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ●● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ● ●● ● ● ● ●●●● ● V4 ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●●● ● ●● ●●● ● ● ● ●●●● ● ●● ●● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ●● ● ●●● ● ● ● ● ●● ●● ● ●● ●●● ● ● ● ● ● ●● ● ● ● ●● ●●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ● ●● ● ● ●●●●● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ●● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ●● ● ●● ●● ●● ●●● ● ● ● ● ●● ● ● ●●● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ●● ● ● ● ●● ● ●● ● ● ● ●● ●● ● ● ● ●● ●● ● ● ●● ● ● ● ●● ●● ● ● ● ●● ● ●● ● ●● ● ● ●●● ● ● ●● ●● ● ●● ● ● ●● ● ●●● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● V5 ● ●● ● ● ● ● ● ●● ●●● ●● ●● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●●●●● ●● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ●●● ● ● ●●● ●● ● ● ●● ● ● ●●● ● ● ● ● ●● ● ● ●● ●●●● ●● ● ●●● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●●● ● ●● ● ●●● ● ●● ● ●●● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●●● ●●● ● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●●● ●● ●● ● ●● ●●● ●● ● ● ● ● ● ● ● ●● ● ●●●● ●● ●● ●●● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ●● ● ● ● ● ● ●● ● ●● ● ●● ●● ● ●● ● ●● ● ●● ● ● ● ● ●●● ● ●● ● ● ● ●● ● ●● ●● ●●● ● ●● ● ●● ●● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ●● ●● ●●● ●● ●●● ● ● ● ● ●●● ●● ● ●● ●● ●● ●● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ●● ● ● ● ●●● ● ● ● ● ● ●● ●●●● ● ●●●● ● ● ● ● ● ● ●● ● ●● ●●● ● ● V6 ● ● −5 0 5 −3 0 2 −6 −2 2 5 0 5 ●●● ● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ●● ●● ● −5 0 −10 ● ● ● ● ●● ● ● ● ● ●● ● ●●● ● ●●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ●● ● ●● ●●● ● ● ● ● ● ● ● ● ●●● ● ●● ● 2 4 0 0 ●●● ●● ●● ● ●●●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ●●●● ● ● ●● ● ● ● ● ●● ●● ● ●● ● −3 −4 ●●● ●● ●● ●● ● ●● ● ●● ● ● ● ●● ●●● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ●● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ●●● ● ●●● ●● ● ●● ● ● ● ●● ● ● ● ●● ● 2 10 0 −10 4 0 −4 10 0 −10 10 ●●● ● ● ●● ●● ●● ●●● ● ●● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ●●● ●● ● ● ● ● ● ●●●● ● ●● ●● ● ● V1 The following data set is simulated and has 400 observations for 6 variables. It looks like an elliptical data set. 0 −6 −2 −10 Multivariate Models Whitening PCA ICA ICS Finding the structure II PCA 0.5 ● ●● ●● ● ●● ● ● ● ●●● ● ● ●● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ●● ● ● ● ● ●● ● ● ● ●● ●●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●●●● ●●● ● ● ● ● ● ● ● ● ●●●● ● ● ●● ● ● ● ● ● ● ●● ●● ●●● ●●● ● ● ● ● ● ● ● ● ● ● ●●● ● ●●●● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ●● ● ●● ●● ●● ● ●●● ● ● ●● ●● ●● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ●● ●●● ● ● ● ● ● ● ●● ● ● ● ● ●●●● ●●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ●● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ●● ●● ● ●● ● ●●● ●● ● ● ●● ● ● ●● ●● ● ● ● ●●● ● ● ● ● ● ●● ●● ● ●●● ●● ●● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ●● ●● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ●●● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●●● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ●● ●● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● Comp.3 ● ● ● ● ● ● ●●● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ●●●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●● ●●● ●●● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ●●● ● ●● ● ●● ● ● ● ●● ● ●●● ● ● ● ●● ● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ● ●●● ● ● ● ● ●● ●● ●● ● ●● ● ●● ● ●●● ● ● ●● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ●● ●●● ●● ● ●● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ●● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ●● ● ● ● ●●●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● Comp.4 ● ●● ● ● ● ● ● ●● ● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ● ●● ● ● ●● ● ●● ● ● ●● ● ●● ● ●● ● ● ● ● ●● ● ●●● ● ●● ● ●● ● ●● ● ● ● ● ●● ● ●● ●● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ●●● ●●● ●● ●● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ●●● ●● ●● ●● ● ●● ● ●●● ● ● ● ●● ●● ●● ●● ●● ● ● ● ●●● ●● ●● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●●●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●●● ● ●● ● ●●●● ● ● ● ● ●● ●●● ●● ●●● ● ● ● ● ●●● ● ● ● ● ● ●● ●●●● ● ● ● ●● ● ● ● ●● ● ● ●●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ●● ● ●● ●● ● ● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●●●●● ● ● ● ● ●● ●● ● ●● ● ● ●● ●● ●● ●● ● ●● ● ● Comp.5 ●● ● ●● ● ●●●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ●● ●● ● ● ● ●● ●●●● ● ● ● ●● ●●● ●● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ●● ● ●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ●●●●●● ●●● ●● ● ● ●●●●● ●● ●● ●● ● ● ● ●● ● ●● ●● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ●●●● ●●● ●● ● ● ● ● ●● ●● ● ● ● ● ●●● ● ●●● ● ● ● ●●●●●●● ● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●●● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●●●● ● ● ● ●●● ● ●● ● ●●● ●● ● ●● ● ●● ● ● ● ● ● ●●● ●● ● ● ●● ●● ● ● ● ●●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●●●●●●● ●● ● ● ● ●● ●● ● ●● ● ●● ●● ●● ● ●● ● ● ● ● ● ● ● Comp.6 −10 0 10 −6 −2 2 6 −2 0 2 5 Comp.2 −10 ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ●●●●● ● ●● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●●●● ●● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ●● 4 Comp.1 ● ●● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ●●● ● ● ●● ●● ● ●● ● ● ● ●●● ● ● ● ● ●●●● ● ●●●● ●● ● ●● ●● 15 0.5 ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 −0.5 ● ● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ●●●● ● ●●● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ●●● ●● ●● ● ● ● ●●● ● ●● ● ● ●●● ●●● ● ●●● ● ● ● ●● −6 2 4 ● ● ●● ●● ● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ●●● ●● ●●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● 2 −2 ● ● ●● ●● ● ● ●● ● ●●● ● ●● ● ●● ● ● ●● ● ●● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●●● ● ●● ● ●● ●●● ● ●● ●● ●● ● ●● ●● ● ● ● ● ● ● ●● ● 0 5 −2 −2 2 4 −5 5 −5 0 −0.5 Preliminaries R Multivariate Models Whitening PCA ICA ICS Finding the structure III ICS ● ● ● ● ● ●●● ● ●● ●●●●●●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ●● ● ● ● ● ●●● ● ● ● ●● ● ● ● ●●● ● ●● ● ● ●● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● −3 0 2 4 ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ●● ●● ● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ●● ●● ●● ●● ● ●●●● ● ● ●● ● ● ● ● ●● ●● ● ●●●●● ● ●●● ●● ● ● ●● ● ● ● ●● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●●●●● ● ● ● ● ●●● ●● ● ●●● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ●● ● ● ●●● ●● ●●● ●● ● ● ●● ●● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●●●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●●●● ● ●●● ●● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ●● ●●●● ●●● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ● ●● ●● ● ● ● ●● ●●●●● ● ●● ● ● ●● ● ● ● ●●● ● ●● ● ● ●● ● ●●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ●● ●● ● ●● ●●● ● ●● ● ●● ●●●● ● ● ● ● ●●● ● ● ●●● ● ● ●●●● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ●● ● ● ● ●● ● ●●● ● ● ● ●● ● ● ●● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● IC.4 ● ● ● ● ●● ●● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ●● ●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●●● ●● ● ● ●● ● ●● ● ● ● ● ●● ●● ●● ● ●● ●● ● ●●●● ● ●● ● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ●●● ● ●● ●● ● ●●● ●● ● ● ●●● ● ●● ● ●● ● ● ● ● ● ●● ●●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ●● ● ● ● ● ● ● ● ●●●● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ●● ●● ● ●● ● ●●● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●●●● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●●● ●●●● ● ●●● ●● ●●● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ●●● ● ● ● ●● ●● ● ● ● ● IC.5 ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●●●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ●● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ●● ● ● ● ● ●● ●● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ●● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ●●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●●● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●●●●●● ● ● ● ● ● ● ●● ● ● ●●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●●●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● IC.6 −3 0 2 −2 0 2 2 IC.3 ● ● ●●● ● ● ● ●● ● ●● ●● ● ● ●● ● ●● ● ●● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ●●● ● ●● ● ●● ●● ●●● ● ● ● ● ● ●●●● ● ● ● ● ● ● 0 ● ● ●●● ●● ● ● ● ●● ● ● ●● ●● ● ●●● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ●● ● ● 2 ●● ●● ●● ● ●● ● ● ● ●●●● ●●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●●● ● ●● ● ● ●● ● ● ● ●● 4 0 1 2 ● ● ● ● ●● ●● ●● ● ● ●● ● ● ●● ● ● ●●● ● ● ● ●● ● ● ● ●● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ●● ●● ● ● ● ●●●● ●● ● ●● ●● ● 0 −2 ● ● ● ●● ● ● ●●● ●● ● ●● ●● ●● ● ● ●●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ●● ●● ● ● ●●● ● ● ● ●● ● −3 2 −3 ● 0 ● ● ● ●● ● ● ● ● ● ● ●● ●●● ●● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ●● ● ● ● ● ●●● ●●● ● ● ●● ●● ● ●● ● ●● 2 2 0 −2 IC.2 −2 ● ●●● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ●● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●●● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● 2 ● ● ● ● ● ● ● ● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ●●● ●● ● ●● ● ● ● ●●● ● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● 0 2 0 ● ● ● ●● ●●● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● −4 −1 2 IC.1 −1 −2 −4 −2 Preliminaries R Preliminaries Multivariate Models Whitening PCA ICA ICS R Clustering and dimension reduction 10 −1.5 ● 1.0 0.0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ●●● ●● ●● ● ● ●● ● ●● ● ● ●●● ●● ● ● ●● ● ●● ● ●●●● ●● ● ● ● ● ●● ●● ●● ● ● ● ●● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ●● ● ●●● ● ●● ● ●● ● ● ●● ● ● ● ● ●● ● ●● ●● ● ● ● ● IC.2 ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ●●● ● ● ● ● ●●● ● ●● ● ● ●● ●● ● ● ●● ●●● ● ● ● ● ● ● ● ●●● ● ●● ● ●●●● ● ● ●●● ● ●●● ● ●● ● ● ●●● ●● ●●● ● ● ● ● ● ●●● ●● ●●● ●●● ●●●● ● ● ● ● ●●● ● ●● ● ● ●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●●● ● ● ●● ● ● ● ●●●● ●●● ● ●● ● ● ●●● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ●●●● ● ● ●●● ● ● ● ● ● ● ● ● ● ●●●● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●● ●● ● ●● ● ●● ●● ● ● ●● ●● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ●●●● ● ●●● ●●●● ●● ●● ● ● ●● ● ● ● ● ●●● ●● ●● ● ●● ● ●● ● ●● ● ● ● ●●●●● ●● ●●● ● ● ●● ●●●●● ● ●●● ●●● ● ● ●● ● ● ● ● ●● ● ●● ●●●● ●● ● ● ● ● ●●● ●●● ● ●●●●● ● ● ● ●● ● ● ● ● ● ● 5 6 7 IC.3 ● 9 1.0 ● ●● ●● ● ● ● ● ● ●● ● ● ●● ●● ● ● ●● ●●● ● ●● ● ●● ● ●●● ● ● ● ●● ● ● ●●●● ●● ● ●● ● ● ● ● ● ● ● ●● ●●● ● ● ● ●● ● ●●● ● ●●● ●●● ● ●● ● ●● ●● ●● ●● ●● ● ● ● ●● ● ●● ●●●● ●● ●●● ●●● ●●● ● ● ● ●●● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ●●● ● ● ● ●● ●●● ● ●● ●● ● ● ●● ●● ● ●● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ● ● 4 5 6 7 9 7 8 ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ●● ●● ●● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ●●● ●● ● ● ● ● ●● ●● ●● ● ●●●●● ●● ●● ● ●● ●● ● ● ● ●● ● ● ●● ● 3 2.0 ●●● ●● ● ● ● ● ● ●● ● ●● ● ● ●● ● ●●● ●● ● ● ● ●● ●● ●●●● ● ● ●● ●●● ● ● ●● ● ●● ● ●●●● ● ●● ●● ● ● ●●●● ● ● ●● ● ● ●● ● ●● ●● ● ● ● ● ●●● ●●●● ● ● ● ●● ● ● ● ● ●●●● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● 8 ● ●● ● ●● ●● ●●●●● ●●● ● ●● ● ● ●●● ● ● ● ●● ● ●● ●● ●● ● ● ● ● ● ●●● ● ●● ● ● ● ●●● ●● ●●● ● ● ●● ● ● ● ● ●●● ●● ● ● ● ●● ● ● ●●● ●● ● ● ● ●● ● ● ● ● ● ● ● 4 ● ● ● ● ●● ● ● ●●● ● ●● ● ●● ● ● ●● ● ● ● ●● ● ●● ● ● ●● ●● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ●● ●● ●● ● ●●● ● ●● ●● ●● ●● ● ● ● ●● ● ● ●●●● ●● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ●●●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ●● ●● ●● ●● ●●●● ● ● ●● ●●● ● ● ● ● ● ●● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● 2.0 5 6 7 8 9 10 ● −1.5 To illustrate clustering via an ICS we use the Iris data set. The different species are colored differently and we can see, that the last component is enough for doing the clustering. 0.0 ●● ●●● ● ● ● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ● ● ●● ●●●● ●● ● ●● ● ●● ● ● ●● ● ● 6 IC.1 ● ● ●● ● ● ● ● ● ●● ● ● ● ●●●●● ●● ●● ● ●●● ● ●●● ● ● ●● ● ● ●●● ● ● ● ●● ●● ● ● ● ●●● ● ● ●● ● ● ● ●● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●●●● ●●●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●●●●● ● ● ● ● ● ● 5 ● ● ● 4 ● IC.4 ● ● ● ●● ●●● ● ●● ● ● ● ●● ● ● ● ●● ●●● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● 7 9 6 8 ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ●●●● ●●● ●● ●● ●● ●● ● ● ● ● ● ●●● ●● ● ● ● ● ●● ● ●● ●●●● ● ● ●● ●● ●● ● ●● ● ● ● ● ●●●● ● ● ●●● ● ●● ● ● ●● ●●● ●● ●●● ● ●● ● ●● ●● ● ● ●● ●●● ● ●● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● 5 7 4 6 ● 3 5 Preliminaries Multivariate Models Whitening PCA ICA ICS ICS and ICA When the IC model holds then the ICS transformation recovers the independent components upto scale, sign and permutation if • S1 and S2 have the independence property. • the diagonal elements of D are all distinct. In the case of S1 = COV and S2 = COV4 this corresponds to the FOBI algorithm and the condition of D having distinct elements means the independent components must have all different kurtosis values. R Preliminaries Multivariate Models Whitening PCA ICA ICS Implementation in R The two scatter transformation for ICS / ICA is implemented in R in the package ICS. The main function ics can take the name of two functions for S1 and S2 or two in advance computed scatter matrices. The R package offers then several functions to work with an ics object and offers also several scatter matrices and two tests of multinormality. Basically everything shown in this talk can be easily done in R using the methods of base R and the package ICS. R Preliminaries Multivariate Models Whitening PCA ICA ICS R Australian Athletes Data, a More Detailed Example Using R We have now a closer look at the Australian Athletes data set using PCA and ICS on the four variables: • Body Mass Index • Body Fat • Sum of Skin Folds • Lean Body Mass > > > > + > library(DAAG) library(ICS) data(ais) data.ana <- data.frame(BMI=ais$bmi, Bfat=ais$pcBfat, ssf=ais$ssf, LBM=ais$lbm) pairs(data.ana) Preliminaries Multivariate Models Whitening PCA ICA ICS Australian Athletes Data, a More Detailed Example Using R > pairs(data.ana) 35 25 15 100 80 ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ●● ●● ● ● ●●●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●●●● ● ●● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ●●● ● ●● ● ●● ● ● ● ●● ● ● ● ● ● ● ●●●● ● ●● ●● ●●●● ●● ● ● ● ● 20 25 ● 30 35 ● ● ● ●●● ● ●● ● ● ●● ●● ●● ● ● ●● ● ●● ● ●● ●● ● ● ●● ● ●● ● ●●●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ● ●● ● ●● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ●●● ● ● ●● ● ●● ● ●● ● ● ● ●● ● ● ● ●●● ●● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ●● ●●●● ● ● ● ●● ● ●● ● ●●● ● ●●● ●● ●● ● ssf ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ●●● ● ● ● ● ●● ● ● ●● ● ●●● ● ●●● ● ● ● ●● ● ●● ● ● ●● ● ●● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ●●●●●●● ●● ● ●● ●●● ● ● ●● ● ● ●● ● ●● ● ●● ●●● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ●● ●● ●●●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●● ● ●● ● ● ●● ● ● ● ● ●●● ● ●● ●●● ● ●● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ●●● ●● ● ●● ● ● ● ● ● ●● ● ●● ●●●●● ● ●● ● ● ●● ● ● ● ● ● ● ●●● ●● ●● ● ●● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ●●●● ● ●● ● ●●●●●● ● ●●● ● ● ● ● ● ● ● ●●●● ● ● ● ●● ● ●●● ●● ● ● ● 50 100 150 200 35 30 ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ●● ● ● ● ●●●● ● ● ●● ● ●● ●● ●●●● ● ● ● ● ● ●● ● ● ● ●●● ●●● ● ● ● ● ●● ● ●● ● ● ●●●●● ● ● ● ● ●●● ● ●● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ●● ●●● ● ● ●● ● ● ●● ● ●●● ●● ● ● ● ● ● ●●● ● ●● ●● ● ● ●● ●● ● ●● ●● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ●● ● ● ● ●●●● ●●●● ●●●● ● ●●● ● ● ● ● ●●●●● ● ● ●● ● ● ● ● ● ●● ●● ●● ●● ●● ● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●●●● ●● ● ● ●● ● ● ●● ●● ●●● ●●● ● ●● ●● ● ● ● ●● ● ● ●● 25 ● ● ● ● ● ●● ● ● ●●● ● ●● ●●● ●● ● ● ● ●● ●●● ● ● ● ●● ● ●● ●● ● ● ● ●● ● ●● ●●● ●● ● ● ●● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●●● ● ●● ● ●●● ● ●●● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●●● ● ● 100 ● ● Bfat 80 ● ● ●●● ● ●● ● ● ●● ● ●●● ●● ● ●●● ●● ● ●● ● ● ● ● ● ●● ●● ●●● ●● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● 60 ● 60 ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●●●● ●●●●● ● ● ● ● ● ● ●● ● ● ●● ●● ● ●● ● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ●● ●●●●● ●● ● ●●● ● ● ● ● ● ●●●● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ●●●●● ● ● ● ●●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ●● ● ●● ●● ●●● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ●●●● ●● ● ●● ●● ● ● ●● ● ● ● ● ● ●● ●●● ● ●● ● ●● ●● ● ● ● ●● ●● ● ● ●● ● ●●● ● ●● ●●● ●● ●●●● ●● ●● ● ● ●● ●● ● ● ●●● ● ●● ●● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● 40 40 ● ● 20 35 ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ●●● ●●●●● ● ●●● ● ●● ● ● ● ●●●● ● ● ● ●● ●● ● ● ● ● ● ●● ●●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●●●●●● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ●●● ● ● ●● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ●●●● ●● ●● ● ● ● ● ● ● ● ●●● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● BMI 5 25 ● LBM 100 150 200 15 ● ● 50 5 R Preliminaries Multivariate Models Whitening PCA ICA Australian Athletes Data II Performing the PCA > pca.AS<-princomp(data.ana,score=TRUE) > pairs(pca.AS$scores) ● ● ● ●● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ●● ● ● ●● ● ● ●● ● ●●● ●● ● ●● ● ●● ●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ●●● ●●● ● ● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●●● ●●● ● ● ● ●● ●● ●● ● ● ● ● ●● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● 4 ● ● 2 0 −2 ● ● ● ●● ● ● ● ● ●● ●●● ●● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ●●●● ● ●● ●● ● ● ● ● ● ●●●●● ●●● ● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ●● ●● ●●●●● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ●●●●● ● ●● ● ● ● ● ●● ● ● ● ●● ● ●● ● ●● ●● ● ●● ●● ● ● ●● −100 0 ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●●●●●● ●● ● ● ● ●● ●●● ● ●● ●●● ●●● ●● ● ●● ● ● ●● ●● ● ●●● ● ● ●●●● ● ●●● ● ● ● ●●● ●● ● ●● ●● ● ● ●● ●● ●● ●● ● ●●● ● ● ●● ●● ● ● ●● ● ●●●●● ● ●● ● ● ● ●● ● ● ●●● ● ●● ●● ● ● ●● ● ● ● ● ●●● ●● ● ● ● ● ● ●●●● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●●● ● ● ●●●● ● ●● 0 ● ● ●● ● ● ●● ●● ●● ● ●●● ● ● ●● ● ●● ● ●● ●● ● ● ● ● ●● ●●● ● ● ●●●●● ● ● ●●● ● ● ●● ●● ● ● ●● ●● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ●● ● ●●● ●● ●● ● ●●● ●●● ● ●● ● ● ●●● ● ●●●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ●●●●●●● ● ● ● ● ● ●● ●●● ●● ●● ●● ●● ● ●● ● ● ● ●● ● ● ●● ●● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●●●● ● ●●● ●● ● ● ●●●●●●●●● ●● ● ● ● ● ● ● ● ●●●● ●● ● ●●●● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ●●● ●● ●●● ●●●●● ● ●● ●●●● ● ● ● ●● ● ● ●●●● ● ● ●● ● ●● ● ●● ● ● ● ● ●●●● ● ●● ●● ● ●● ● ●● ● ● ● ●● ●● ● ●●● ●●●● ●● ●● ●● ● ● ● ● ●● ● ●●● ● ●● ● ● ● ●● ●● ●●●●●●● ● ● ● ● ● Comp.3 ● ● ● ●● ● ●● ● ● ●●● ●● ●● ●● ●●● ●● ● ● ●●● ●● ●●● ● ● ● ● ●●●●●● ●●● ●● ● ●● ● ● ● ●● ●● ● ●● ● ● ●● ● ●● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ●● ●● ● ●● ●● ●● ● ● ● ● ● ● ●● ●●●● ● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ●● ● ● ●● ●● ●● ● ● ●●● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ● ●● ● ● ● ●● ●● ●● ● ● ●● ●● ●● ●●●●● ● ● ● ● ● ●●●●● ●● ●● ● ● ●● ●●● ● ●● ●●● ● ● ●● ●● ●● ●● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ●● ● ●●● ● ● ●● ● ●● ●● ● ● ● ● ● ● ●● ● ● ●● ●●●●● ● ●●● ●● ● ●●● ●● ● ● ● ●● ● ●● ● Comp.4 ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ●● ●●● ●● ● ●●●● ●●● ● ● ● ●● ● ●● ● ●● ● ● ● ●● ● ●● ●●● ● ●●● ● ● ● ●● ● ● ●● ● ●● ●● ● ● ●● ● ● ● ●● ● ● ●●●● ● ● ● ●● ● ●●● ●● ● ●● ● ●● ●● ● ● ●●● ● ● ● ●● ● ● ●● ● ● ●●● ● ●● ●● ● ● ●●● ● ● ●●● ● ● ● ● ● ● ● ●● ● ●●● ●● ●● ● ●●●●●●● ●●● ● ● ● ● ● ● ● ● ●● ● 4 ●● ● −4 −2 0 2 4 4 ● Comp.2 2 2 −40 0 20 ● ● ● ● ● ●● ● ●● ●●●● ●● ● ● ●● ●● ● ● ● ● ● ●●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ● ● ● ●● ● ●● ●● ●● ● ● ●● ● ●● ●●● ●● ● ● ● ● ●●● ●● ●●● ● ● ● ●● ●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ●● ●● ● ● ● ●● ●● ●● ●● ●● ●● ● ● ●● ● ●● ● ●● ●●● ● ● ● ● 0 ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ●● ● ● ●●● ● ●● ●● ●●● ● ● ● ● ●● ● ●● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●●● ● ●● ● ●●● ● ● ●●● ●● ● ● ●● ● ● ● ● ● ●●●● ●● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●● ● ●● ● ● ● ●● ● ●●●● ● ●●● ● ●● ●● ● ● ● ● ● ●● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● 0 Comp.1 −2 ● ● ● ● ● ● ● ●● ● ●●●●● ●●●● ●● ● ● ● ●● ● ●●●●●● ●● ● ●● ●● ●● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ●● ● ●● ● ●●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ●● ● ●● ● ●● ● ● ●●●● ●●●● ● ● ● ●● ● ●● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● −100 20 −2 0 ● ●●● ● ●● ● ● ● ●● ●● ● ● ●●● ● ● ●● ● ●● ● ● ●● ● ● ●● ●●● ● ●● ●● ● ● ● ● ● ● ●●● ● ●● ● ● ● ●● ● ● ●●●●● ●● ● ● ●● ● ●●●● ● ● ●● ● ●● ●●● ●●● ● ● ●● ●● ● ● ● ●●● ●● ●● ● ● ●●●● ●● ● ● ● ●● ● ●●● ● ●●●● ●● ● ●● ● ● ● ● ●●● ● ● ●● ● ●●● ● ●● ● ● ● ●● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ● −4 −40 ICS R Preliminaries Multivariate Models Whitening PCA ICA Australian Athletes Data III > pairs(pca.AS$scores,col=ais$sex) 20 0 −40 ● 4 ● ● ● ●● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ●● ● ● ●● ● ● ●● ● ●●● ●● ● ●● ● ●● ●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ●●● ● ● ● ● ● ●●● ● ●● ● ● ● ●● ●●●● ● ● ●●● ● ●● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ●● ● ●● ● ● ●●● ●●●● ● ● ● ●● ●● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● 2 ● ● ●● ● ● ● ● ●● ●●● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ●●● ● ● ● ● ●●●● ● ●● ●● ● ● ● ● ●●● ● ● ●● ● ●●●● ● ●●●●● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ●● ●● ●●●●● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ●●● ● ●● ● ● ●●● ● ● ●● ● ● ● ●● ●● ● ●● ●● ● ●● ●● ● −100 0 ● ● ● ● ● ●● ●● ● ● ● ●●●● ● ●●● ●● ● ● ●●●●●●●●● ●● ● ● ● ● ● ● ● ●●●● ●● ● ●●●● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ●●● ●● ●●● ●●●●● ● ●● ●●●● ● ● ● ●● ● ● ●●●● ● ● ●● ●● ●● ●● ●● ● ● ●● ● ●● ●● ●●● ● ●● ● ●● ● ● ●● ●● ● ●●● ●●●● ●● ●● ● ●● ● ● ●● ● ● ●●● ● ●● ● ● ● ●● ●● ●●●●●●● 0 2 ● ● ● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ● ●● ● ● ● ●● ●● ●● ● ● ● ●● ●● ●●●●● ● ● ● ● ● ● ●●●●● ●● ●● ● ● ●● ●●● ● ●● ●●● ● ● ●● ●● ●● ●● ● ●●● ● ● ● ● ●● ● ●● ●● ●● ●● ●● ●● ●●●● ● ●● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ● ● ● ● ● ● ●● ● ●●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●●●●● ● ●●● ● ●● ●●● ●● ● ● ● ●● ● ●● ● ●● ● −2 0 ● ● ● ● ● ●● ● ●● ● ● ●●● ●● ●● ●● ●●● ●● ● ● ●●● ●● ●●● ● ● ● ● ●●●●●● ●●● ●● ● ●● ● ● ● ●● ●● ● ●● ● ● ●● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ●● ● ●●●● ● ● ●● ● ● ● ●● ● ●● ●● ● ●●● ● ● ● ● ● ●● ●●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ●● ● ● ●● ●● ●● ● ● ●●● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● Comp.3 −4 ● ● ● ● ●● ● ● ●● ●● ●● ● ●●● ● ● ●● ● ●● ● ●● ●● ● ● ● ● ●● ●●● ● ● ●●●●● ● ● ●●● ● ● ●● ●● ● ● ●● ●● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ●●● ●● ● ●● ● ●●● ●●● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ●●● ● ●●●●● ●● ●● ● ● ● ●● ●●● ● ● ● ●● ● ● ● ● ● ● ●● ●● ●●● ●● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●●●●●● ●● ● ● ● ●● ●●● ● ●● ●●● ●●● ●● ● ●● ● ● ●● ●● ● ●●● ● ● ●●●● ● ●●● ● ● ● ●●● ●● ● ●● ●● ● ● ●● ●● ●● ●● ● ●●● ● ● ●● ●● ● ● ●● ● ●●●●● ● ●● ● ● ● ●● ● ●●●● ● ●● ●● ●● ● ● ●● ●● ● ● ● ● ● ● ● ●●● ●●●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●●● ● ● ●●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ●● ●●● ●● ● ●●●● ●●● ● ● ● ●● ● ●● ● ●● ● ● ● ●● ● ●● ●●● ● ●●● ● ● ● ●● ● ● ●● ● ●● ●● ● ● ●● ● ● ● ●● ● ●●●● ● ● ● ●● ● ●● ●● ● ●● ● ●● ●● ●● ● ● ●●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●●● ● ●●● ● ● ● ● ●● ● ●●● ●● ●● ● ●●●●●●● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● −2 0 ● ● ● ● Comp.2 4 4 Comp.4 4 ● ● 2 2 ● ● ● ● ● ●● ● ●● ●●●● ●● ● ● ●● ●● ● ● ● ● ● ●●● ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ● ● ● ●● ● ●● ●● ●● ● ● ●● ● ●● ●●● ●● ● ● ● ● ●●● ●● ●●● ● ● ● ●● ●●● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ●●● ● ● ●● ● ●● ● ● ●● ●● ● ● ● ● ● ● ●● ●● ● ●● ● ● ● ●● ● ● ●● ● ●● ●● ●● ●● ●● ● ● ●● ● ● ●● ● ●● ●●● ● ● ● ● 0 ● ● ●● ● ● ●● ●● ● ● ●● ● ● ●● ● ● ● ●● ● ● ●● ● ●● ● ● ●●● ● ●● ●● ●●● ● ● ● ● ●● ● ●● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●●● ● ●● ● ●●● ● ● ●●● ●● ● ● ●● ● ● ● ● ● ●●●● ●● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●● ● ●● ● ● ● ●● ● ●●●● ● ●●● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● 0 ● Comp.1 −2 ● ●● ● ● ● ● ● ● ●●●●● ● ●●●● ●● ● ● ● ●● ● ●●●●●● ●● ● ●● ●● ●● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ●● ● ●● ● ●●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ●● ● ●● ● ●● ● ● ●●●● ●●●● ● ● ● ●● ● ●● ● ● ● ●● ● ●●●●● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ● ●●●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● −100 20 −2 0 ● ●●● ● ●● ● ● ● ●● ●● ● ● ●●● ● ● ●● ● ●● ● ● ●● ● ● ●● ●●● ● ●● ●● ● ● ● ● ● ● ●●● ● ●● ● ● ● ●● ● ● ●●●●● ●● ● ● ●● ● ●●●● ● ● ●● ● ●● ●●● ●●● ● ● ●● ●● ● ● ● ●●● ●● ●● ● ● ●●●● ●● ● ● ● ●● ● ●●● ● ●●●● ●● ● ●● ● ● ● ● ●●● ● ● ● ● ●● ●●● ● ● ● ● ●●● ● ● ●●● ● ●● ● ● ●● ● ● ● ● −4 −40 ICS R Preliminaries Multivariate Models Whitening PCA ICA Australian Athletes Data IV Descriptive statistics using ICS > LOCATION <- colMeans(data.ana) > LOCATION BMI Bfat ssf LBM 22.95589 13.50743 69.02178 64.87371 > > SCATTER <- cov(data.ana) > SCATTER BMI Bfat ssf LBM BMI 8.202111 3.324882 29.94890 26.72126 Bfat 3.324882 38.313946 194.11891 -29.27451 ssf 29.948901 194.118912 1060.50092 -88.42529 LBM 26.721255 -29.274514 -88.42529 170.83006 ICS R Preliminaries Multivariate Models Whitening PCA ICA ICS R Australian Athletes Data V > > + > data.ana.centered <- sweep(data.ana, 2, LOCATION, "-") ics.AS <- ics(data.ana.centered, stdKurt = FALSE, stdB="Z") summary(ics.AS) ICS based on two scatter matrices S1: cov S2: cov4 The generalized kurtosis measures of the components are: [1] 1.7154 1.2788 0.9353 0.7659 The Unmixing matrix is: [,1] [,2] [,3] [,4] [1,] 0.1777 -0.4586 0.0975 -0.0527 [2,] 0.5371 0.1698 -0.0552 -0.0418 [3,] 0.3740 -0.3108 0.0256 -0.1471 [4,] 0.1142 0.4964 -0.0775 -0.0147 Preliminaries Multivariate Models Whitening PCA ICA Australian Athletes Data VI > + > > > Z <- sweep(ics.components(ics.AS),2, sign(mean3(ics.components(ics.AS))),"*") SKEWNESS <- mean3(Z) SKEWNESS IC.1 IC.2 IC.3 IC.4 0.48211688 0.09281212 0.14727342 0.16866117 > KURTOSIS <- ics.AS@gKurt > KURTOSIS [1] 1.7154201 1.2787838 0.9352998 0.7658729 ICS R Preliminaries Multivariate Models Whitening PCA ICA Australian Athletes Data VII > pairs(Z) 0 1 2 3 −2 2 1 0 ● ● ●● ● ● ●● ●● ●● ● ● ● ●● ●● ● ● ● ●●● ● ●● ● ●●● ● ● ● ● ● ● ●● ● ●●● ● ●●●●● ● ● ●●● ● ●●●● ● ●● ●● ● ● ● ●●● ●● ●● ●●● ●● ●●● ● ● ●● ● ●●● ●●● ●●● ●● ● ●●●●● ●● ● ●●● ● ● ● ● ● ●● ●●●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ●●● ● ● ●● ● ●● ● ●●● ● ●● ●● ●● ● ● ●●● ● ● IC.2 ● ● ● ● ●●● ●● ●● ●● ● ● ● ●● ●● ● ● ● ●● ● ●● ● ● ●● ● ●● ●● ● ● ● ● ● ●●● ●●● ● ● ●● ● ●●● ● ●● ●● ●● ●●● ● ●●● ●● ●● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ● ● ●●● ● ●● ● ●●●●●● ●● ● ●● ●●●● ● ● ●● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ●● ●● ● ● ●●●● ● ● ● −2 0 1 2 3 4 ● ●● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ●● ●●● ● ● ●●● ● ● ●● ● ● ●● ● ●● ● ● ●● ●● ● ● ● ● ●● ● ● ● ●● ● ●●● ●● ● ● ●● ●● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●●●● ● ●● ●●● ●● ● ●●● ● ●● ● ● ● ● ● ● ●●● ● ●● ●●● ● ● ● ●● ● ● ●● ●●● ●● ●●●● ● ● ●●● ● ●● ●● ● ●● ●●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●●● ● ● ● ● ● ● ● ●●● ●●● ●● ● ●● ● ● ● ● ●● ● ●● ●● ● ●● ●● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ●● ● ●● ● ●● ●●● ● ● ● ● ● ●●● ● ●●● ●●● ● ● ●● ● ● ●● ● ● ● ● ● ● ●● ● ●●● ● ●●● ● ● ● ●●● ● ●●●●●● ● ● ● ●●● ●●●●● ●●● ●● ●●●● ● ● ●● ● ●● ● ● ●●● ● ●● ●●●● ● ●● ●● ●●● ●●● ● ● ● ● ● ●● ● ● ●● ● ●●● ●● ● ● ●● ● ● ● ●● ● ●● ● ● ●● ●●● ● ●● ● ● ● ●● ●● ● ●● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ●● ● ● ● ● ●● ●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●●● ● ● ●●● ●● ●●●● ●● ● ● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●● ●● ● ●● ●●● ●● ●●● ● ● ●● ●●● ●● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ●●● ●● ● ●●●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ●● ● ● ● ●● ●●● ●● ●● ●● ● ●●● ●● ● ● ● ● ● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ● ●● ● ● ●●● ●● ●●● ●● ● ●●● ● ●● ● ● ● ● ● ●●● ●●● ● ● ●●● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ●● ● ●●● ●●●● ● ●●●● ●● ●● ●● ● ● ● ●●● ●● ● ● ●● ● ● ● ●●● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ●●● ●● ● ● ● ●●●●● ●● ● ●●● ● ●● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ●●● ● ●● ●●● ● ● ●● ●● ● ● ● ● ●●●●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ● ●●● ● ●●● ●● ● ●●● ●●●● ●● ● ● ● ●● ● ● ●● ●● ● ● ● ●● ●●● ●● ●● ● ● ●●●● ●● ● ●● ● ● ●● ● ●● ● ●● ●● ● ●● ● ●● ● ● ● ●●●● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ●● ●● ● ● ●● ● ● ●●● ●●●● ●●● ● ● ●● ● ●● ● 2 ● ● 0 1 2 3 4 ● ● ● ● ● ● ● ●● ●● ● ● ●●● ●● ● ●● ●● ● ●● ● ● ●● ●● ●● ●●● ●●●● ● ●●● ●●●●●● ●● ● ● ●● ● ●● ● ● ● ●●●●● ● ●● ●● ●● ● ● ●● ●● ● ● ● ● ● ●● ●● ● ●● ●●● ● ● ● ● ● ●● ●● ● ●●●● ● ● ●● ●● ●●● ● ● ● ●●●●● ● ● ● ●● ● ● ●● ● ●● ● ● ●●● ● ●● ● ● ●● ● ● ●●● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● IC.3 ● ● ● ●● ● ● ●● ●● ● ● ● ●●● ● ●● ●●● ● ●● ●● ● ● ●● ● ●● ●● ● ● ● ● ●●● ●●●● ● ●●● ●●● ●● ● ● ● ● ●●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ●● ●● ● ● ●●●●● ● ● ● ● ●● ● ●● ● ●●● ●● ● ● ●● ● ●●● ● ●● ● ● ●● ● ●● ● ● ●● ● ●●● ●●● ●● ● ● ●● ●●● ●● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ● ●● ●● ● ●● −3 −1 1 2 3 IC.4 ● 1 2 3 ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ●●● ● ● ● ●● ●● ●● ●● ● ●●●● ●● ●●● ● ● ●● ●● ● ●●●● ●●● ● ● ● ●●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ●●●●● ●●● ● ● ● ● ● ●●●●●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●●●●● ●● ● ● ● ● ● ● ● ● ● ●●● ●● ●●●● ● ● ● ●● ●● ●● ● ● ● ● ● ●●●●● ●● ● ● ● ● ● ●●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●●● ●● ● ●● ● ● ● ● −2 ● 1 ● ● ● −1 −2 0 1 2 3 IC.1 0 ● ● −2 ● ● ●● −3 −2 ICS R Preliminaries Multivariate Models Whitening PCA ICA Australian Athletes Data VIII > pairs(Z, col=ais$sex) 0 1 2 3 −2 2 1 0 ● ● ●● ● ● ●● ●● ●● ● ● ● ●● ●● ● ● ● ●●● ● ●● ● ●●● ● ● ● ● ● ● ●● ● ●●● ● ●●●●● ● ● ●●● ● ●●●● ● ●● ●● ● ● ● ●●● ●● ●● ●●● ●● ●●● ● ● ●● ● ●●● ●●● ●●● ●● ● ●●●●● ●● ● ●●● ● ● ● ● ● ●● ●●●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ●● ●●● ● ● ●● ● ●● ● ●●● ● ●● ●● ●● ● ● ●●● ● ● IC.2 ● ● ● ● ●●● ●● ●● ●● ● ● ● ●● ●● ● ● ● ●● ● ●● ● ● ●● ● ●● ●● ● ● ● ● ● ●●● ●●● ● ● ●● ● ●●● ● ●● ●● ●● ●●● ● ●●● ●● ●● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ● ● ●●● ● ●● ● ●●●●●● ●● ● ●● ●●●● ● ● ●● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ●● ●● ● ● ●●●● ● ● ● −2 0 1 2 3 4 ● ●● ● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ●● ●●● ● ● ●●● ● ● ●● ● ● ●● ● ●● ● ● ●● ●● ● ● ● ● ●● ● ● ● ●● ● ●●● ●● ● ● ●● ●● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●●●● ● ●● ●●● ●● ● ●●● ● ●● ● ● ● ● ● ● ●●● ● ●● ●●● ● ● ● ●● ● ● ●● ●●● ●● ●●●● ● ● ●●● ● ●● ●● ● ●● ●●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●●●● ● ● ● ●● ● ● ● ● ●●●●● ● ● ● ● ● ● ● ●●● ●●● ●● ● ●● ● ● ● ● ●● ● ●● ●● ● ●● ●● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ●● ● ●● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ●● ● ●● ● ●● ●●● ● ● ● ● ● ●●● ● ●●● ●●● ● ● ●● ● ● ●● ● ● ● ● ● ● ●● ● ●●● ● ●●● ● ● ● ●●● ● ●●●●●● ● ● ● ●●● ●●●●● ●●● ●● ●●●● ● ● ●● ● ●● ● ● ●●● ● ●● ●●●● ● ●● ●● ●●● ●●● ● ● ● ● ● ●● ● ● ●● ● ●●● ●● ● ● ●● ● ● ● ●● ● ●● ● ● ●● ●●● ● ●● ● ● ● ●● ●● ● ●● ● ● ●● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ●● ● ● ● ● ●● ●●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●●● ● ● ●●● ●● ●●●● ●● ● ● ● ●●● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●● ●● ● ●● ●●● ●● ●●● ● ● ●● ●●● ●● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ●●● ●● ● ●●●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ●● ● ● ● ●● ●●● ●● ●● ●● ● ●●● ●● ● ● ● ● ● ● ● ● ●● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●●●● ● ● ● ● ●● ● ● ●●● ●● ●●● ●● ● ●●● ● ●● ● ● ● ● ● ●●● ●●● ● ● ●●● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ●● ● ●●● ●●●● ● ●●●● ●● ●● ●● ● ● ● ●●● ●● ● ● ●● ● ● ● ●●● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ●●● ●● ● ● ● ●●●●● ●● ● ●●● ● ●● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ●●● ● ●● ●●● ● ● ●● ●● ● ● ● ● ●●●●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ● ●●● ● ●●● ●● ● ●●● ●●●● ●● ● ● ● ●● ● ● ●● ●● ● ● ● ●● ●●● ●● ●● ● ● ●●●● ●● ● ●● ● ● ●● ● ●● ● ●● ●● ● ●● ● ●● ● ● ● ●●●● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ●● ●● ● ● ●● ● ● ●●● ●●●● ●●● ● ● ●● ● ●● ● 2 ● ● 0 1 2 3 4 ● ● ● ● ● ● ● ●● ●● ● ● ●●● ●● ● ●● ●● ● ●● ● ● ●● ●● ●● ●●● ●●●● ● ●●● ●●●●●● ●● ● ● ●● ● ●● ● ● ● ●●●●● ● ●● ●● ●● ● ● ●● ●● ● ● ● ● ● ●● ●● ● ●● ●●● ● ● ● ● ● ●● ●● ● ●●●● ● ● ●● ●● ●●● ● ● ● ●●●●● ● ● ● ●● ● ● ●● ● ●● ● ● ●●● ● ●● ● ● ●● ● ● ●●● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● IC.3 ● ● ● ●● ● ● ●● ●● ● ● ● ●●● ● ●● ●●● ● ●● ●● ● ● ●● ● ●● ●● ● ● ● ● ●●● ●●●● ● ●●● ●●● ●● ● ● ● ● ●●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ●● ●● ● ● ●●●●● ● ● ● ● ●● ● ●● ● ●●● ●● ● ● ●● ● ●●● ● ●● ● ● ●● ● ●● ● ● ●● ● ●●● ●●● ●● ● ● ●● ●●● ●● ● ● ●● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ● ●● ●● ● ●● −3 −1 1 2 3 IC.4 ● 1 2 3 ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ●●● ● ● ● ●● ●● ●● ●● ● ●●●● ●● ●●● ● ● ●● ●● ● ●●●● ●●● ● ● ● ●●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ●●●●● ●●● ● ● ● ● ● ●●●●●●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ●●●●● ●● ● ● ● ● ● ● ● ● ● ●●● ●● ●●●● ● ● ● ●● ●● ●● ● ● ● ● ● ●●●●● ●● ● ● ● ● ● ●●● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●●● ●● ● ●● ● ● ● ● −2 ● 1 ● ● ● −1 −2 0 1 2 3 IC.1 0 ● ● −2 ● ● ●● −3 −2 ICS R Preliminaries Multivariate Models Whitening PCA ICA ICS Australian Athletes Data IX Analyzing only the females > data.ana.females<-data.ana[ais$sex=="f",] > pca.AS.f<-princomp(data.ana.females,score=TRUE) > pairs(pca.AS.f$scores) ●● ● ● Comp.1 5 15 −3 ● ●● ●● ● ●●●● ● ● ● ●●● ●● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●● ● ●● ● ● ●● ● ● ●● ●●● ●●●● ● ● ● ●●● ● ●●● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ●● ●● ● ● ● ● ● ● −15 −5 5 15 ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ●● ● ● ● ●● ● ● ● ● ●● ●● ●● ●●● ● ●● ●● ●●● ●●● ● ●● ● ●● ● ● ●● ●● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 3 1 −3 −1 ● −100 ● ● ●● ● ●● ● ● ● ●●●● ● ● ●●●● ● ●● ● ●● ●● ●● ● ● ● ● ● ● ●● ● ●● ●●● ●●● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ●●● ● ● ● ●●● ●● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● 0 50 ● ● ● ●● ●● ● ● ●● ● ●● ● ● ● ●● ● ●● ●● ● ●● ● ● ● ● ● ●● ●●● ●●● ● ● ●● ● ● ● ● ● ● ●● ● ●●●● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ●● ●● ●● ●● ●● ● ●● ● ●● ● ● ● ● ●●● ●● ● ●● ● ● ● ● ● ●● ●● ●● ●● ● ●● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ●● ● ● ● ●●● ●● ● ●●● ●● ●● ●● ● ● ●● ● ● ●● ●●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●●● ● ● ●●● ●●● ● ● ● ●● ● ● ●● ●● ● ●● ● ●●● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ●●● ● ● ●● ●●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ●●● ● ● ● ●● ●●● ● ●● ● ●● ● ● ● ● ●● ●● ●● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● Comp.3 ● ● ● ● ●● ●● ●● ● ●● ● ●●●●●● ● ●● ●●●● ● ●● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ● ●●●● ● ● ●●● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ●● ● ●● ● ● ●●● ● ● ● −2 0 2 ● 4 Comp.4 0 ● ● ● −2 ● ● 2 4 ● 3 ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ●● ● ●● ● ● ●●●● ●● ● ●● ●● ● ●● ● ●● ●●● ● ● ● ● ●●● ● ●● ● ● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● Comp.2 1 ● ● ●● ● ●● ● ● ● ● ●● ●● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●●●● ●●● ● ●●● ● ●● ● ● ●●●● ●● ● ● ●● ● ● ● ● ●●● ● ●●●● ● ●● ● ● ● ● ● ● ●● ●● ● ●● ● ● ● −1 ● ● ● ●● ●● ●● ● ● ●● ● ● ● ● ●● ●● ●●● ●● ● ● ●●● ●● ● ●● ● ●● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●●● ●● ●● ● ●●● ●● ● ● ●● ●● ● ● ● ●● ●● ●● ● ● ●●● ●● ●● ●● ● ● 50 −5 ● ●● ● ● ● ●● ●● ●●● ● ● ● ● ●● ● ● ● ● ●● ●● ●● ●●● ● ● ● ●●● ● ● ●● ● ● ●● ●● ● ● ● ● ●●● ● ● ● ● ●●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● 0 ● −100 −15 R Preliminaries Multivariate Models Whitening PCA ICA Australian Athletes Data X Analyzing only the females > LOCATION <- colMeans(data.ana.females) > LOCATION BMI Bfat ssf LBM 21.9892 17.8491 86.9730 54.8949 > > SCATTER <- cov(data.ana.females) > SCATTER BMI Bfat ssf LBM BMI 6.969747 9.508302 60.63405 13.66024 Bfat 9.508302 29.734018 178.96012 15.33174 ssf 60.634049 178.960117 1145.86058 95.24906 LBM 13.660242 15.331741 95.24906 47.91675 ICS R Preliminaries Multivariate Models Whitening PCA ICA ICS R Australian Athletes Data XI > data.ana.f.centered <- sweep(data.ana.females, 2, + LOCATION, "-") > ics.AS.f <- ics(data.ana.f.centered, stdKurt = FALSE, + stdB="Z") > summary(ics.AS.f) ICS based on two scatter matrices S1: cov S2: cov4 The generalized kurtosis measures of the components are: [1] 1.3583 1.1320 1.0322 0.8330 The Unmixing matrix is: [,1] [,2] [,3] [,4] [1,] 0.2996 -0.3870 0.0640 -0.0034 [2,] -0.2892 -0.0861 0.0496 -0.0496 [3,] 0.5923 0.1401 -0.0378 -0.2158 [4,] 0.0414 0.6212 -0.0863 0.0232 Preliminaries Multivariate Models Whitening PCA ICA Australian Athletes Data XII > + > > > Z.f <- sweep(ics.components(ics.AS.f),2, sign(mean3(ics.components(ics.AS.f))),"*") SKEWNESS <- mean3(Z.f) SKEWNESS IC.1 IC.2 IC.3 IC.4 0.32308843 0.16111629 0.13685658 0.06543187 > KURTOSIS <- ics.AS.f@gKurt > KURTOSIS [1] 1.3583203 1.1320473 1.0322094 0.8329946 ICS R Preliminaries Multivariate Models Whitening PCA ICA Australian Athletes Data XIII > pairs(Z.f) 0 1 2 3 −2 −1 0 ● 1 2 ● 3 ● 4 −2 ● ● 3 ● ● −2 0 1 2 ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ●● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ●●● ●●●● ●● ● ● ●●● ● ●● ●● ● ●● ● ● ●● ●● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● IC.2 ● 2 1 ● IC.3 ● ● ● ● ● ● ●● ●● ● ● ●● ●●● ● ● ● ● ● ● ●● ● ●●●● ● ●●●● ●● ● ● ● ● ●● ●● ● ●● ● ● ● ●● ●● ● ● ● ●● ● ● ● ●● ● ● ●●●● ●●● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● 2 ● ● ● ● ●● ● ●●● ● ● ● ●● ● ● ●● ● ●● ●● ● ● ●● ● ●●●● ● ●● ● ●●●●● ● ● ●● ● ●●●● ● ● ● ● ●●● ●● ● ● ● ●● ● ● ● ●● ● ● ● ●●● ●● ●●●● ●●●● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ●● ● ● ●●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ●●● ● ● ● ●● ●● ● ● ● ● ●●● ● ● ● ● ●● ● ● ●● ●● ●● ● ● ● ● ●● ● ●● ● ●●● ● ● ● ● ● 1 ● ● ● ● ●●●● ● ● ●● ● ● ● ●●● ● ● ● ●● ● ●● ● ●● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ●●●● ● ● ● ● ●● ● ● ●●●● ● ● ● ● ●●●●● ●● ● ● ●● ● ●● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● 0 ● ● ● ● ●●● ● ●● ● ● ●● ●● ● ● ●● ● ●● ●● ● ● ●● ● ●●● ●●●● ● ●● ●● ●● ●● ● ● ●● ● ● ●● ● ●● ●●● ● ● ●● ● ●●●●●● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● −2 −1 ● ● ● ● ● ● ● ● ●●● ● ● ●●● ●● ● ● ● ●● ●● ● ● ● ●● ●●● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ●● ● ● ●● ● ● ●●● ● ● ●●● ● ●●● ●● ●● ● ●●● ●● ● ●● ● ● ● ●●● ● −1 0 ● ●● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ●● ● ●●●● ● ●● ●● ● ●● ●● ● ● ●● ●● ●●● ● ● ● ●● ● ● ●● ●● ● ● ● ● ●● ●● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ●● ● ●● ●● ● ● ● ● ●● ● ● 3 4 ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ●● ● ●● ●● ●●● ●● ● ●● ● ●● ●●● ●●●● ●● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ●● ● ●● ●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● 2 ● 1 ● −1 0 ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ●●● ●●●● ● ● ● ●● ●●● ●●● ● ● ● ●●●●●● ●● ● ● ●● ●● ●● ● ●●● ● ●● ● ●● ● ●● ●● ● ● ●● ● ● ●● ● ● ● 1 2 3 ● ● ● ● ● −3 ●● ●● ● ●● ● ● ● ● ● ● ●●● ● ●●● ● ● ● ● ● ● ●● ●● ●●●●● ● ●● ● ● ● ● ● ● ● ●● ●● ●●● ● ●●● ● ●●●●● ● ●● ● ●●● ● ● ●● ● ● ●●● ●● ● ● ●●● ● ● ● ● ● ● −1 1 2 3 IC.4 −3 ● ● IC.1 ● ● ● ●● ● ● ● ●●●●●● ●● ● ●●●●● ● ● ●● ● ● ● ●●● ●● ● ● ●● ●●● ● ●●● ● ● ●●● ● ●● ● ●● ●●● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ●● ●● ● ●● ● ● ●● ● ●● ● ● ● −1 ● ICS R Preliminaries Multivariate Models Whitening PCA ICA ICS Key references I Tyler, D. E., Critchley, F., Dümbgen, L. and Oja, H. (2008). Exploring multivariate data via multiple scatter matrices. Journal of the Royal Statistical Society, Series B, accepted. Oja, H., Sirkiä, S. and Eriksson, J. (2006). Scatter matrices and independent component analysis. Austrian Journal of Statistics, 19, 175–189. Nordhausen, K., Oja, H. and Tyler, D. E. (2008). Two scatter matrices for multivariate data analysis: The package ICS. Journal of Statistical Software, accepted. Nordhausen, K., Oja, H. and Ollila, E. (2008). Multivariate models and their first four moments. Submitted. R Preliminaries Multivariate Models Whitening PCA ICA ICS Key references II Nordhausen, K., Oja, H. and Ollila, E. (2008): Robust independent component analysis based on two scatter matrices. Austrian Journal of Statistics, 37, 91–100. Nordhausen, K., Oja, H. and Tyler, D. E. (2006): On the efficiency of invariant multivariate sign and rank tests. In Liski, E.P., Isotalo, J., Niemelä, J., Puntanen, S., and Styan, G.P.H. (editors) “Festschrift for Tarmo Pukkila on his 60th birthday”, 217–231, University of Tampere, Tampere. Kankainen, A., Taskinen, S. and Oja, H. (2007). Tests of multinormality based on location vectors and scatter matrices. Statistical Methods & Applications, 16, 357–379. R
Documentos relacionados
Invariant coordinate selection for multivariate data analysis
Finding outliers, structure and clusters In general, that the coordinates are ordered according to their kurtosis is a useful feature. One can assume that interesting directions are the ones with e...
Leia mais