<- file 95mvtest.html -> Achieved classification; averaged tests; sphericity (Mauchley's test). ******************* ********** Doesn't good classification prove that the Discriminant Function works? ===================Rich Ulrich, 19 Jul 1995==========ssc Subject: Re: Help with Canon. discriminant analysis Message-ID: <3ujumg$n30@usenet.srv.cis.pitt.edu> : I've been running canonical discriminant analysis on SAS and I'm getting : results that I just don't understand. I am weak on the theory although I : have read Klecka (1980). : I have 46 observations (plots), 37 variables (bird species) and 4 classes. : I'm getting an excellent class separation on plot of the first and second : canon. axis and overall correlation is 0.94 and the Eigenvalue is 7.79 : for the first function. Wilks' Lambda is 0.012. : According to Klecka, these all suggest very high discrimination. : But...the p-value for Wilks' Lambda is 0.63. There are three other tests WHAT suggests high discrimination? A correlation of .94 is `high' unless it is the result of artifact, like it is here. Similarly, you (apparently) good `class separation' only because you have far too many variables. Look at a simpler problem: If there were only 2 classes, then, with 37 variables and 46 observations, you would have an expected R-squared of 37/46 or .80 for an expected overall correlation of .90. Having two more classes seems to boost the Expected correlation to the vicinity of .95 since your tests report p-values of 0.63, for the correlation of .94 . The 1990 edition of J. Cohen's book on power analysis gives some discussion of how awful it is to have too many predictors, or too many classes, in a multivariate analysis. I think he includes an exact formula for the Expected correlation for your case above, or else some Shrinkage formula. But that is your problem. Basically, your results are sensible, and (I would suggest) show why people in your position ought to be doing some other analyses entirely.

How do I read Multivariate tests that are provided?

=======================Rich Ulrich, 5 Jan 1996==========ssc Subject: Re: Multivariate vs Univariate tests? Desperately needed Message-ID: <4ck43r$65i@usenet.srv.cis.pitt.edu> : I have a question relating to the power of multivariate tests (Pillai's, : Wilk's...) vs. those of univariate tests when the conditionof sphericity : holds. [ ...some detail and example deleted ] : Here is another case: : Mauchly's test of sphericity .367 : Pillai .219 ) : Hotelling .219 } Multivariate tests : Wilks .219 ) : Averaged F 0.0001 ) : G-G adj 0.0001 } Univariate and adjusted tests (Greiger-Greenhouse, : H-F adj 0.0001 ) and Huyhn and Feldt) : I am totally stuck here. I remember from a stat class that when the : sphericity condition holds, the univariate tests (averaged Fs) are more : powerful than mutivariate tests. Is this true? In general, two tests will give different results because a) they are testing different things, or b) the conditions/assumptions of (at least) one of the tests are not being met. Speaking to the point of b) : it will be true that a general MANOVA has the hazard of running out of degrees of freedom - so it CAN be a very weak test, one that is so weak that it is useless. That is one condition where SOME other test has to be employed... I usually try to set up single-variables as contrasts (i.e., composite scores), and to ignore the MANOVA programs entirely. Speaking to the point of a) : I was hoping that someone else would respond to this question, because I do *not* know what is in these so-called Univariate results, or whether it depends on the Contrasts that you have requested when you set up the procedure. But, I am sure, it is certainly just a SUBSET of hypotheses that are tested by the MANOVA. So, what can one say about "power"? You "have more power" when you carry out a FEW specified contrasts, than if you look at ALL possible contrasts and control for the multiplicity through the math. But if you are not interested in THOSE contrasts, then there was no point in looking at those tests. In other words: More power for certain SIMPLE hypotheses; but it may not be what you want to test.

MV - What is Mauchley's test?

====================Rich Ulrich, 10 Jan 1996==========ssc Subject: Re: Multivariate vs Univariate tests? Desperately needed Message-ID: <4d0qmq$l5g@usenet.srv.cis.pitt.edu> Here was the original problem, : >Serigne F. Diop (sfdiop01@homer.louisville.edu) wrote: : >[ ...some detail and example deleted - showing NS multivariate tests, significant 'univariate tests', and NS for Mauchly's test for sphericity. Earlier, I commented that I did not know everything about THOSE named tests, but: : >In general, two tests will give different results because a) they are : >testing different things, or b) the conditions/assumptions of (at least) : >one of the tests are not being met. Providing some good clarification, but not addressing the original question, David Nichols (nichols@spss.com) wrote: : The null hypothesis tested by the univariate and multivariate tests : is the same. The results differ because different deductions from : that common state are used to test for it. The univariate test : results referred to here (SPSS calls them averaged univariate or : split plot or mixed model tests) are obtained using the traces of : the SSCP matrices used in the multivariate tests. In order to do : this, you have to use a set of orthonormal contrasts if you're : treating the data in what we call the multivariate setup. The value : for this test is invariant to individual comparisons; if the data : is set up as a univariate approach or split plot, it's just a : regular F, which is of course invariant to the individual contrasts : used if done properly. Checking my SPSS manuals, I see that the 'averaged univariate' tests will follow the use of a set of orthonormal contrasts AND they ASSUME that the variances of those contrasts are equal. Effectively, it seems, the averaged tests use (approximately) the *sums* of the Sums of Squares, for hypothesis and for error from the separate ANOVAs, which is valid if they are indeed uncorrelated and of equal variance. When I look at the data in an example in the _SPSS Reference Guide_ (1990), page 811, I see that the Mauchly test for sphericity (testing: uncorrelated? equal variance?) is not powerful for small N. (Please note, this is entirely incidental to the presentation of the manual's example.) There is a p-value for Mauchly's of .470 although the Error-SS across three groups ranges from 12.0 to 74.4. In this example, the Contrast with the LARGE error also had a large F, so the combined statistic is also impressively large. This gave a result with smaller 'p' than the multivariate tests, which implicitly weighted the three orthogonal contrasts equally, by not-assuming equal variances. Anyway: in the original problem that was asked, I would guess that either there is a small N or too-large n-of-vars, so that the multivariate tests lacked power. Or else, despite the Mauchly test, the conditions for the averaged tests were poorly met; which might throw doubt on the validity of the MODEL that the user has in mind.... ************ <> * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Document by Rich Ulrich. E-mail to wpilib+@pitt.edu

FAQ top.

Ulrich home page.

Ulrich FAQ. http://www.pitt.edu/~wpilib/stats99.html