<- file stat 97anocha.html -> Outcome can be measured by raw POST score (ignore PRE), simple change score, or regressed change score. If your results are not large and "robust" enough to test-out the same regardless how you test, then it matters that you can tell which Outcome is appropriate. This file has notes on paired t, assumptions, Krantz slopes in ANCOVA, Krantz difference scores, Ulrich ANOVA vs ANCOVA, Ulrich

paired t, assumptions

=======================Dave Krantz, 08 Feb 1997==========sse Message-ID: <199702081650.LAA27441@paradox.psych.columbia.edu> From: dhk@paradox.psych.columbia.edu (Dave Krantz) Subject: Re: Matched Pairs T Assumptions, Again Hi, Paul, Since no one else wants to answer, I'll take a crack at it. First, the formula var(X-Y) = var(X) + var(Y) - 2 * rXY * sqrt(var(X)*var(Y)) (where rXY is the product-moment correlation of X, Y) is an algebraic identity, that is, it is always true, without any assumptions of any kind. Therefore the two methods that you give for calculating the standard error of difference scores should always give exactly the same answer apart from rounding errors or the like. I prefer to use difference scores directly, since I like to look at the distribution of any variable that I am thinking about (skewness, heavy tails, graininess, multimodality, etc.) But it is also worth knowing the value of rXY (more about that below); and sometimes it is convenient to use var(X), var(Y) and rXY in the calculation, if those are readily available and the original distribution of X-Y is not. Second, one should give careful consideration to whether difference scores should be used at all. The model underlying the pair t test is Y - X = mu + error , which can be rewritten as Y = X + mu + error , i.e., one is postulating a structural relation between Y and X which is linear with slope = 1 and intercept = mu, with constant variance (homoscedasticity). There are many instances in which this is not a sensible model. For example, if one thinks about a treatment that moves an individual subject a fixed fraction R of the way toward an asymptotic level A, from a starting position X, with some random variation in effect, then the relationship between posttreatment score Y and pretreatment X would be Y = X + R * (A - X) + error which implies a linear relation between Y and X with slope 1-R and intercept R*A. True, the null hypothesis R=0 reduces to Y = X + error, which is the same as the null hypothesis for the pair t test; but confidence intervals for the mean difference score are not sensible; what one would want here is confidence intervals for R and for A. For another example, if rXY is negative, you certainly cannot expect that the underlying relation of Y and X is linear with slope +1. Still another example is suggested by Paul's mention of homoscedasticity: the conditional variance of Y, given X, may vary considerably with X. Looking at the (X, Y) scatterplot is generally a good idea, before you reduce to difference scores. More generally, statistical tests usually involve a family of models fitted to the data; and it is important to think about whether the models make scientific sense, in light of existing theory, and also whether the whole family might be rejected out of hand by the data, before going ahead with a statistical test. I strongly prefer to think about models, which of course entail various assumptions, than about "assumptions" in isolation from underlying models. Dave Krantz (dhk@columbia.edu) ---Paul Bernhardt's query is excerpted below---- > Paul.Bernhardt@m.cc.utah.edu Fri Feb 7 13:08:02 1997 > Matched Pairs T Assumptions, Again > There are two ways to calculate the matched pairs (aka, correlated > measures, dependent measures, repeated measures) t test. One is the most > common by hand method of figuring a difference score from each pair of > scores and testing the mean difference against a null hypothesis > difference using a single sample t. The other is to pool the variance > taking into account the correlation between the two sets of paired > measures. > SE**2(diff) = SE**2(Mean1) + SE**2(Mean2) - (2 * r * SE**(Mean1) * > SE**2(Mean2)) > Is there an assumption of homogeneity of variance between the two sets of > paired measures? I have found some texts that clearly state there is, > Glass & Hopkins, 1996. But most other texts make no mention of it but do > mention this assumption for independent samples t tests. Checking in > older texts I eventually found two texts that mention a different > assumption: Johnson & Jackson (1959) and Hald (1952). Both texts describe > a different assumption involving variance. The assumption is that a plot > of the pairs of scores should fall on either side of a regression line > with a slope of 1. Also, the variance of the scores away from the line > should not increase or decrease as scores increase. This sounds like a > sort of homoscedasticity assumption. *--------

Slopes in ANCOVA

=======================Dave Krantz, 19 May 1997==========sse Message-ID: <199705191400.KAA08094@paradox.psych.columbia.edu> From: dhk@paradox.psych.columbia.edu (Dave Krantz) Subject: Re: ANCOVA with heterogeneous slopes? Much of what you say about ANCOVA with heterogeneous slopes is correct. If slopes are not equal (or nearly equal), then there is no sensible concept of "adjusted mean", but despite such an inherently interactive setting, one can just use ANCOVA software to fit what is essentially a different regression model for each group. The reservation I have about your statement derives from what I perceive as excessive reliance on the F test for unequal slopes to decide about what model to use in approaching the data. To be sure, the F test is relevant; but in the situations that most often arise in psychology (noisy data with medium sample sizes), the test has small power to detect interaction magnitudes that have considerable theoretical and/or practical importance; thus, failure to reject the equal-slopes assumption does not mean much. If there are strong theoretical reasons to think that slopes should vary substantially, and the trends in the data seem to accord with the expected variation in slope, then I would be very reluctant to use any other than the full regression model, in thinking about the findings. Dave Krantz (dhk@columbia.edu) *--------

Difference scores?

=======================Rich Ulrich, 28 May 1997==========ssc Subject: Re: Two Group Non-randomized Design Message-ID: <5mhl8f$k1k@usenet.srv.cis.pitt.edu> Harmon Jordan (Jordan_Harmon_S/bos1_BCBSMA@BCBSMA.COM) wrote: ... : I have a straightforward design involving a comparison group, treatment : group (one pre and one post treatment measure on the same individuals, : i.e. paired data), an intervention consisting of and educational : program, and no randomization: -- NO RANDOMIZATION is a potentially serious flaw. What makes it especially serious is if the samples do not MATCH on all the relevant variables, for example, Pre-score, plus age and sex (which you mention). << snip, detail >> : Options for analyzing such data include: : - "simple" gain score (X2-X1) - (Y2-Y1); : - alternatively, use the pre-measure as the covariate and the : post-measure as the response variable in an analysis of covariance to : take into account the estimated correlation between pre and post : treatment scores . Do not forget the third important option: compare POST directly, ignoring PRE. If the groups match on Pre, then you don't have to worry. But if they do not match, they you DO have to worry, because it *might* not be fair to say that a group did worse when it did *in fact* have the best average score - If the three criteria do not come out the same, then you do have to explain the differences, and justify the choice you make as what is proper to interpret. Some people would say: "You really can't judge anything if the groups were not random"; other folks would only be wholly negative if they they think the groups can't even be IMAGINED to have been random, even after matching on Sex, age, pre.... When groups match on Pre, then the covariance analysis is a safe one which minimizes error variance. The raw change-score has the advantage of being easy to describe and interpret, at least, if Pre and Post VARIANCES do match. : I'd welcome any recommendations about the pros and cons of these (or : other approaches) and suggestions on the best SPSS routine to implement : either. Thanks! -- Change scores are analysed in Repeated measures, which is not an SPSS forte. SPSS can manage something in MANOVA. But directly computing the raw Change is often more intelligible than reading the interaction terms that are apt to be the useful ones from Repeated Measures. ANOVA or MANOVA or regression can be used for covariance. *--------

ANOVA vs ANCOVA

=======================Rich Ulrich, 19 Jun 1997==========sse Subject: Help: Alternative statistical test for ANOVA for unequal two sample sizes Message-ID: <5oc3pi$67j@usenet.srv.cis.pitt.edu> Michael Babyak (mbabyak@acpub.duke.edu) wrote: : The merits of establishing 'baseline' homogeneity have been hotly debated : in the clinical trials community lately. I tend to favor the position of : Stephen Senn, who argues that such analyses are unecessary and even : potentially misleading. -- I wondered at 'hotly debated' and thought that had to be an overstatement, until I looked up the article by Senn. I sure to see how someone could respond to him 'hotly', anyway. Blithe arrogance, even while grandly mis-constructing an argument. ...He ultimately suggests that confounding variables : should be selected a priori, and included in the model irrespective of what : baseline tests might show. -- which ASSUMES than regressed-change score is ALWAYS the appropriate measure of a change. I will admit that it is best, usually, but that is a far statement from "ideal", "always". : In your particular case, it may be that you haven't sufficient power to : detect a meaningful difference anyway--and you certainly wouldn't want to : conclude that the groups were equal based on failure to reject the set of : null hypotheses. Although some authoritative texts endorse performing : baseline homogeneity tests, you might have a look at the paper cited below : in order to make an informed decision. (BTW, regardless of what you : decide, the unequal sample size would not necessarily be a problem for : ANOVA.) I'd be rather interested in hearing from others on this matter. : Ulrich? -- Does it hurt anything at ALL to DO the baseline tests, and gain that information? - no, so do it. Senn mentions the possibility of doing all the testing TWO ways, covariance, and otherwise. He does not like that, because "demanding BOTH tests to be positive" would place a greater burden on the testing. I would say that, if you are smart enough, then you can explain WHICH test is the preferrable one. If you cannot explain, then looking at BOTH tests is the better position, and you must present them both to your readers, because SOMETIMES the regressed-change score given to you by covariance is an unjustifiable criterion - and Senn would never know it. : SENN, S. (1994). TESTING FOR BASELINE BALANCE IN CLINICAL TRIALS, : STATISTICS IN MEDICINE 13, 1715-1726. * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Document by Rich Ulrich. E-mail to wpilib+@pitt.edu

FAQ top.

Ulrich home page.

Ulrich FAQ. http://www.pitt.edu/~wpilib/stats99.html