<- file 95power.html -> ******************* **********

A generic power statement, Ulrich

==========================Rich Ulrich, ???=========ssc Subject: Re: If the Null is Rejected, Does Power Matter? Here's a statement or a few concerning Power: "Assuming that the underlying difference between two groups is .5 SDs, then with 84 subjects per group, the power of detecting a group difference using a two-tailed t-test at the 5% level is 90%. (For your further information: the discernable difference for two groups of 84 is .30 SDs, i.e., for .30 SDs, the 5% test will JUST BARELY be large enough to reject. - .30 is the column in the Cohen Table 2.3.5 labeled d-sub-c.) Assuming the underlying difference is .70 SDs, the power of the experiment to say that the groups differ would be 97%." On Pilot data, where the result met an alpha= .1% (.001): "If the effect size that was observed [with the p-level = .001] were the true, underlying effect, and if this experiment were re-run exactly, then the power of analysis (i.e., the likely of rejecting the Null hypothesis in the new experiment) would be 50% for a .1% level test; or 95% for a 5%-level test." - The "50%" [for a .1% test] is an immediate inference; on replication, one presumes that there is a 50% chance that the observed effect will be LARGER than what just happened, and thus meet the same test, and 50% chance of being smaller and NOT meeting that limit. The "95%" [for a 5%] is based on the estimate that effect size at .001 is about twice the distance as .05, so that z-score(.999)+ z-score(.50) =~ z-score(.95)+ s-score(.95) Making this kind of statement about POWER is equivalent to making statements about effect sizes, once one understands the mechanical substitution rules.... More on power - effect size ===================Rich Ulrich, 23 Mar 1995==========ssc Subject: Re: power problem Joe Wheaton (jwheaton@postbox.acs.ohio-state.edu) wrote: < ... details, setting alpha and beta at .01 ... > : My problem is that this chi-square is significant when its my understanding : that it shouldnt be. That , the Cramers is less than .115 (indicating a ES : less than .20) and yet the chi-square is still significant at an alpha of .01. : What happened? Ah, nothing is wrong, except for a faulty `understanding.' The power analysis is to ensure that if there is a big enough (underlying) effect, then you will get a significant difference. It does not say that if you get a significant difference, the effect will be BIG. (If you had asked for 50% power, instead of 99% power, then only the `interesting', BIG, outcomes would show up as significant.) Your 99% power guarantees that you will get a 1% significant result most of the time, so long as the UNDERLYING difference is .20; and that includes a number of occasions when the actual MANIFEST difference is rather less than .20, the size you deemed important. ===================Rich Ulrich, 26 Oct 1995==========sse Subject: Re: sample size needed Message-ID: <46p1l0$1oi@usenet.srv.cis.pitt.edu> Dr. John Jamieson (John.Jamieson@LakeheadU.Ca) wrote: : Cohen's rules (Statistical Power Analysis, 1988) for computing sample size : needed yield depressingly large numbers. Does anyone know of articles : critical of his estimates, or of Monte Carlo studies evaluating the : accuracy of his methods. It is miscalling it, to say "Cohen's rules ... " - Cohen is providing numbers that do not need any Monte Carlo studies because they are (if you accept the assumptions for a particular table) EXACT. His assumptions are not always the most detailed or precise, but I don't think you will find any source that will you give you GREATLY differing numbers. ... If the Ns seem to be too large to you, perhaps you are mis-applying something? - [added comment] If you are responding to what Cohen labels as "small", "medium", and "large" effects, you could be misled if you think those terms are *essential*. Those terms are indeed merely a convention he adopted, and they show categories that are typically appropriate for his audience, folks in the social sciences, who are using human subjects. By contrast, Epidemiologists require much larger samples to demonstrate what they consider Large effects; the Cohen tables still work, but in terms of "mean difference" or "correlation" - that is, the Cohen "effect sizes" - the effects sought by Epidemiologists are QUITE small, even when the Risk Ratio is large. Other folks, at the other extreme, do useful experiments with a handful of rats or petri dishes, because what they consider small, is LARGE by the numbers used by Cohen. If you are looking at the numeric "effect sizes" that are examined by Cohen, and those are what make you despair, then I guess that is really bad news for you, unless you may have just slipped up in the computations.... which is something that has been known to have happened. ===================Rich Ulrich, 14 Jan 1995==========ssm Subject: Re: |--- SOFTWARE TO PERFORM POWER ANALYSIS ? ---| Message-ID: <40nuf5$3k2@usenet.srv.cis.pitt.edu> Chiocchio Francois (chiocchf@ERE.UMontreal.CA) wrote: : Hello all. I have to do a power analysis and I would : like to know if there is software that handles it. I'm : used to take Cohen's book and to figure it out but this : time I will have to preform a 3*(6) MANOVA on 7 dependent : variables and Cohen uses only simple design as examples. I assume that you are trying to describe the sample size that you will for an experiment that is going to be carried out. - Since I have read a lot of power statements before, I don't think that I would BELIEVE any *complex* statement if it were not based on pilot data in the same form. - Power analyses are approximations. Approximate your final MV strategy by some simpler strategy. Multiple groups, multiple variables, multiple time periods : I think Cohen emphasizes how BAD multiplicity is for your eventual power. You PROBABLY ought to specify a few SIMPLE hypotheses, and then test those, for two reasons - power, and interpretability. Complex MV tests are hardly interpretable, anyway, without specific contrasts done in followup. ===================Rich Ulrich, 25 May 1995==========ssc Subject: Re: power estimates Message-ID: <3q2t0j$21e@usenet.srv.cis.pitt.edu> kbloom@ac.dal.ca wrote: : I am interested in the possibility of power and sample size estimates : for repeated measures anovas, and mixed (ws/bs) anovas. I am fairly : familiar with Cohen's text on Power, but there is no mention in it of : repeated anovas. I'd appreciate any advice or direction. The last comment that I saw about power for repeated measures (was it here?) was that four available computer programs gave at least four different answers for some simple test problems. That reaffirmed my long-standing opinion that Cohen was WISE to avoid the complications .... Substitute: Can you look at some change score, as a score? Pre/Post? (average of some Pre)/(average of some Post)? The other safe possibility, as I see it, is simple extrapolation from some pilot data that is very close to the intended design. Power is then easy to derive from the tables if you have a decent understanding of what the non-central distributions mean.

power example?: 2x10 contingency table =======================Rich Ulrich, 19 Jan 1996==========ssc Subject: Re: Q: Sample size determination for a chi-2 test Message-ID: <4dp01a$fdi@usenet.srv.cis.pitt.edu> : How do I perform a sample size determination for a chi-2 test ? : We would like to study a 2x10 table and test for difference in treatment effect.Is Wilcoxon-Mann-Whitney a serious alternative to the chi-2 test ? I have never seen anyone propose to evaluate outcomes into 10 CATEGORIES and to use those as criteria for a power analyses... even if power were not lost by having categories, that is asking for too much detail in hypothesizing relative QUANTITIES for 10 varieties of outcome. Do rank-order (as you guessed), or compress your outcomes to just 2 or 3 categories. ************ <> * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Document by Rich Ulrich. E-mail to wpilib+@pitt.edu

FAQ top.

Ulrich home page.

Ulrich FAQ. http://www.pitt.edu/~wpilib/stats99.html