[R-lang] Re: Effect size in linear mixed effects models

[Scott Jackson]

Just to add my two cents to Roger's good suggestions:

There are two senses in which people talk about "effect sizes." Strictly
speaking, any mention of the size of an effect in either meaningful units
(like RT change per change in word frequency or something like that) or in
standardized units (like effect on RT per standard deviation of your
predictor) is a kind of effect size.  I think your suggestion about
standardized coefficients and Roger's other suggestions are all good.
Personally, if I report standardized or transformed effects (like logits or
log RT or something), I like to also give an example in the scale of the
observed data, in units that are more easily accessible/meaningful to
readers, but that may just be a presentation preference.

The other sense is that some people are used to certain kinds of stats, and
may expect to see a specialized effect size stat like Cohen's d or
eta-squared or R-squared or something. AFAIK, there's not a "standard"
effect size stat for LMEs, and many of the standards from other methods
don't really apply, or it's not clear how to best apply them.  For example,
the standard Cohen's d is the difference between two groups, divided by the
standard deviation.  You could still apply this to your data (if you're
talking about group differences), but it's not clear how to best parcel up
the variance since you're doing an LME. It looks like there may be some
suggestions from the HLM literature (see here:
http://rmcs.buu.ac.th/statcenter/HLM.pdf for an example), but I'm *really*
not clear how this would apply to LMEs where you have crossed random
effects (like subject and item), which is usually the case for us
psycholinguists.  So I suppose if you are doing group comparisons (like an
effect between conditions) and someone really twists your arm, you could
standardize the outcome variable as well, so the interpretation of your
coefficient would be "a one-unit change in my predictor results in a change
of (coef) standard deviations in the outcome", and that's more or less the
interpretation of a Cohen's d.

hope that helps,
-scott


On Thu, Jul 18, 2013 at 6:02 PM, Levy, Roger <rlevy@ucsd.edu> wrote:

> Hi Zhenguang,
>
> I am not familiar with exactly what JEP:G asks for, but if you're dealing
> with continuous predictors, I suggest you keep in mind that in general such
> predictors have units (e.g., milliseconds for time; # characters for word
> length; log parts per million for word frequency; bits, nats, or bans for
> log probability).  If your predictor has units of type S and your response
> variable has type T, then you could say, for example, "effect size: 10
> T/S".  If you are doing a mixed logit model, then the response unit is the
> logit, so you could say, e.g., "effect size: 2 logits per S".
>
> If you're talking about a categorical predictor, then you want to
> standardize the contrast to be size 1.  So, for example, code a 2-level
> predictor as (-0.5,0.5) and then report the parameter estimate associated
> with the predictor.  Or you could code it as (-1,1) and report the
> parameter estimate divided by two.
>
> Best & hope this helps,
>
> Roger
>
> On Jul 17, 2013, at 12:01 PM, Zhenguang Cai <zhenguangcai@gmail.com>
> wrote:
>
> > Hi,
> >
> > Some journals (JEP:G) requires effect sizes in the data analysis. I
> wonder how to do this in LME. Can I just scale the predictors (z-scores, if
> I understand it correctly) and then use the coefficient as a measure of
> effect size? Or is there a more standard way to do it?
> >
> > Thanks,
> > Zhenguang
> >
> > --
> > Zhenguang G. Cai
> >
> > Research Fellow
> > Institute of Cogntion/ School of Psychology
> > University of Plymouth
> > Portland Square, Drake Circus, Plymouth, PL4 8AA
> >
> >
> > https://sites.google.com/site/zhenguangcai/
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucsd.edu/pipermail/ling-r-lang-l/attachments/20130718/f31a5b99/attachment.html