[R-lang] Re: Effect size in linear mixed effects models

[Zhenguang Cai]

Hi Florian, Roger, and Scott,

Thank you very much for all the suggestions. I submitted the paper based 
on Florian's suggestion so read Roger's and Scott's suggestions later.

Basically, I have a categorical variable (Stimulus: Long vs. Short) and 
a continuous one (Duration) (in a balanced design though, i.e., every 
stimulus combines with every duration). I then did the following 
transformations.

E1$csStimulus <- scale (as.numeric(E1$Stimulus), center = TRUE, scale = 
TRUE)
E1$csDuration <- scale (E1$Duration, center = TRUE, scale = TRUE)

And then used these transformed predictors in an LME model, with RTs as 
the dependent variable. It seems a bit different from what Roger 
suggests (using 0.5 and -0.5 for categorical variable).

Best,
Zhenguang



? 2013/7/18 23:51, Scott Jackson ??:
> Just to add my two cents to Roger's good suggestions:
>
> There are two senses in which people talk about "effect sizes." 
> Strictly speaking, any mention of the size of an effect in either 
> meaningful units (like RT change per change in word frequency or 
> something like that) or in standardized units (like effect on RT per 
> standard deviation of your predictor) is a kind of effect size.  I 
> think your suggestion about standardized coefficients and Roger's 
> other suggestions are all good. Personally, if I report standardized 
> or transformed effects (like logits or log RT or something), I like to 
> also give an example in the scale of the observed data, in units that 
> are more easily accessible/meaningful to readers, but that may just be 
> a presentation preference.
>
> The other sense is that some people are used to certain kinds of 
> stats, and may expect to see a specialized effect size stat like 
> Cohen's d or eta-squared or R-squared or something. AFAIK, there's not 
> a "standard" effect size stat for LMEs, and many of the standards from 
> other methods don't really apply, or it's not clear how to best apply 
> them.  For example, the standard Cohen's d is the difference between 
> two groups, divided by the standard deviation.  You could still apply 
> this to your data (if you're talking about group differences), but 
> it's not clear how to best parcel up the variance since you're doing 
> an LME. It looks like there may be some suggestions from the HLM 
> literature (see here: http://rmcs.buu.ac.th/statcenter/HLM.pdf for an 
> example), but I'm *really* not clear how this would apply to LMEs 
> where you have crossed random effects (like subject and item), which 
> is usually the case for us psycholinguists.  So I suppose if you are 
> doing group comparisons (like an effect between conditions) and 
> someone really twists your arm, you could standardize the outcome 
> variable as well, so the interpretation of your coefficient would be 
> "a one-unit change in my predictor results in a change of (coef) 
> standard deviations in the outcome", and that's more or less the 
> interpretation of a Cohen's d.
>
> hope that helps,
> -scott
>
>
> On Thu, Jul 18, 2013 at 6:02 PM, Levy, Roger <rlevy@ucsd.edu 
> <mailto:rlevy@ucsd.edu>> wrote:
>
>     Hi Zhenguang,
>
>     I am not familiar with exactly what JEP:G asks for, but if you're
>     dealing with continuous predictors, I suggest you keep in mind
>     that in general such predictors have units (e.g., milliseconds for
>     time; # characters for word length; log parts per million for word
>     frequency; bits, nats, or bans for log probability).  If your
>     predictor has units of type S and your response variable has type
>     T, then you could say, for example, "effect size: 10 T/S".  If you
>     are doing a mixed logit model, then the response unit is the
>     logit, so you could say, e.g., "effect size: 2 logits per S".
>
>     If you're talking about a categorical predictor, then you want to
>     standardize the contrast to be size 1.  So, for example, code a
>     2-level predictor as (-0.5,0.5) and then report the parameter
>     estimate associated with the predictor.  Or you could code it as
>     (-1,1) and report the parameter estimate divided by two.
>
>     Best & hope this helps,
>
>     Roger
>
>     On Jul 17, 2013, at 12:01 PM, Zhenguang Cai
>     <zhenguangcai@gmail.com <mailto:zhenguangcai@gmail.com>> wrote:
>
>     > Hi,
>     >
>     > Some journals (JEP:G) requires effect sizes in the data
>     analysis. I wonder how to do this in LME. Can I just scale the
>     predictors (z-scores, if I understand it correctly) and then use
>     the coefficient as a measure of effect size? Or is there a more
>     standard way to do it?
>     >
>     > Thanks,
>     > Zhenguang
>     >
>     > --
>     > Zhenguang G. Cai
>     >
>     > Research Fellow
>     > Institute of Cogntion/ School of Psychology
>     > University of Plymouth
>     > Portland Square, Drake Circus, Plymouth, PL4 8AA
>     >
>     >
>     > https://sites.google.com/site/zhenguangcai/
>
>
>

-- 
Zhenguang G. Cai

Research Fellow
Institute of Cogntion/ School of Psychology
University of Plymouth
Portland Square, Drake Circus, Plymouth, PL4 8AA

https://sites.google.com/site/zhenguangcai/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucsd.edu/pipermail/ling-r-lang-l/attachments/20130719/2392f5f2/attachment-0001.html