[R-lang] Re: Effect size in linear mixed effects models

[T. Florian Jaeger]

Dear Zhenguang,

I think Gelman, e.g., in this 2008 article, recommends standarizing
continuous predictors by dividing them by twice their SE.


@article{gelman2008scaling,
  title={Scaling regression inputs by dividing by two standard deviations},
  author={Gelman, Andrew},
  journal={Statistics in medicine},
  volume={27},
  number={15},
  pages={2865--2873},
  year={2008},
  publisher={Wiley Online Library}
}

Here's the relevant part from their paper:

   - Here we propose dividing each numeric variable by two times its
standard deviation, so that the generic comparison is with inputs
equal to the mean ±1 standard deviation. The resulting coefficients are
then directly comparable for untransformed binary predictors. We have
implemented the procedure as a function in R. [Florian: under the
assumption that the categorical binary contrasts have a distance of 1
-- as is the case sum-coding that Roger suggested]

Gelman also discusses rescaling of categorical predictors as an
additional option. have a look at the article and see the function
*standardize()* in the R package *arm. *Note also Gelman's comments
about transformations and standardization towards the end of his
article.

Florian



On Fri, Jul 19, 2013 at 3:14 AM, Zhenguang Cai <zhenguangcai@gmail.com>wrote:

>  Hi Florian, Roger, and Scott,
>
> Thank you very much for all the suggestions. I submitted the paper based
> on Florian's suggestion so read Roger's and Scott's suggestions later.
>
> Basically, I have a categorical variable (Stimulus: Long vs. Short) and a
> continuous one (Duration) (in a balanced design though, i.e., every
> stimulus combines with every duration). I then did the following
> transformations.
>
> E1$csStimulus <- scale (as.numeric(E1$Stimulus), center = TRUE, scale =
> TRUE)
> E1$csDuration <- scale (E1$Duration, center = TRUE, scale = TRUE)
>
> And then used these transformed predictors in an LME model, with RTs as
> the dependent variable. It seems a bit different from what Roger suggests
> (using 0.5 and -0.5 for categorical variable).
>
> Best,
> Zhenguang
>
>
>
>  于 2013/7/18 23:51, Scott Jackson 写道:
>
>   Just to add my two cents to Roger's good suggestions:
>
>  There are two senses in which people talk about "effect sizes." Strictly
> speaking, any mention of the size of an effect in either meaningful units
> (like RT change per change in word frequency or something like that) or in
> standardized units (like effect on RT per standard deviation of your
> predictor) is a kind of effect size.  I think your suggestion about
> standardized coefficients and Roger's other suggestions are all good.
> Personally, if I report standardized or transformed effects (like logits or
> log RT or something), I like to also give an example in the scale of the
> observed data, in units that are more easily accessible/meaningful to
> readers, but that may just be a presentation preference.
>
>  The other sense is that some people are used to certain kinds of stats,
> and may expect to see a specialized effect size stat like Cohen's d or
> eta-squared or R-squared or something. AFAIK, there's not a "standard"
> effect size stat for LMEs, and many of the standards from other methods
> don't really apply, or it's not clear how to best apply them.  For example,
> the standard Cohen's d is the difference between two groups, divided by the
> standard deviation.  You could still apply this to your data (if you're
> talking about group differences), but it's not clear how to best parcel up
> the variance since you're doing an LME. It looks like there may be some
> suggestions from the HLM literature (see here:
> http://rmcs.buu.ac.th/statcenter/HLM.pdf for an example), but I'm
> *really* not clear how this would apply to LMEs where you have crossed
> random effects (like subject and item), which is usually the case for us
> psycholinguists.  So I suppose if you are doing group comparisons (like an
> effect between conditions) and someone really twists your arm, you could
> standardize the outcome variable as well, so the interpretation of your
> coefficient would be "a one-unit change in my predictor results in a change
> of (coef) standard deviations in the outcome", and that's more or less the
> interpretation of a Cohen's d.
>
>  hope that helps,
> -scott
>
>
> On Thu, Jul 18, 2013 at 6:02 PM, Levy, Roger <rlevy@ucsd.edu> wrote:
>
>> Hi Zhenguang,
>>
>> I am not familiar with exactly what JEP:G asks for, but if you're dealing
>> with continuous predictors, I suggest you keep in mind that in general such
>> predictors have units (e.g., milliseconds for time; # characters for word
>> length; log parts per million for word frequency; bits, nats, or bans for
>> log probability).  If your predictor has units of type S and your response
>> variable has type T, then you could say, for example, "effect size: 10
>> T/S".  If you are doing a mixed logit model, then the response unit is the
>> logit, so you could say, e.g., "effect size: 2 logits per S".
>>
>> If you're talking about a categorical predictor, then you want to
>> standardize the contrast to be size 1.  So, for example, code a 2-level
>> predictor as (-0.5,0.5) and then report the parameter estimate associated
>> with the predictor.  Or you could code it as (-1,1) and report the
>> parameter estimate divided by two.
>>
>> Best & hope this helps,
>>
>> Roger
>>
>> On Jul 17, 2013, at 12:01 PM, Zhenguang Cai <zhenguangcai@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > Some journals (JEP:G) requires effect sizes in the data analysis. I
>> wonder how to do this in LME. Can I just scale the predictors (z-scores, if
>> I understand it correctly) and then use the coefficient as a measure of
>> effect size? Or is there a more standard way to do it?
>> >
>> > Thanks,
>> > Zhenguang
>> >
>> > --
>> > Zhenguang G. Cai
>> >
>> > Research Fellow
>> > Institute of Cogntion/ School of Psychology
>> > University of Plymouth
>> > Portland Square, Drake Circus, Plymouth, PL4 8AA
>> >
>> >
>> > https://sites.google.com/site/zhenguangcai/
>>
>>
>>
>
> --
> Zhenguang G. Cai
>
> Research Fellow
> Institute of Cogntion/ School of Psychology
> University of Plymouth
> Portland Square, Drake Circus, Plymouth, PL4 8AA
> https://sites.google.com/site/zhenguangcai/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucsd.edu/pipermail/ling-r-lang-l/attachments/20130720/7fe26274/attachment.html