[R-lang] Re: Comparing effect sizes in lmer

[Ariel M. Goldberg]

Thank you both so much for your advice.  This will be very helpful!

AG

On Feb 2, 2011, at 8:52 PM, Levy, Roger wrote:

> 
> On Feb 2, 2011, at 5:12 PM, <lngmyers@ccu.edu.tw> wrote:
>> On Thu, Feb 3, 2011 8:15 am, Ariel M. Goldberg wrote:
>>> Dear R-lang,
>>> 
>>> 
>>> I am interested in comparing the effect sizes of two predictors, A and B.
>>> When I standardize them and enter them both into my lmer model, the
>>> coefficient of one is twice the size of the other (-9 vs -4.5).  Is there
>>> a principled way to determine if A has a "significantly" greater effect?
>>> I was thinking that one way might be to compare the increase in
>>> loglikelihood over a baseline model but I'm not sure how to do it.  I'm
>>> envisioning something along the lines of creating 3 models: baseline,
>>> which contains neither of the two predictors, and baseline+A and
>>> baseline+B (which contain the baseline model and A and B, respectively),
>>> and then comparing how much each of the latter two models increases the
>>> loglikelihood over the baseline model.
> 
>> I raised this question a few years ago on this list - you can see the
>> discussion here:
>> 
>> http://www.mail-archive.com/r-lang@ling.ucsd.edu/msg00058.html
>> 
>> I think the "best" answer is here:
>> 
>> http://www.mail-archive.com/r-lang@ling.ucsd.edu/msg00057.html
>> 
>> James S. Adelman wrote:
>> 
>> [begin quote]
>> 
>> If I've understood your question correctly, you are asking about a linear
>> regression model with response, say z, and two predictors x and y:
>> 
>> K: z = a  +  mx  +  ny  +  error
>> 
>> and you wish to know whether H0: m=n.  If so, anova(lm(z~x+y),lm(z~I(x+y)))
>> should be valid under the usual conditions.
> 
> Ariel: I agree that this is a better way to test whether A and B have "significantly" different effects on your response.  The same logic is applicable to lme4 models, too, with the standard caveat that the likelihood-ratio test may in general be slightly anti-conservative for some multilevel models.  (However, the anti-conservativity is generally pretty minimal for models with a small number of parameters compared to the number of observations.)
> 
> Best
> 
> Roger
> 
> --
> 
> Roger Levy                      Email: rlevy@ucsd.edu
> Assistant Professor             Phone: 858-534-7219
> Department of Linguistics       Fax:   858-534-4789
> UC San Diego                    Web:   http://idiom.ucsd.edu/~rlevy
> 
> 
> 
> 
> 
> 
> 
> 
> 

--
Ariel M. Goldberg
Assistant Professor
Tufts University
Department of Psychology
490 Boston Avenue
Medford, MA 02155

(617) 627-3525
http://ase.tufts.edu/psychology/psycholinglab/