[R-lang] Re: Comparing effect sizes in lmer

[Levy, Roger]

On Feb 2, 2011, at 5:12 PM, <lngmyers@ccu.edu.tw> wrote:
> On Thu, Feb 3, 2011 8:15 am, Ariel M. Goldberg wrote:
>> Dear R-lang,
>> 
>> 
>> I am interested in comparing the effect sizes of two predictors, A and B.
>> When I standardize them and enter them both into my lmer model, the
>> coefficient of one is twice the size of the other (-9 vs -4.5).  Is there
>> a principled way to determine if A has a "significantly" greater effect?
>> I was thinking that one way might be to compare the increase in
>> loglikelihood over a baseline model but I'm not sure how to do it.  I'm
>> envisioning something along the lines of creating 3 models: baseline,
>> which contains neither of the two predictors, and baseline+A and
>> baseline+B (which contain the baseline model and A and B, respectively),
>> and then comparing how much each of the latter two models increases the
>> loglikelihood over the baseline model.

> I raised this question a few years ago on this list - you can see the
> discussion here:
> 
> http://www.mail-archive.com/r-lang@ling.ucsd.edu/msg00058.html
> 
> I think the "best" answer is here:
> 
> http://www.mail-archive.com/r-lang@ling.ucsd.edu/msg00057.html
> 
> James S. Adelman wrote:
> 
> [begin quote]
> 
> If I've understood your question correctly, you are asking about a linear
> regression model with response, say z, and two predictors x and y:
> 
> K: z = a  +  mx  +  ny  +  error
> 
> and you wish to know whether H0: m=n.  If so, anova(lm(z~x+y),lm(z~I(x+y)))
> should be valid under the usual conditions.

Ariel: I agree that this is a better way to test whether A and B have "significantly" different effects on your response.  The same logic is applicable to lme4 models, too, with the standard caveat that the likelihood-ratio test may in general be slightly anti-conservative for some multilevel models.  (However, the anti-conservativity is generally pretty minimal for models with a small number of parameters compared to the number of observations.)

Best

Roger

--

Roger Levy                      Email: rlevy@ucsd.edu
Assistant Professor             Phone: 858-534-7219
Department of Linguistics       Fax:   858-534-4789
UC San Diego                    Web:   http://idiom.ucsd.edu/~rlevy