[R-lang] Weird outcome logit mixed-effect model

[Roger Levy]

A clarification on my response below: I failed to note that I had
defined the factor Info1 as

  dat$Info1 <- factor(ifelse(dat$Info %in%
"accessible","accessible","givenOrNew"))

Best to all,

Roger

> The next question, of course, is how to deal with the dataset you
> actually have.  If you weren't using a mixed-effects model, you could
> use a likelihood-ratio test by comparing your full model with a simpler
> model; the likelihood-ratio test isn't susceptible to problems with
> large parameter estimates the way the Wald test is.  For example:
> 
>> deacc.glm = glm (Deacc ~ Info, data = dat, family = "binomial")
>> deacc.glm1 = glm (Deacc ~ Info1, data = dat, family = "binomial")
>> anova(deacc.glm,deacc.glm1,test="Chisq")
> Analysis of Deviance Table
> 
> Model 1: Deacc ~ Info
> Model 2: Deacc ~ Info1
>   Resid. Df Resid. Dev Df Deviance P(>|Chi|)
> 1        97     52.452
> 2        98     71.600 -1  -19.148 1.210e-05
> 
> In principle, you could do this with your mixed-effects model, being
> sure to use ML instead of REML fitting:
> 
>> deacc.lmer = lmer (Deacc ~ Info + (1 | Subject), data = dat, family =
> "binomial",REML=F)
>> deacc.lmer1 = lmer (Deacc ~ Info1 + (1 | Subject), data = dat, family
> = "binomial",REML=F)
>> anova(deacc.lmer,deacc.lmer1)
> Data: dat
> Models:
> deacc.lmer1: Deacc ~ Info1 + (1 | Subject)
> deacc.lmer: Deacc ~ Info + (1 | Subject)
>             Df     AIC     BIC  logLik Chisq Chi Df Pr(>Chisq)
> deacc.lmer1  3  77.600  85.416 -35.800
> deacc.lmer   4  60.400  70.821 -26.200  19.2      1  1.177e-05 ***
> 
> There is an argument that the likelihood-ratio test is anti-conservative
> and hence inappropriate for comparing mixed-effects models differing
> only in fixed-effects structure.  (See Pinheiro & Bates, 2000, around
> page 76 or so.  The argument is made only for linear mixed-effects
> models and I'm not sure of the status for logit mixed-effects models.)
> That being said, it's not clear how strong the anti-conservativity is,
> and in your case it seems like you have such an exceedingly powerful
> effect that you might be safe in using the likelihood-ratio test here
> and just mentioning the potential anti-conservativity as a caveat.
> 
> So the summary is: believe it or not, how to assess the significance of
> the parameter estimate for "new" for your model & dataset is a bit of an
> open question, but it seems pretty clear that the estimate is
> significantly non-zero.
> 
> Best & hope this helps.
> 
> Roger
> 
> 
> 


-- 

Roger Levy                      Email: rlevy at ling.ucsd.edu
Assistant Professor             Phone: 858-534-7219
Department of Linguistics       Fax:   858-534-4789
UC San Diego                    Web:   http://ling.ucsd.edu/~rlevy