[R-lang] Re: p-values for mixed effects models with random slopes

[Nathaniel Smith]

On Mon, Jan 17, 2011 at 9:19 PM, Steven Piantadosi <piantado@mit.edu> wrote:
> Unless I'm out of date, p values are broken on glmer too? I wonder if an
> easy solution to these two problems might be to implement a
> bootstrapping/resampling algorithm on mixed effect regressions. Does
> anyone know about this--would it be conservative or anticonservative or
> a problem on data sets of typical size in psycholinguistics?
>
> If this is actually a good idea, and someone could point me to a
> reference on how bootstrapping would work on such models (I know
> references for simple non-mixed effect regressions, but not how
> bootstrapping interfaces with repeated subject/item measurements and
> random effects), I'd be happy to try to put some friendly code
> together.

The problem, as you say, is that you need your resampling procedure to
somehow respect the structure you have in your data. If you just have
one random effect (e.g., subjects), then things are relatively
straightforward -- see Davison and Hinkley, section 3.8. They
recommend just resampling subjects, which is slightly
anti-conservative, but still closer to correct than the "natural"
approach of first resampling subjects, and then resampling the cases
within each resampled subject, which turns out to be rather
conservative.

If you have crossed random effects -- which I guess we psycholinguists
always do or we wouldn't be using lmer in the first place -- then
things are trickier and I don't know what the best approach is.
Probably some simulations and things are needed. A quick google finds
a presentation on the 'merBoot' package, but I'm not sure it was ever
released...

-- Nathaniel