[R-lang] How to compare mixed logit models with crossed random effects

[Linda Mortensen]

Dear LanguageR users,
 
I'm trying to fit a mixed logit model using the lmer function in the lme4 package. My question concerns the random effects part of this model (i.e., the random effects for my subjects and items) and how I decide between models that differ in the number of random effect terms that are estimated. So far, I have used two procedures: 
1. For a given model, I remove a random effect term if it correlates very strongly with either the intercept or any of the other random effect terms. Eventually, I end up with a model in which all correlations are modest.  
 
2. I compare the quasi-log likelihood (logLik) values of a model with a given random effect term (e.g. an interaction term, ... (1 + a * b | sub) and of a model without that term (... (1 + a + b | sub). If the logLik values are very similar (i.e., if the value is not, or at least not much, smaller for the model without the term than for the model with the term), I go for the former model. 
Is it acceptable to select a model on the basis of this comparison? Or, when the logLik values are similar (which they usually are for my models), should I instead look at the measures of likelihood that take into account the number of parameters in a model when evaluating its fit (i.e., AIC, BIC, deviance)? According to these other measures, a simple model seems always to be better than a more complex one, but if I want to rule out that my fixed effects can be explained, in part, by random effects for subjects and items, then a simple model (with few random effects) is not necessarily better than a complex one, I would think.  
 
>From prior postings on this lists and from other sources, I get the impression that a direct comparison of the likelihoods of two mixed logit models that differ in the random effect part (or in the fixed effect part) using the anova () function is not recommended. Please correct me if I'm wrong in assuming this. 
 
Any advice on how I go about making these model comparisons is much appreciated. 
 
Linda
 
 
Linda Mortensen
Post-doctoral research fellow
Department of Psychology
University of Copenhagen 
Øster Farimagsgade 2A
1353 Copenhagen K
Denmark
Tel.: +45 3532 4889
E-mail: linda.mortensen at psy.ku.dk