[R-lang] Determining whether a model fits a second data set

[Amanda J. Owen]

Hi,
   I think this is a more general statistics question rather than a
particular question about the use of lmer, but I would appreciate the
advice from this group since the question is specific to the types of
models fit under lmer/LanguageR.

I have fit a (rather complex) model to a dataset based on elicited
data. The model is a mixed model logistic regression with 2 random
factors and 6 predictors.  I have both a 'real' model using true
values for the predictors and a standardized model which has
standardized all predictors so that their relative contributions to
predicting the outcome (past tense accuracy) can be compared more
directly.  Some factors are significant but make a relatively small
contribution.  The data set is fairly rich with around 8000 datapoints
(216 past tense opportunities per child).

    I have a similar but smaller data set from spontaneous data from
the same children and a third data set from spontaneous data from a
new group of children that is even smaller.  I can quantify the
predictor variables (lexical frequency, phonotactics, etc) in such a
way that I should be able to fit the same model to this second
dataset.  I'm especially interested in whether we could actually use
the same beta values to have a reasonably good fit because I'd like to
be able to comment on whether the relative contributions of the
predictors was an artifact of the data set, the particular children
involved,  or are true more generally.

   I'm not sure how to assess model fit without simply comparing a
reduced/full model on the same dataset.  I think if I was using linear
regression I would compare R-squared values across the two datasets
and have some comment about the fact that the models explain similar
amounts of variance.  But I'm stumped when it comes to logistic
regression/the introduction of random effects.

Thanks so much for any advice or help you can provide.
Amanda


Amanda J. Owen
ajowen@gmail.com


amanda-owen@uiowa.edu
Dept of Speech Pathology & Audiology
University of Iowa
Iowa City, IA