<br><br><div class="gmail_quote">On Thu, Jul 30, 2009 at 5:59 AM, Jakke Tamminen <span dir="ltr"><<a href="mailto:j.tamminen@psych.york.ac.uk">j.tamminen@psych.york.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<div dir="ltr" align="left"><font size="2" face="Arial"><span>My thanks to Andy, James, and Florian for their
responses to my question. The replies were, as always, prompt, helpful, and
lucid. I have a couple of quick further questions about model comparison: I
think all three replies included suggestions of doing likelihood ratio
tests to assess the significance of a single fixed factor in the model. How
reliable is this? As far as I can recall, Baayen in his book and in the JML
paper only uses this to evaluate random factors, and the paper by Bolker et al
that Andy cited recommends against it in the case of fixed factors. Are there
good alternatives? </span></font></div></div></blockquote><div><br>It's still being debated what's best to be done there, but I think it's a valid alternative for now and especially so for simple models.<br> <br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div dir="ltr" align="left"><font size="2" face="Arial"><span></span></font></div>
<div dir="ltr" align="left"><font size="2" face="Arial"><span></span></font> </div>
<div dir="ltr" align="left"><font size="2" face="Arial"><span>Finally, a quick follow up question regarding Florian's
six-step procedure, reproduced below. In step 5 you suggest I interpret the
coefficients in the full _or_ the reduced model. So is it acceptable to look at
the coefficients of a factor or an interaction even if the factor or interaction
does not "survive" a likelihood ratio test, i.e. does not significantly
contribute to the fit of the model?</span></font></div></div></blockquote><div><br>I would usually leave non-significant predictors in the model <i>if they are theoretically motivated </i>(which is why they should be why you put them in there to begin with ;)). There are many different traditions and approaches, but I feel that, if you have enough data to avoid overfitting or other problems, you should leave even relatively insignificant predictors into the model (p>.7 [sic] is often given as a removal threshold).<br>
<br>HTH,<br>Florian<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div dir="ltr" align="left"><font size="2" face="Arial"><span></span></font></div>
<div dir="ltr" align="left"><font size="2" face="Arial"><span></span></font> </div>
<div dir="ltr" align="left"><font size="2" face="Arial"><span>I
hope that makes sense, thank you again for all the help!</span></font></div>
<div dir="ltr" align="left"><font size="2" face="Arial"><span></span></font> </div><font color="#888888">
<div dir="ltr" align="left"><font size="2" face="Arial"><span>Jakke</span></font></div></font><div class="im"><br>
<blockquote style="border-left: 2px solid rgb(0, 0, 0); padding-left: 5px; margin-left: 5px; margin-right: 0px;">
<div class="gmail_quote"><font size="2" face="Arial"></font>
<div><br>1) l <- lmer(logRT~A*B+(1+A*B|Subject)+(1+A*B| Item), data) <br>2)
follow the procedure outline on our lab blog to figure out which random
effects you need: <a href="http://hlplab.wordpress.com/2009/05/14/random-effect-should-i-stay-or-should-i-go/" target="_blank">http://hlplab.wordpress.com/2009/05/14/random-effect-should-i-stay-or-should-i-go/</a><br>3)
take the resulting model and compare it against a model without the
interaction, using anova(l, l.woInteraction).<br>4) <i>if removal of the
interaction is not significant</i>, you could further compare the model
against a model with only A (see above).<br>5) Interpret coefficients in the
full model or in the reduced model (I would do the former unless I don't have
much data or cannot reduce collinearity, but you may prefer the latter).<br>6)
If you find any of the scripts of references given above useful, cite/refer to
them, so that others can find them ;)<span><font size="2" face="Arial"> </font></span><span> </span><span> </span></div></div></blockquote></div></div>
<br>_______________________________________________<br>
R-lang mailing list<br>
<a href="mailto:R-lang@ling.ucsd.edu">R-lang@ling.ucsd.edu</a><br>
<a href="http://pidgin.ucsd.edu/mailman/listinfo/r-lang" target="_blank">http://pidgin.ucsd.edu/mailman/listinfo/r-lang</a><br>
<br></blockquote></div><br>