[R-lang] Comparing PQL fits for logistic regression models

[David Reitter]

H Roger and Florian,

thanks for your replies - I wanted to follow up on a few things:

On 10 Oct 2007, at 18:39, Roger Levy wrote:

> 1) you can always use cross-validation.

T. Florian Jaeger wrote:

> predictive power? you can, of course, use predict() [you have to  
> write your own version if you want to use the random effects]. and  
> then use cross-validation to get a measure for unseen data.

Sure, and then use a non-parametric test to compare the two  
accuracies, or can we even assume (Central Limit Theorem) that the  
means obtained through cross-validation are approximately normally  
distributed?

Regarding log-likelihood and BIC, I also found a paper by Song & Lee:

Model comparison of generalized linear mixed models
Xin-Yuan Song, Sik-Yum Lee
Statistics in Medicine 25 (10), pp 1685-1698, 2005
http://www3.interscience.wiley.com/cgi-bin/abstract/112100165/ABSTRACT

.. which I'll have to look at.

Thanks for your hint regarding quasi-likelihood and Laplace fits. I  
might go with that for now.

On 10 Oct 2007, at 18:39, Roger Levy wrote:

> 2) I've recently become aware of the "Vuong test" (Vuong 1989,  
> Econometrika):
>
> http://links.jstor.org/sici?sici=0012-9682(198903)57:2%3C307:LRTFMS% 
> 3E2.0.CO;2-J
>
> but I haven't had a chance to learn what it is.  It might be  
> useful. Would love to hear what you learn if you have a chance to  
> look into it.

Vuong uses the Log-Likelihood Ratio to compare two non-nested models  
and discusses cases where models do not and do overlap partially. I  
note that one problematic assumption (A.1) in this paper is that all  
samples are i.i.d., which excludes mixed effects models with a  
grouping factor. So time-series and repeated measures experiments (by- 
subject and by-item analyses etc.) are not covered.


--
David Reitter
ICCS/HCRC, Informatics, University of Edinburgh
http://www.david-reitter.com




-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2450 bytes
Desc: not available
Url : http://pidgin.ucsd.edu/pipermail/r-lang/attachments/20071022/aec1ed7a/attachment.bin