[R-lang] Using BIC to calculate the Bayes Factor for model selection

[Zhenguang Cai]

Hi all,

We have a paper in which we want to conclude that Construction A and 
Construction B have the same syntactic representation (partly) on the 
basis that they elicited the same priming behaviours. Now of course the 
problem with the reviewers is that the conclusion is based on a null 
effect, which, in terms of traditional statistics, is non-informative.

Thus, I am trying to resolve this issue using the Bayes Factor. The 
BayesFactor package, however, seems to only work with linear linear 
models but not with logit models. However, there seems to be an easy way 
of calculating the Bayes Factor (B12) using BICs from two models:

B12 = e (BIC1 - BIC2)

So the following is what I did.

1. Run an LME model with intercept only (i.e., no primes) - this is Model 1.
2. Add the primes into Model 1 - this is Model 2.
3. Get the BIC for both models (633.9 for Model 1 and 638.7 for Model 2)
4. Calculate the Bayes Factor: B12 = e (633.9 - 638.7) = e (-4.8) = 0.008

This seems to strongly support Model 2, which is a bit strange, because 
the prime does not have any main effect at all (and the BICs suggest 
that Model 1 is preferred to Model 2). Or did I misunderstand/misdo the 
calculation?

Thank you very much for any help,

Zhenguang Cai


-- 
Zhenguang Cai

Psychology, University of Plymouth

https://sites.google.com/site/zhenguangcai/