[R-lang] Re: Using BIC to calculate the Bayes Factor for model selection

[Henrik Singmann]

Dear Zhenguang,

I think your BF calculation is incorrect. Unfortunately, you haven't given the reference where you took it from.

If you look into Wagenmakers (2007) you will see that he defines it differently.

The BIC approximation to the Bayes Factor he advocates for BF_01 is given by exp( (BIC_1 - BIC_0)/2 ) (see page 796).

Hence, it seems you have to turn around the two numbers and divide them by two before applying the exponential function:

> exp((638.7 - 633.9)/2)
[1] 11.02

I hope this helps,
Henrik

Reference:

Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems ofp values. Psychonomic Bulletin & Review, 14(5), 779–804. doi:10.3758/BF03194105



Am 05.12.2013 18:10, schrieb Zhenguang Cai:
> Hi all,
>
> We have a paper in which we want to conclude that Construction A and Construction B have the same syntactic representation (partly) on the basis that they elicited the same priming behaviours. Now of course the problem with the reviewers is that the conclusion is based on a null effect, which, in terms of traditional statistics, is non-informative.
>
> Thus, I am trying to resolve this issue using the Bayes Factor. The BayesFactor package, however, seems to only work with linear linear models but not with logit models. However, there seems to be an easy way of calculating the Bayes Factor (B12) using BICs from two models:
>
> B12 = e (BIC1 - BIC2)
>
> So the following is what I did.
>
> 1. Run an LME model with intercept only (i.e., no primes) - this is Model 1.
> 2. Add the primes into Model 1 - this is Model 2.
> 3. Get the BIC for both models (633.9 for Model 1 and 638.7 for Model 2)
> 4. Calculate the Bayes Factor: B12 = e (633.9 - 638.7) = e (-4.8) = 0.008
>
> This seems to strongly support Model 2, which is a bit strange, because the prime does not have any main effect at all (and the BICs suggest that Model 1 is preferred to Model 2). Or did I misunderstand/misdo the calculation?
>
> Thank you very much for any help,
>
> Zhenguang Cai
>
>

-- 
Dipl. Psych. Henrik Singmann
PhD Student
Albert-Ludwigs-Universität Freiburg, Germany
http://www.psychologie.uni-freiburg.de/Members/singmann