[R-lang] Re: p-values from pvals.fnc

[Levy, Roger]

Hi Jakke,

It's a bit hard to give an answer to this question on the basis of anecdotal reports.  Do you have a specific dataset that gives you this behavior which you could share with the list?  That might be helpful in giving more pinpointed.

In general, one thing to check for when you find this kind of divergence, though, might be whether the Markov chain from which your "pMCMC" values are computed looks like it has converged.

Best

Roger


On Jul 29, 2011, at 1:58 PM, Jakke Tamminen wrote:

> Dear R-users,
> 
> I have been wondering about something with the pvals.fnc function. As we know, the pvals function gives two p-values, one based on the posterior distribution (pMCMC) and one based on the t-distribution. In my experience most of the time the two values are very similar. However, I have recently come across situations where they are wildly different. I have been particularly surprised to see t-values above 2 that have associated pMCMC values that are not even close to significance, while at the same time the t-distribution based p-value is significant. For example, a recent model I worked with looked something like this:
> 
> model1 = lmer(RT~x*y+(1+x|Subject)+(1|Item)
> 
> and gave me a t-value of 2.07 for the interaction, with a pMCMC p-value of 0.4756 and a t-distribution p-value of 0.0381. Obviously I like one of these better than the other! I know that the latter p-value is anticonservative, but the magnitude of the discrepancy is nonetheless surprising to me, given the t-value. I'd be very grateful for any advice on how to proceed in cases like this. I'm using lme4 version 0.99875-6.
> 
> Many thanks,
> 
> Jakke

--

Roger Levy                      Email: rlevy@ucsd.edu
Assistant Professor             Phone: 858-534-7219
Department of Linguistics       Fax:   858-534-4789
UC San Diego                    Web:   http://idiom.ucsd.edu/~rlevy