[R-lang] Re: p-values from pvals.fnc

[Daniel Ezra Johnson]

"Unfortunately" this is starting to sound like correct behavior. If subjects behave very differently with respect to x, then the significance of the x and x*y terms will be reduced. My money's on the MCMC.

Dan

On Jul 30, 2011, at 3:30 AM, Jakke Tamminen <jjt379@gmail.com> wrote:

> Many thanks to David and Roger for helpful ideas to explore. Roger: could you please explain how to check whether the Markov chain has converged? 
> 
> Another thing I noticed that might provide a clue is that the strange behaviour of the p-values disappears if I remove the random slope for x. So
> 
> model1 = lmer(RT~x*y+(1+x|Subject)+(1|Item)
> 
> shows the problem while
> 
> model2 = lmer(RT~x*y+(1|Subject)+(1|Item)
> 
> does not. I wonder if that helps?
> 
> Jakke
> 
> 
> On 30 July 2011 07:08, Levy, Roger <rlevy@ucsd.edu> wrote:
> Hi Jakke,
> 
> It's a bit hard to give an answer to this question on the basis of anecdotal reports.  Do you have a specific dataset that gives you this behavior which you could share with the list?  That might be helpful in giving more pinpointed.
> 
> In general, one thing to check for when you find this kind of divergence, though, might be whether the Markov chain from which your "pMCMC" values are computed looks like it has converged.
> 
> Best
> 
> Roger
> 
> 
> On Jul 29, 2011, at 1:58 PM, Jakke Tamminen wrote:
> 
> > Dear R-users,
> >
> > I have been wondering about something with the pvals.fnc function. As we know, the pvals function gives two p-values, one based on the posterior distribution (pMCMC) and one based on the t-distribution. In my experience most of the time the two values are very similar. However, I have recently come across situations where they are wildly different. I have been particularly surprised to see t-values above 2 that have associated pMCMC values that are not even close to significance, while at the same time the t-distribution based p-value is significant. For example, a recent model I worked with looked something like this:
> >
> > model1 = lmer(RT~x*y+(1+x|Subject)+(1|Item)
> >
> > and gave me a t-value of 2.07 for the interaction, with a pMCMC p-value of 0.4756 and a t-distribution p-value of 0.0381. Obviously I like one of these better than the other! I know that the latter p-value is anticonservative, but the magnitude of the discrepancy is nonetheless surprising to me, given the t-value. I'd be very grateful for any advice on how to proceed in cases like this. I'm using lme4 version 0.99875-6.
> >
> > Many thanks,
> >
> > Jakke
> 
> --
> 
> Roger Levy                      Email: rlevy@ucsd.edu
> Assistant Professor             Phone: 858-534-7219
> Department of Linguistics       Fax:   858-534-4789
> UC San Diego                    Web:   http://idiom.ucsd.edu/~rlevy
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucsd.edu/pipermail/ling-r-lang-l/attachments/20110730/a92f4d8b/attachment.html