[R-lang] Re: p-values from pvals.fnc

[Jakke Tamminen]

Roger: Thanks for the information, I guess I have a lot of reading to do!

Alex and Roger: Looks like my version of lme4 is pretty old, 0.99875-6. If
the more recent versions don't give you p-values for models with random
slopes, should I be looking at them (the p-values) at all, or rely on the
t-statistic (and probably update my packages!)?

Jakke

On 30 July 2011 18:45, Alex Fine <afine@bcs.rochester.edu> wrote:

> Jakke,
>
> I'm probably missing something, so I'm not replying-all.  How do you even
> get pvals.fnc() to work with a model that has random slopes?  I have the
> most up-to-date version of the languageR package and it won't take models
> that have anything other than random intercepts.
>
> thanks,
> Alex
>
> Jakke Tamminen wrote:
>
>> Many thanks to David and Roger for helpful ideas to explore. Roger: could
>> you please explain how to check whether the Markov chain has converged?
>> Another thing I noticed that might provide a clue is that the strange
>> behaviour of the p-values disappears if I remove the random slope for x. So
>>
>> model1 = lmer(RT~x*y+(1+x|Subject)+(1|**Item)
>>
>> shows the problem while
>>
>> model2 = lmer(RT~x*y+(1|Subject)+(1|**Item)
>>
>> does not. I wonder if that helps?
>>
>> Jakke
>>
>>
>> On 30 July 2011 07:08, Levy, Roger <rlevy@ucsd.edu <mailto:rlevy@ucsd.edu>>
>> wrote:
>>
>>    Hi Jakke,
>>
>>    It's a bit hard to give an answer to this question on the basis of
>>    anecdotal reports.  Do you have a specific dataset that gives you
>>    this behavior which you could share with the list?  That might be
>>    helpful in giving more pinpointed.
>>
>>    In general, one thing to check for when you find this kind of
>>    divergence, though, might be whether the Markov chain from which
>>    your "pMCMC" values are computed looks like it has converged.
>>
>>    Best
>>
>>    Roger
>>
>>
>>    On Jul 29, 2011, at 1:58 PM, Jakke Tamminen wrote:
>>
>>    > Dear R-users,
>>    >
>>    > I have been wondering about something with the pvals.fnc
>>    function. As we know, the pvals function gives two p-values, one
>>    based on the posterior distribution (pMCMC) and one based on the
>>    t-distribution. In my experience most of the time the two values
>>    are very similar. However, I have recently come across situations
>>    where they are wildly different. I have been particularly
>>    surprised to see t-values above 2 that have associated pMCMC
>>    values that are not even close to significance, while at the same
>>    time the t-distribution based p-value is significant. For example,
>>    a recent model I worked with looked something like this:
>>    >
>>    > model1 = lmer(RT~x*y+(1+x|Subject)+(1|**Item)
>>    >
>>    > and gave me a t-value of 2.07 for the interaction, with a pMCMC
>>    p-value of 0.4756 and a t-distribution p-value of 0.0381.
>>    Obviously I like one of these better than the other! I know that
>>    the latter p-value is anticonservative, but the magnitude of the
>>    discrepancy is nonetheless surprising to me, given the t-value.
>>    I'd be very grateful for any advice on how to proceed in cases
>>    like this. I'm using lme4 version 0.99875-6.
>>    >
>>    > Many thanks,
>>    >
>>    > Jakke
>>
>>    --
>>
>>    Roger Levy                      Email: rlevy@ucsd.edu
>>    <mailto:rlevy@ucsd.edu>
>>
>>    Assistant Professor             Phone: 858-534-7219
>>    Department of Linguistics       Fax:   858-534-4789
>>    UC San Diego                    Web:      http://idiom.ucsd.edu/~rlevy<
>> http://idiom.ucsd.edu/%**7Erlevy <http://idiom.ucsd.edu/%7Erlevy>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucsd.edu/pipermail/ling-r-lang-l/attachments/20110730/9a34d31c/attachment.html