[R-lang] Re: p-values from pvals.fnc

[Nathaniel Smith]

On Sun, Jul 31, 2011 at 4:24 PM,  <kliegl@uni-potsdam.de> wrote:
> Your last paragraph (What do we gain by the exercise?) nicely summarizes
> what motivated my question about the generalization. Remember this thread
> got started because in the current lme4 implementation mcmcsamp() [or
> wrappers like HPDinterval(), pvals.fnc() and friends] do not work for models
> with random correlation parameters. Psycholinguists and psychologists would
> like to use MCMC to get CIs primarily for the fixed effect estimates.

Right. The problem is, essentially, that mcmcsamp() does not know how
to resample the correlation parameter.

> Jon's proposal was to reparameterize the models in a way that the
> correlation parameter is zero. Then, we can use mcmcsamp() to get pvalues
> and reviewers/editors will be happy. I am pretty sure that it is only a
> reparameterization because logLIK, deviance, and REMLdev do not change, they
> are the same for the three models in the illustration. Therefore, in the
> simple varying intercept/varying slope model, I think you get usable MCMC
> statistics for the fixed effect of slope with this trick, because the offset
> on X does not change the interpretation of the slope. I do not see anything
> anti-conservative here. Am I missing something?

What I *think* you may be missing is that this reparametrization
depends on the data. So, you could apply the same reparametrization to
other data, and the logLik, deviance and REMLdev would be the same as
if you didn't reparametrize. But on this new data, the
reparametrization will probably not produce a zero correlation
coefficient -- you'd have to calculate the right reparametrization for
that new data, and it would be different.

So the point is, there is some uncertainty about the right
reparametrization to use. To get proper p-values, mcmcsamp() should
resample the reparametrization, but it doesn't know how to do that
either.

If we are willing to assume that our estimates for the random effect
parameters are correct, then it's easy to get exact p-values without
any MCMC or anything. The whole reason we need MCMC is that there is
noise in our estimates of random effect parameters, and this
reparametrization technique doesn't take that noise into account.
IIUC.

-- Nathaniel