[R-lang] Re: Investigating random slope variance

[Scott Jackson]

Glad my mistaken intuitions have lead to such excellent clarifications :-)

Offline, Dan and I played a bit with some simulations, and saw empirical
confirmation of the great quote from Kliegl et al. that Florian gave. To
put it yet another way, if the residual variance is low in comparison to
the random effect variance, there is not much shrinkage at all.  And to
clarify my mistaken claim earlier, normality only comes into play because
(a) the random effect estimation assumes that the random effects are
normally distributed, and (b) residual variance is usually (hopefully)
normal.  But this does not at all enforce normality on the BLUPs.  If they
are all shrunk massively (perhaps because of large residual variance), they
will end up looking pretty normal, but this is a byproduct of shrinking
towards the marginal mean, which will be more dominated by the residual
variance than the grouping level variance, not because of any other
pressure towards normality.

Dan put it to me even more intuitively: if there is enough relative
certainty in the random effects (i.e., with low residual variance), then
the BLUPs will be pretty close to the empirical estimates.

It's always good to see questions that help clarify misunderstandings that
I didn't realize I had! thanks!
-scott




On Fri, Apr 4, 2014 at 3:21 PM, T. Florian Jaeger
<tiflo@csli.stanford.edu>wrote:

> and to follow-up on my email, which immediately raised some questions ;),
> here's what I meant when I said that "as far as i know, shrinkage does not
> enforce / bias towards normality of blups":
>
> shrinkage itself depends on the (estimated) marginal and by-grouping level
> (e.g., by-item) means and variances. These variance (estimates) are derived
> under the assumption of normality. But the shrinkage itself makes no
> reference to normality. to simplify somewhat, shrinkage means that we
> describe the posterior estimates of the by-grouping level means (e.g., the
> by-item means) are described as a weighted sum of the marginal (overall)
> mean and the by-grouping level mean. the weights of each of these
> components depends (in the simplest case) on the relation between the
> residual variance and the between-grouping level variance AND the amount of
> data available for that grouping level.
>
> see
>
> Kliegl, R., Masson, M. E. J., & Richter, E. M. (2010). A linear mixed
> model analysis of masked repetition priming. Visual Cognition, 18(5),
> 655-681. doi:10.1080/13506280902986058
>
> footnote 3
>
> for further details.
>
> Florian
>
>
> On Fri, Apr 4, 2014 at 2:40 PM, T. Florian Jaeger <tiflo@csli.stanford.edu
> > wrote:
>
>> Hi Titus,
>>
>> shrinkage has larger effects on cells with
>>
>> a) means further away from the predicted marginal mean
>> b) fewer cells counts.
>>
>> A good paper to see that is Kliegl et al 2010. I also have some
>> demonstration of this effect in my lmer intro slides (see a recent blog
>> post on the HLP lab blog). Note that you might see deviation away from the
>> marginal mean, because of correlations between the grouping identity (e.g.,
>> item) and other fixed effects in the model. I don't recall whether you had
>> other fixed effects predictors in your model? If so, that could also be the
>> reason for estimates of the random effect correlations.
>>
>> As far as I know shrinkage does not enforce / bias towards normality of
>> BLUPs.
>>
>> I hope of this is helpful.
>>
>> florian
>>
>>
>> On Fri, Apr 4, 2014 at 11:20 AM, Titus von der Malsburg <
>> malsburg@posteo.de> wrote:
>>
>>>
>>> On 2014-04-04 Fri 05:10, T. Florian Jaeger <tiflo@csli.stanford.edu>
>>> wrote:
>>> > I would be careful making anything out of this. The BLUP estimates of
>>> the
>>> > random effects (and, I assume, their distribution) are affected by
>>> > shrinkage, which is often a desirable (conservative) feature, although
>>> it
>>> > will make differences appear smaller. So, it's not surprising that the
>>> > fixed effect model mirrors the empirical means more closely. That
>>> doesn't
>>> > mean though that it's the better model to draw conclusion from (about
>>> those
>>> > differences).
>>>
>>> Florian, your comment is spot on.  Here is a plot showing the effect of
>>> shrinkage in my data set:
>>>
>>>     http://users.ox.ac.uk/~sjoh3968/R/effect_of_shrinkage.png
>>>
>>> Unfilled circles show the empirical mean reading times and differences
>>> between conditions, one circle for each item.  The dots show the BLUP
>>> estimates for each item.
>>>
>>> The difference is fairly dramatic.  I assumed that shrinkage would pull
>>> all data points to the mean with the same force (I have the same amount
>>> of data for all items).  If that were the case, the ordering of items
>>> would be preserved.  However, shrinkage affects the individual items in
>>> quite different ways, and some items are even pushed away from the
>>> overall means (1, 5, 7, 8, 9, 10, 13, 14, 35) effectively expanding a
>>> subset of the estimates instead of shrinking them.
>>>
>>> I must say that I find it hard to swallow that two seemingly valid ways
>>> to analyze the data (item as random effect or fixed effect) yield
>>> results that are so different.
>>>
>>> Another observation: in the BLUP estimates, the correlation of
>>> intercepts and slopes seems to be much higher than in the raw data.  The
>>> correlation of the estimated random intercepts and slopes is -0.86. (The
>>> summary of the model reports -0.62.)  The correlation of the empirical
>>> item means and differences is only -0.4.  Why does lmer believe in such
>>> a high correlation?
>>>
>>>   Titus
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucsd.edu/pipermail/ling-r-lang-l/attachments/20140404/b32066df/attachment-0001.html