[R-lang] Re: Self Paced Reading experiment using residuals with lmer

[Bruno Nicenboim]

Thanks for the detailed answer.
It's a pity that for  some reason, my question appeared 1 month and a half
after I posted it!
I'll take a look to the links you sent me, I'm still interested in
understanding the effect of the transformations.

Bruno


On Fri, Oct 15, 2010 at 7:44 PM, T. Florian Jaeger
<tiflo@csli.stanford.edu>wrote:

> Hi Bruno,
>
> good question. Here's what I'd say:
>
> If I wanted to analyze *all* word-by-word RTs in my data, I would
> definitely do what Baayen et al did. However, in most SPR experiments, we
> are only analyzing the *items*, but not the fillers. Often folks are even
> only analyzing specific regions in the items. At that point then, I prefer
> to derive per-word residual RTs *for* the region of interest in the items
> *from* the entire data (to decorrelate properties of the region of
> interest that I am interested in from properties that can be reduced to
> commonalities with the remainder with the data). That cannot be achieved by
> the procedure employed in Harald's article.
>
> This also explains why I moved the position terms (which, btw, I model as a
> non-linear term in order to be maximally conservative) into the
> residualization (in addition to the standard terms for word length). This
> procedure is especially useful when -in your *items*- condition is
> confounded with position (of the word in the sentence or the sentence in the
> list, although the latter is usually easy to avoid).
>
> I know for a fact (unpleasant experience) that this type of confound (which
> is rarely controlled in experiments with positional confounds) does matter
> and that this only shows in full clarity in the procedure I proposed. I had
> a nicely significant result, which went away once I applied this procedure
> (which is why the procedure is described in a blog article rather than in a
> journal ;)) and when I checked further I became very certain that the
> analysis was doing the right thing (for the type of situation I am
> describing here).
>
> So, long story short: if you're analyzing all data, then use Baayen's
> method. If you are analyzing only your items, I think there are reasons to
> use the method I propose. For examples of the analysis I proposed, see
> Hofmeister et al, submitted and Hofmeister in press (which also shows that
> the analysis doesn't *always* kill effects ;).
>
> *As for transformations*, several researchers have looked into this for
> various tasks. Obviously, there is Smith and Levy's work, suggesting that
> raw RTs should be used. There is work by Kliegl et al (2010) comparing raw,
> log, and reciprocal transforms of RTs (though not for SPR), arguing -if I
> recall correctly- for a reciprocal link.
>
> Here's what I usually do (I think Baayen, Kuperman, etc. would do the
> same): I look at the data (qqplots, shapiro test for normality over
> residuals of model or -faster- over raw data by condition if you have a
> factorial design) and compare the usual suspects (raw, log, reciprocal). For
> some examples, you might find Victor Kuperman and my slides prepared for
> WOMM 2009 useful (
> http://hlplab.wordpress.com/2009-pre-cuny-workshop-on-ordinary-and-multilevel-models-womm/,
> there are some updated version of these slides at:
> http://hlplab.wordpress.com/2010/05/10/mini-womm-montreal-slides-now-available/
> ).
>
> When you do that, it becomes pretty obvious that the decision will in part
> depend on whether you prefer to exclude outliers or not. log transforms are
> useful when you do not remove outliers (as they make them less extreme
> values, since most outliers in SPR experiments are large rather than small
> values). I have to say that, in my experience, the transform did hardly ever
> change the results (and if that happens and I know it then I try to
> understand why).
>
> HTH,
> Florian
>
> On Sun, Aug 29, 2010 at 4:22 AM, Bruno Nicenboim <
> bruno.nicenboim@gmail.com> wrote:
>
>> Hi,
>> I'm analyzing the results of a SPR experiment.
>>
>> I saw that in Jaeger's blog (HLP/Jaeger lab blog) and in Jaeger, Fedorenko
>> and
>> Gibson's article "Anti-locality in English: Consequences for Theories of
>> Sentence Comprehension" in order to analyze the results,  they use a
>> linear
>> model that takes as dependent variables the residuals of a model that
>> looks
>> roughly like this: (I didn't include the transformations they use)
>>
>> l <- lmer(RT ~  Wordlenght + positionofword + positionofstimulus +  (1 |
>> SUBJ)...
>>
>> RTresidual <- residuals(l)
>>
>> (http://hlplab.wordpress.com/2008/01/23/modeling-self-paced-reading-data-
>> effects-of-word-length-word-position-spill-over-etc/#more-46<http://hlplab.wordpress.com/2008/01/23/modeling-self-paced-reading-data-effects-of-word-length-word-position-spill-over-etc/#more-46>
>> )
>>
>> Then, the final linear model looks like this:
>>
>> l <- lmer(RTresidual ~ CONDITION +
>>            SPILLOVER_1 + SPILLOVER_2 + SPILLOVER_3 +
>>            (1 | SUBJ) + (1 | ITEM)
>>
>> On the other hand, Baayen and Milim in "Analyzing Reaction Times" use a
>> model
>> that takes that takes as a dependent variable the RT (instead of
>> residuals), and
>> includes the word lenght and the position of the word and line in the same
>> model, roughly like:
>>
>> l <- lmer(RT ~ CONDITION + Wordlenght + positionofword +
>> positionofstimulus +
>>            SPILLOVER_1 + SPILLOVER_2 + SPILLOVER_3 +
>>            (1 | SUBJ) + (1 | ITEM)
>>
>>
>> My questions are:
>> Is there any advantage or disadvantage that should persuade me to use one
>> approach or the other?
>> Shouldn't I get similar results? (Because I don't)
>> And finally, I've noticed that each researcher (not only in these two
>> examples)
>> uses different transformations on length, positions and reading times. Is
>> there
>> any way to check which transformation is the most appropriate?
>>
>> Thanks !
>>
>>
>>
>


-- 
Bruno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://mailman.ucsd.edu/mailman/private/ling-r-lang-l/attachments/20101015/601f0d8d/attachment.html