[R-lang] Re: lmer: Significant fixed effect only when random slopeisincluded
J. Vogels
J.Vogels@uvt.nl
Fri May 13 02:37:57 PDT 2011
Hi Roger,
>
> I don't think we have got enough information from you to interpret the
> conditions when they are named a-f...
I'm sorry, I should have changed the names that came from the data frame:
AGTOP+ AGTOP- AGTOPo
AGVIS+ AGVIS- AGVIS+ AGVIS- AGVIS+ AGVIS-
mean % pronoun use: 0.87234043 0.88043478 0.08695652 0.13953488 0.25274725 0.22471910
>
> I take it pronoun use is the response not a predictor?
Yes, that's right.
>
> Also: if there are only 2 responses per subject per condition, then
> there must be *huge* inter-subject variation in order to support a
> subject random effect. I'd think that plotting the subject-by-
> condition means would still be useful in these circumstances.
Okay, so if I understand it correctly, the fact that there is a random effect of subject, while only having a few observations per subject per condition, means that there is lots of variation between my subjects, especially for the predictor AGVIS. And such an amount of variation may cause the model to detect effects of AGVIS that are actually not there? When I look at the subject-by-condition means, I see some subjects that produce pronouns either almost exclusively or hardly at all, irrespective of condition. Should I remove these subjects in order to reduce variation?
@René: thanks, I will have a look at that, although I don't think I really understand the concept of marginal models.
Jorrig
>
> Roger
>
>
> On May 12, 2011, at 2:45 AM, J. Vogels wrote:
>
> > The condition means are as follows (1=pronoun; 0=no pronoun):
> >
> > a b c d e f
> > 0 0.12765957 0.11956522 0.91304348 0.86046512 0.74725275 0.77528090
> > 1 0.87234043 0.88043478 0.08695652 0.13953488 0.25274725 0.22471910
> >
> > so many of them around 90%. Would that be a problem?
> >
> > The subject-by-condition means are not very informative, because each
> subject has only 2 responses per condition. So the means here are
> either 0%, 50%, or 100%.
> > Starting with the model with the full fixed effects structure and the
> random slope for cAGVIS, a likelihood-ratio test indicates that
> removing the interaction term between the fixed effects does not result
> in a significant difference (p=.096). A second test shows that further
> removing cAGVIS results in a difference in fit that does not reach
> the .05 significance level either (p=.056). However, comparing a model
> with only cAGTOP as predictor to the first model does give a
> significant result (p=.031), suggesting that cAGVIS has some influence.
> >
> > @Ian & Daniel: I used the anova() function in R to do model
> comparisons. When the model did not converge, I started by removing the
> interaction terms for the random slopes, first in the item effects,
> then in the subject effects, as suggested by Florian Jaeger in his blog.
> Next, I tried either removing cAGVIS or cAGTOP, again first for the
> items. The first model that converged had by-subjects random slopes for
> cAGVIS and cAGTOP, and no by-items random slopes.
> >
> > Jorrig
> >
> >
> >
> > > -----Original Message-----
> > > From: ling-r-lang-l-bounces@mailman.ucsd.edu [mailto:ling-r-lang-l-
> > > bounces@mailman.ucsd.edu] On Behalf Of Levy, Roger
> > > Sent: woensdag 11 mei 2011 20:17
> > > To: ling-r-lang-l@mailman.ucsd.edu
> > > Subject: [R-lang] Re: lmer: Significant fixed effect only when
> random
> > > slopeisincluded
> > >
> > > Just as a brief follow-up: since this is categorical data, given
> the
> > > magnitudes of some of the coefficients in question I would indeed
> worry
> > > a bit. It looks like some condition means may be close to 100%,
> but
> > > that different subjects may have really dramatically different
> behavior
> > > and that this factor may dominate everything else. Also, getting
> close
> > > to 100% can mess with the Z statistic. What do your condition-mean
> and
> > > subject-by-condition mean tables look like? And what do
> likelihood-
> > > ratio tests for your fixed effects tell you in these models? You
> may
> > > be fine and the random-slope model you included in your first
> message
> > > may indeed be a good one for interpreting your data, but eyeballing
> > > these tables would be useful as well.
> > >
> > > Roger
> > >
> > >
> > > On May 11, 2011, at 6:04 AM, Finlayson, Ian wrote:
> > >
> > > > What was the random effect structure of this first converging
> model?
> > > As I said earlier the significant interaction seems fine (as does
> your
> > > explanation), but I'm just curious about how you carried out
> backward
> > > stepwise elimination when the full model didn't converge.
> > > >
> > > > Ian
> > > >
> > > >
> > > > From: ling-r-lang-l-bounces+ifinlayson=qmu.ac.uk@mailman.ucsd.edu
> > > [mailto:ling-r-lang-l-bounces+ifinlayson=qmu.ac.uk@mailman.ucsd.edu]
> On
> > > Behalf Of Jorrig Vogels
> > > > Sent: 11 May 2011 11:56
> > > > To: ling-r-lang-l@mailman.ucsd.edu
> > > > Subject: [R-lang] Re: lmer: Significant fixed effect only when
> random
> > > slopeisincluded
> > > >
> > > > Hello Ian,
> > > >
> > > > In the full model, I included random slopes for cAGTOP and cAGVIS
> and
> > > their interaction for both subjects and items. However, this did
> not
> > > converge. The first model that converged showed a significant
> > > interaction between cAGTOP1 and cAGVIS, but not between cAGTOP2 and
> > > AGVIS.
> > > >
> > > > Jorrig
> > > >
> > > >
> > > > From: Finlayson, Ian
> > > > Sent: Wednesday, May 11, 2011 12:36 PM
> > > > To: Jorrig Vogels ; ling-r-lang-l@mailman.ucsd.edu
> > > > Subject: RE: [R-lang] lmer: Significant fixed effect only when
> random
> > > slope isincluded
> > > >
> > > > Hello,
> > > >
> > > > I assume that you only have one random slope because the removal
> of
> > > the other two (cAGTOP, and its interaction with cAGVIS) didn't
> > > significantly harm fit. Were the fixed effects for the interaction
> > > significant in the full model?
> > > >
> > > > FWIW, I have seen this happen before and it seems perfectly
> > > reasonable to me that an effect may only become significant after
> > > controlling for some of the noise.
> > > >
> > > > Ian
> > > >
> > > > From: ling-r-lang-l-bounces@mailman.ucsd.edu [mailto:ling-r-lang-
> l-
> > > bounces@mailman.ucsd.edu] On Behalf Of Jorrig Vogels
> > > > Sent: 11 May 2011 10:50
> > > > To: ling-r-lang-l@mailman.ucsd.edu
> > > > Subject: [R-lang] lmer: Significant fixed effect only when random
> > > slope isincluded
> > > >
> > > > Dear R users,
> > > >
> > > > I have a logit mixed model with two categorical predictors (two
> types
> > > of salience measures) and a categorical dependent variable (pronoun
> > > used Y/N). One predictor has 2 levels, and the other has 3. I
> centered
> > > the 2-level predictor, and transformed the 3-level predictor into
> two
> > > binary predictors using contrast (sum) coding. I determined the
> random-
> > > effects structure by starting from a full model, and eliminating
> step
> > > by step all terms without a significant contribution to the model.
> > > >
> > > > In the final model, I end up with random intercepts for subjects
> and
> > > items, and a by-subject random slope for my 2-level predictor. In
> this
> > > model, I get significant interactions between the fixed factors,
> which
> > > I had not expected to be significant by just looking at the data.
> > > Removing the random slope from the model completely eliminates
> these
> > > interactions, but model comparison suggests the random slope should
> be
> > > included. I have attached the two model summaries below.
> > > >
> > > > Now my question is: is it normal to find such a large influence
> of
> > > random effects on the fixed effects structure? How do I know the
> > > interaction effects are not spurious? And what exactly do these
> > > findings mean? Participants varied greatly in their reaction to
> > > predictor B, but when this variation is accounted for, predictor B
> > > affects pronoun use, but differently for each level of predictor A?
> > > >
> > > >
> > > > Jorrig Vogels
> > > > PhD candidate
> > > > Tilburg Univ., Netherlands
> > > >
> > > > ================================================================
> > > >
> > > > Model with random slope:
> > > >
> > > > Generalized linear mixed model fit by the Laplace approximation
> > > > Formula: PRO ~ cAGTOP * cAGVIS + (1 + cAGVIS | SUBJ) + (1 | ITEM)
> > > > Data: vislingag
> > > > AIC BIC logLik deviance
> > > > 318.4 361.4 -149.2 298.4
> > > > Random effects:
> > > > Groups Name Variance Std.Dev. Corr
> > > > SUBJ (Intercept) 49.6457 7.0460
> > > > cAGVIS 21.8342 4.6727 0.663
> > > > ITEM (Intercept) 1.3205 1.1491
> > > > Number of obs: 544, groups: SUBJ, 48; ITEM, 12
> > > >
> > > > Fixed effects:
> > > > Estimate Std. Error z value Pr(>|z|)
> > > > (Intercept) -2.578 1.217 -2.117 0.03422 *
> > > > cAGTOP1 -6.627 0.913 -7.259 3.90e-13 ***
> > > > cAGTOP2 9.868 1.502 6.569 5.05e-11 ***
> > > > cAGVIS -1.699 1.008 -1.685 0.09207 .
> > > > cAGTOP1:cAGVIS -3.223 1.170 -2.755 0.00587 **
> > > > cAGTOP2:cAGVIS 3.120 1.371 2.275 0.02289 *
> > > > ---
> > > > Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> > > >
> > > > Correlation of Fixed Effects:
> > > > (Intr) cAGTOP1 cAGTOP2 cAGVIS cAGTOP1:
> > > > cAGTOP1 0.075
> > > > cAGTOP2 -0.041 -0.867
> > > > cAGVIS 0.535 0.108 -0.059
> > > > cAGTOP1:AGV 0.074 0.562 -0.346 0.128
> > > > cAGTOP2:AGV -0.049 -0.480 0.528 -0.054 -0.668
> > > >
> > > >
> > > > Model without random slope:
> > > >
> > > > Generalized linear mixed model fit by the Laplace approximation
> > > > Formula: PRO ~ cAGTOP * cAGVIS + (1 | SUBJ) + (1 | ITEM)
> > > > Data: vislingag
> > > > AIC BIC logLik deviance
> > > > 324.3 358.7 -154.2 308.3
> > > > Random effects:
> > > > Groups Name Variance Std.Dev.
> > > > SUBJ (Intercept) 21.63217 4.65104
> > > > ITEM (Intercept) 0.61539 0.78447
> > > > Number of obs: 544, groups: SUBJ, 48; ITEM, 12
> > > >
> > > > Fixed effects:
> > > > Estimate Std. Error z value Pr(>|z|)
> > > > (Intercept) -1.41142 0.77639 -1.818 0.0691 .
> > > > cAGTOP1 -4.59707 0.52139 -8.817 <2e-16 ***
> > > > cAGTOP2 7.13115 0.84489 8.440 <2e-16 ***
> > > > cAGVIS -0.35538 0.40416 -0.879 0.3792
> > > > cAGTOP1:cAGVIS -0.59940 0.58255 -1.029 0.3035
> > > > cAGTOP2:cAGVIS -0.08268 0.56682 -0.146 0.8840
> > > > ---
> > > > Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> > > >
> > > > Correlation of Fixed Effects:
> > > > (Intr) cAGTOP1 cAGTOP2 cAGVIS cAGTOP1:
> > > > cAGTOP1 0.070
> > > > cAGTOP2 -0.040 -0.867
> > > > cAGVIS 0.000 0.061 -0.036
> > > > cAGTOP1:AGV 0.038 0.082 -0.012 0.102
> > > > cAGTOP2:AGV -0.020 0.008 -0.037 0.037 -0.575
> > >
> > > --
> > >
> > > Roger Levy Email: rlevy@ucsd.edu
> > > Assistant Professor Phone: 858-534-7219
> > > Department of Linguistics Fax: 858-534-4789
> > > UC San Diego Web: http://idiom.ucsd.edu/~rlevy
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
>
> --
>
> Roger Levy Email: rlevy@ucsd.edu
> Assistant Professor Phone: 858-534-7219
> Department of Linguistics Fax: 858-534-4789
> UC San Diego Web: http://idiom.ucsd.edu/~rlevy
>
>
>
>
>
>
>
>
More information about the ling-r-lang-L
mailing list