[R-lang] Re: lmer: Significant fixed effect only when random slopeisincluded
J. Vogels
J.Vogels@uvt.nl
Thu May 12 02:45:36 PDT 2011
The condition means are as follows (1=pronoun; 0=no pronoun):
a b c d e f
0 0.12765957 0.11956522 0.91304348 0.86046512 0.74725275 0.77528090
1 0.87234043 0.88043478 0.08695652 0.13953488 0.25274725 0.22471910
so many of them around 90%. Would that be a problem?
The subject-by-condition means are not very informative, because each subject has only 2 responses per condition. So the means here are either 0%, 50%, or 100%.
Starting with the model with the full fixed effects structure and the random slope for cAGVIS, a likelihood-ratio test indicates that removing the interaction term between the fixed effects does not result in a significant difference (p=.096). A second test shows that further removing cAGVIS results in a difference in fit that does not reach the .05 significance level either (p=.056). However, comparing a model with only cAGTOP as predictor to the first model does give a significant result (p=.031), suggesting that cAGVIS has some influence.
@Ian & Daniel: I used the anova() function in R to do model comparisons. When the model did not converge, I started by removing the interaction terms for the random slopes, first in the item effects, then in the subject effects, as suggested by Florian Jaeger in his blog. Next, I tried either removing cAGVIS or cAGTOP, again first for the items. The first model that converged had by-subjects random slopes for cAGVIS and cAGTOP, and no by-items random slopes.
Jorrig
> -----Original Message-----
> From: ling-r-lang-l-bounces@mailman.ucsd.edu [mailto:ling-r-lang-l-
> bounces@mailman.ucsd.edu] On Behalf Of Levy, Roger
> Sent: woensdag 11 mei 2011 20:17
> To: ling-r-lang-l@mailman.ucsd.edu
> Subject: [R-lang] Re: lmer: Significant fixed effect only when random
> slopeisincluded
>
> Just as a brief follow-up: since this is categorical data, given the
> magnitudes of some of the coefficients in question I would indeed worry
> a bit. It looks like some condition means may be close to 100%, but
> that different subjects may have really dramatically different behavior
> and that this factor may dominate everything else. Also, getting close
> to 100% can mess with the Z statistic. What do your condition-mean and
> subject-by-condition mean tables look like? And what do likelihood-
> ratio tests for your fixed effects tell you in these models? You may
> be fine and the random-slope model you included in your first message
> may indeed be a good one for interpreting your data, but eyeballing
> these tables would be useful as well.
>
> Roger
>
>
> On May 11, 2011, at 6:04 AM, Finlayson, Ian wrote:
>
> > What was the random effect structure of this first converging model?
> As I said earlier the significant interaction seems fine (as does your
> explanation), but I'm just curious about how you carried out backward
> stepwise elimination when the full model didn't converge.
> >
> > Ian
> >
> >
> > From: ling-r-lang-l-bounces+ifinlayson=qmu.ac.uk@mailman.ucsd.edu
> [mailto:ling-r-lang-l-bounces+ifinlayson=qmu.ac.uk@mailman.ucsd.edu] On
> Behalf Of Jorrig Vogels
> > Sent: 11 May 2011 11:56
> > To: ling-r-lang-l@mailman.ucsd.edu
> > Subject: [R-lang] Re: lmer: Significant fixed effect only when random
> slopeisincluded
> >
> > Hello Ian,
> >
> > In the full model, I included random slopes for cAGTOP and cAGVIS and
> their interaction for both subjects and items. However, this did not
> converge. The first model that converged showed a significant
> interaction between cAGTOP1 and cAGVIS, but not between cAGTOP2 and
> AGVIS.
> >
> > Jorrig
> >
> >
> > From: Finlayson, Ian
> > Sent: Wednesday, May 11, 2011 12:36 PM
> > To: Jorrig Vogels ; ling-r-lang-l@mailman.ucsd.edu
> > Subject: RE: [R-lang] lmer: Significant fixed effect only when random
> slope isincluded
> >
> > Hello,
> >
> > I assume that you only have one random slope because the removal of
> the other two (cAGTOP, and its interaction with cAGVIS) didn't
> significantly harm fit. Were the fixed effects for the interaction
> significant in the full model?
> >
> > FWIW, I have seen this happen before and it seems perfectly
> reasonable to me that an effect may only become significant after
> controlling for some of the noise.
> >
> > Ian
> >
> > From: ling-r-lang-l-bounces@mailman.ucsd.edu [mailto:ling-r-lang-l-
> bounces@mailman.ucsd.edu] On Behalf Of Jorrig Vogels
> > Sent: 11 May 2011 10:50
> > To: ling-r-lang-l@mailman.ucsd.edu
> > Subject: [R-lang] lmer: Significant fixed effect only when random
> slope isincluded
> >
> > Dear R users,
> >
> > I have a logit mixed model with two categorical predictors (two types
> of salience measures) and a categorical dependent variable (pronoun
> used Y/N). One predictor has 2 levels, and the other has 3. I centered
> the 2-level predictor, and transformed the 3-level predictor into two
> binary predictors using contrast (sum) coding. I determined the random-
> effects structure by starting from a full model, and eliminating step
> by step all terms without a significant contribution to the model.
> >
> > In the final model, I end up with random intercepts for subjects and
> items, and a by-subject random slope for my 2-level predictor. In this
> model, I get significant interactions between the fixed factors, which
> I had not expected to be significant by just looking at the data.
> Removing the random slope from the model completely eliminates these
> interactions, but model comparison suggests the random slope should be
> included. I have attached the two model summaries below.
> >
> > Now my question is: is it normal to find such a large influence of
> random effects on the fixed effects structure? How do I know the
> interaction effects are not spurious? And what exactly do these
> findings mean? Participants varied greatly in their reaction to
> predictor B, but when this variation is accounted for, predictor B
> affects pronoun use, but differently for each level of predictor A?
> >
> >
> > Jorrig Vogels
> > PhD candidate
> > Tilburg Univ., Netherlands
> >
> > ================================================================
> >
> > Model with random slope:
> >
> > Generalized linear mixed model fit by the Laplace approximation
> > Formula: PRO ~ cAGTOP * cAGVIS + (1 + cAGVIS | SUBJ) + (1 | ITEM)
> > Data: vislingag
> > AIC BIC logLik deviance
> > 318.4 361.4 -149.2 298.4
> > Random effects:
> > Groups Name Variance Std.Dev. Corr
> > SUBJ (Intercept) 49.6457 7.0460
> > cAGVIS 21.8342 4.6727 0.663
> > ITEM (Intercept) 1.3205 1.1491
> > Number of obs: 544, groups: SUBJ, 48; ITEM, 12
> >
> > Fixed effects:
> > Estimate Std. Error z value Pr(>|z|)
> > (Intercept) -2.578 1.217 -2.117 0.03422 *
> > cAGTOP1 -6.627 0.913 -7.259 3.90e-13 ***
> > cAGTOP2 9.868 1.502 6.569 5.05e-11 ***
> > cAGVIS -1.699 1.008 -1.685 0.09207 .
> > cAGTOP1:cAGVIS -3.223 1.170 -2.755 0.00587 **
> > cAGTOP2:cAGVIS 3.120 1.371 2.275 0.02289 *
> > ---
> > Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> >
> > Correlation of Fixed Effects:
> > (Intr) cAGTOP1 cAGTOP2 cAGVIS cAGTOP1:
> > cAGTOP1 0.075
> > cAGTOP2 -0.041 -0.867
> > cAGVIS 0.535 0.108 -0.059
> > cAGTOP1:AGV 0.074 0.562 -0.346 0.128
> > cAGTOP2:AGV -0.049 -0.480 0.528 -0.054 -0.668
> >
> >
> > Model without random slope:
> >
> > Generalized linear mixed model fit by the Laplace approximation
> > Formula: PRO ~ cAGTOP * cAGVIS + (1 | SUBJ) + (1 | ITEM)
> > Data: vislingag
> > AIC BIC logLik deviance
> > 324.3 358.7 -154.2 308.3
> > Random effects:
> > Groups Name Variance Std.Dev.
> > SUBJ (Intercept) 21.63217 4.65104
> > ITEM (Intercept) 0.61539 0.78447
> > Number of obs: 544, groups: SUBJ, 48; ITEM, 12
> >
> > Fixed effects:
> > Estimate Std. Error z value Pr(>|z|)
> > (Intercept) -1.41142 0.77639 -1.818 0.0691 .
> > cAGTOP1 -4.59707 0.52139 -8.817 <2e-16 ***
> > cAGTOP2 7.13115 0.84489 8.440 <2e-16 ***
> > cAGVIS -0.35538 0.40416 -0.879 0.3792
> > cAGTOP1:cAGVIS -0.59940 0.58255 -1.029 0.3035
> > cAGTOP2:cAGVIS -0.08268 0.56682 -0.146 0.8840
> > ---
> > Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> >
> > Correlation of Fixed Effects:
> > (Intr) cAGTOP1 cAGTOP2 cAGVIS cAGTOP1:
> > cAGTOP1 0.070
> > cAGTOP2 -0.040 -0.867
> > cAGVIS 0.000 0.061 -0.036
> > cAGTOP1:AGV 0.038 0.082 -0.012 0.102
> > cAGTOP2:AGV -0.020 0.008 -0.037 0.037 -0.575
>
> --
>
> Roger Levy Email: rlevy@ucsd.edu
> Assistant Professor Phone: 858-534-7219
> Department of Linguistics Fax: 858-534-4789
> UC San Diego Web: http://idiom.ucsd.edu/~rlevy
>
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucsd.edu/pipermail/ling-r-lang-l/attachments/20110512/ace83cb9/attachment-0001.html
More information about the ling-r-lang-L
mailing list