[R-lang] Re: False convergence in mixed logit model
Dale Barr
dale.barr@glasgow.ac.uk
Thu Nov 29 12:47:27 PST 2012
Hi all,
I would just note that Laura was getting convergence problems *without*
a random slope (which is unnecessary given the between-subjects
manipulation). It seems to be arising due to the high variability
across subjects, with some giving all 0s and some all 1s (and
presumably, some in between). This makes it hard to estimate the
by-subject random intercept variance.
Given that you are in the fortunate situation of not having crossed
random effects, I would recommend trying to use Generalized Estimating
Equations (GEE), which fits a marginal model (population-average) rather
than a conditional (subject-specific) model. You can find some
discussion of this distinction in Raudenbush & Bryk (2002). Code:
library(gee)
adultdata.gee <- gee(ditrans~condition, id=subj, data=adultdata,
family="binomial")
summary(adultdata.gee)
If adultdata.gee$error is 0, then the model successfully converged.
Hope you have better luck with that!
-Dale
On 29/11/12 20:00, T. Florian Jaeger wrote:
> HI Laura,
>
> if your predictor variable for which you are entertaining a random
> by-subject slope is almost entirely between-subjects, you should go
> without random by-subject slopes for that predictors. Barr et al
> discuss the obvious case of a between-subject manipulation (--> do not
> include by-subject random slopes for the manipulation). It seems that
> you have a case that doesn't fit into the neat distinction made in
> their paper. You have a predictor that /could /vary within subjects,
> but mostly ended up varying between subjects. The problem is that in
> that case, it is hard for the model to determine the (variance for)
> that random slope. Hence, you get the false convergence.
>
> Think about it this way: only the subjects who have /both/ values for
> your binary predictors provide data from which you can actually figure
> out how the effect of that predictors /differs/ across subjects. For
> subjects, who categorically have one value for that predictor, the
> by-subject /intercept /already captures the same information (or, to
> put it differently, the model has not way to distinguish between the
> by-subject intercept and the by-subject slope). Now even when subjects
> aren't quite categorical (e.g., they have value 1 about 98% of the
> time, and value 0 about 2% of the time), it's hard for the model to
> reliably estimate what of the observed variance is due to the
> by-subject random intercept and what is due to the by-subject random
> slope.
>
> Can you provide a tables of subject by predictor (for each of the
> binary predictors)? I.e., a table that shows the proportion of 0 and 1
> values for the predictors for each of your subjects? if you have
> enough folks with enough variance, you might be able to test on the
> subset. If not, you probably have to exclude the random by-subject
> slope for those predictors.
>
> HTH,
>
> Florian
>
>
> On Thu, Nov 29, 2012 at 2:45 PM, Laura Suttle <lsuttle@princeton.edu
> <mailto:lsuttle@princeton.edu>> wrote:
>
>
> Do you mean that the model converges to the same parameter
> values if you set the starting parameters to wildly different
> values?
>
>
> Ok, I'm not sure what constitutes "wildly different." I just tried
> values like 100 and 1000 and now things aren't converging to the
> same values. Maybe I wasn't being different enough before to see it.
>
> It's not obvious to me why this model wouldn't actually be
> converged, except that the variance of the subject random
> effect is very large. Do you have dramatic differences in
> behavior across subjects?
>
>
> I do, in that participants usually either use this construction
> (so they have all 1's for the DV) or don't use it at all (so they
> have all 0's). The relevant thing that I'm trying to show is that
> more people use in one condition versus another. I don't want to
> convert this to a single binary variable since there are some
> subjects who switch up how they use these novel verbs, but for the
> most part they are either all or none.
>
>
>
>> Another potential issue: are subject random effects relevant
>> when you are only looking at between subjects fixed effects?
>> I'm wondering if that's part of the issue.
>
> They're relevant whenever you have repeated measures from
> individual subjects.
>
> If your condition (Dative, Transitive, and something else)
> varies within subjects, and what you care about is inferences
> about the effect of condition, you'll want to fit a model with
> a by-subjects random slope (i.e., response ~ condition +
> (condition | subj)), as per Barr et al.
>
>
> Those conditions (control is the something else) are between
> subjects.
> Thanks,
> Laura
>
>
> Best
>
> Roger
>
>> Here's the code with output for the simplest model I have
>> with subjects random effect (adding them in in any form
>> causes this issue). ditrans is a binary DV that categorizes
>> speaker usages of a novel verb into double object datives (1)
>> or other sentence types (0). Condition is a categorical
>> variable that has three levels with a control condition as
>> the baseline (I've dummy coded these and there are no
>> differences between running it this way and with the dummy
>> code, so I'm providing this one for the same of simplicity).
>>
>>
>> > ditmodel3 <-lmer(ditrans~condition +
>> (1|subj),data=adultdata,family="binomial",verbose=T)
>> 0: 814.04384: 0.391997 -0.472359 -0.840468 0.222898
>> 1: 474.30352: 1.39176 -0.489556 -0.853831 0.221628
>> 2: 446.52182: 1.60483 -1.29691 -1.39869 0.144777
>> 3: 386.13626: 2.59735 -1.19061 -1.38968 0.204059
>> 4: 383.11685: 2.69684 -1.19357 -1.39816 0.208801
>> 5: 377.92258: 2.89551 -1.20228 -1.41725 0.218187
>> 6: 376.11423: 2.97465 -1.20814 -1.42671 0.221868
>> 7: 372.88540: 3.13258 -1.22176 -1.44712 0.229201
>> 8: 371.72249: 3.19542 -1.22881 -1.45656 0.232118
>> 9: 369.58988: 3.32079 -1.24417 -1.47647 0.237957
>> 10: 369.58183: 3.32129 -1.24424 -1.47656 0.237981
>> 11: 369.56574: 3.32229 -1.24439 -1.47674 0.238028
>> 12: 369.55931: 3.32269 -1.24444 -1.47681 0.238046
>> 13: 369.54645: 3.32349 -1.24456 -1.47695 0.238084
>> 14: 369.54594: 3.32352 -1.24456 -1.47696 0.238085
>> 15: 369.54183: 3.32377 -1.24460 -1.47700 0.238097
>> 16: 369.54183: 3.32377 -1.24460 -1.47700 0.238097
>> 17: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 18: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 19: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 20: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 21: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 22: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 23: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 24: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 25: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 26: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 27: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 28: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 29: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 30: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 31: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 32: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 33: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 34: 369.54182: 3.32377 -1.24460 -1.47700 0.238097
>> 35: 369.54175: 3.32378 -1.24460 -1.47699 0.238081
>> 36: 369.54062: 3.32390 -1.24460 -1.47681 0.237810
>> 37: 369.52871: 3.32514 -1.24460 -1.47496 0.234970
>> 38: 369.41249: 3.33754 -1.24460 -1.45648 0.206662
>> 39: 368.48882: 3.45721 -1.24460 -1.27810 -0.0665675
>> 40: 367.50882: 3.71632 -1.24460 -0.891867 -0.658143
>> 41: 367.42619: 3.80235 -1.24460 -0.763626 -0.854531
>> 42: 367.42069: 3.82935 -1.24460 -0.723378 -0.916133
>> 43: 367.42061: 3.83197 -1.24460 -0.719473 -0.922078
>> 44: 367.42056: 3.83302 -1.24460 -0.717885 -0.924450
>> 45: 367.42030: 3.83643 -1.24460 -0.712740 -0.931948
>> 46: 367.41976: 3.84096 -1.24460 -0.705833 -0.941589
>> 47: 367.41821: 3.84903 -1.24460 -0.693317 -0.957909
>> 48: 367.41433: 3.86190 -1.24460 -0.672862 -0.981743
>> 49: 367.40420: 3.88359 -1.24460 -0.637123 -1.01631
>> 50: 367.37911: 3.91871 -1.24460 -0.576118 -1.05854
>> 51: 367.31987: 3.97369 -1.24460 -0.473099 -1.09171
>> 52: 367.19878: 4.04676 -1.24460 -0.319173 -1.06129
>> 53: 367.00997: 4.11079 -1.24460 -0.148079 -0.875985
>> 54: 366.83505: 4.11105 -1.24460 -0.0694293 -0.533561
>> 55: 366.76596: 4.05082 -1.24460 -0.131236 -0.273508
>> 56: 366.75441: 4.00835 -1.24460 -0.205517 -0.224632
>> 57: 366.75330: 3.99692 -1.24460 -0.231735 -0.238739
>> 58: 366.75328: 3.99700 -1.24460 -0.232466 -0.242660
>> 59: 366.75327: 3.99706 -1.24460 -0.232631 -0.243916
>> 60: 366.75321: 3.99733 -1.24460 -0.233186 -0.248657
>> 61: 366.75310: 3.99766 -1.24460 -0.233948 -0.254660
>> 62: 366.75277: 3.99818 -1.24460 -0.235467 -0.265375
>> 63: 366.75194: 3.99913 -1.24460 -0.237734 -0.282069
>> 64: 366.74974: 4.00078 -1.24460 -0.241634 -0.309537
>> 65: 366.74401: 4.00376 -1.24460 -0.248277 -0.353787
>> 66: 366.72917: 4.00940 -1.24460 -0.260050 -0.425563
>> 67: 366.69014: 4.02037 -1.24460 -0.282150 -0.541757
>> 68: 366.58903: 4.04387 -1.24460 -0.323645 -0.729228
>> 69: 366.33054: 4.09566 -1.24460 -0.407098 -1.03058
>> 70: 365.69165: 4.21044 -1.24460 -0.582890 -1.50206
>> 71: 364.22961: 4.45811 -1.24460 -0.955294 -2.19620
>> 72: 363.88256: 4.50536 -1.24460 -1.03067 -2.28460
>> 73: 363.78744: 4.51528 -1.24460 -1.04951 -2.29787
>> 74: 363.76346: 4.51729 -1.24460 -1.05384 -2.29942
>> 75: 363.71559: 4.52093 -1.24460 -1.06284 -2.30192
>> 76: 363.50871: 4.53406 -1.24460 -1.10070 -2.30424
>> 77: 362.67929: 4.58017 -1.24460 -1.25368 -2.28868
>> 78: 362.34767: 4.59767 -1.24460 -1.31389 -2.27484
>> 79: 362.28332: 4.60067 -1.24460 -1.32582 -2.27114
>> 80: 362.15615: 4.60614 -1.24460 -1.34933 -2.26235
>> 81: 362.10583: 4.60818 -1.24460 -1.35861 -2.25843
>> 82: 361.98440: 4.61586 -1.24460 -1.37557 -2.24974
>> 83: 361.94584: 4.61723 -1.24460 -1.38296 -2.24642
>> 84: 361.93799: 4.61752 -1.24460 -1.38442 -2.24571
>> 85: 361.87531: 4.61979 -1.24460 -1.39595 -2.23983
>> 86: 361.87506: 4.61980 -1.24460 -1.39600 -2.23980
>> 87: 361.87456: 4.61982 -1.24460 -1.39609 -2.23975
>> 88: 361.87424: 4.61985 -1.24460 -1.39611 -2.23974
>> 89: 361.87421: 4.61986 -1.24460 -1.39612 -2.23973
>> 90: 361.87420: 4.61986 -1.24460 -1.39612 -2.23973
>> 91: 361.87418: 4.61986 -1.24460 -1.39612 -2.23973
>> 92: 361.87417: 4.61986 -1.24460 -1.39612 -2.23973
>> 93: 361.87417: 4.61986 -1.24460 -1.39612 -2.23973
>> 94: 361.87417: 4.61986 -1.24460 -1.39612 -2.23973
>> 95: 361.87417: 4.61986 -1.24460 -1.39612 -2.23973
>> 96: 361.87417: 4.61986 -1.24460 -1.39612 -2.23973
>> 97: 361.61308: 4.63208 -1.24460 -1.31790 -2.07725
>> 98: 360.78393: 4.70563 -1.24460 -1.01240 -1.42611
>> 99: 360.30955: 4.80814 -1.24460 -0.794807 -0.933962
>> 100: 359.83679: 4.94614 -1.24460 -0.694790 -0.658108
>> 101: 358.66805: 5.34357 -1.24460 -0.659430 -0.362284
>> 102: 357.21569: 5.94720 -1.24460 -0.865900 -0.407961
>> 103: 355.87363: 6.65147 -1.24460 -1.34871 -0.894451
>> 104: 354.79576: 7.33163 -1.24460 -2.04508 -1.75302
>> 105: 354.79427: 7.33250 -1.24460 -2.04623 -1.75453
>> 106: 354.78176: 7.33863 -1.24460 -2.05548 -1.76699
>> 107: 354.78124: 7.33885 -1.24460 -2.05585 -1.76750
>> 108: 354.78114: 7.33890 -1.24460 -2.05592 -1.76760
>> 109: 354.78092: 7.33897 -1.24460 -2.05608 -1.76780
>> 110: 354.78083: 7.33900 -1.24460 -2.05614 -1.76789
>> 111: 354.78081: 7.33901 -1.24460 -2.05615 -1.76790
>> 112: 354.78077: 7.33902 -1.24460 -2.05618 -1.76794
>> 113: 354.78075: 7.33903 -1.24460 -2.05619 -1.76795
>> 114: 354.78075: 7.33903 -1.24460 -2.05619 -1.76795
>> 115: 354.78074: 7.33903 -1.24460 -2.05619 -1.76796
>> 116: 354.78074: 7.33903 -1.24460 -2.05619 -1.76796
>> 117: 354.78074: 7.33903 -1.24460 -2.05619 -1.76796
>> 118: 354.78074: 7.33903 -1.24460 -2.05620 -1.76796
>> 119: 354.78074: 7.33903 -1.24460 -2.05620 -1.76796
>> 120: 354.78074: 7.33903 -1.24460 -2.05620 -1.76796
>> 121: 354.78074: 7.33903 -1.24460 -2.05620 -1.76796
>> 122: 354.78074: 7.33903 -1.24460 -2.05620 -1.76796
>> Warning message:
>> In mer_finalize(ans) : false convergence (8)
>> >
>> > summary(ditmodel3)
>> Generalized linear mixed model fit by the Laplace approximation
>> Formula: ditrans ~ condition + (1 | subj)
>> Data: adultdata
>> AIC BIC logLik deviance
>> 362.8 381.7 -177.4 354.8
>> Random effects:
>> Groups Name Variance Std.Dev.
>> subj (Intercept) 53.861 7.339
>> Number of obs: 833, groups: subj, 48
>>
>> Fixed effects:
>> Estimate Std. Error z value Pr(>|z|)
>> (Intercept) -1.245 1.951 -0.638 0.523
>> conditionDative -2.056 2.832 -0.726 0.468
>> conditionTrans -1.768 2.791 -0.633 0.526
>>
>> Correlation of Fixed Effects:
>> (Intr) cndtnD
>> conditinDtv -0.689
>> conditnTrns -0.699 0.481
>>
>> Thanks and sorry for the length,
>> Laura
>>
>> On Thu, Nov 29, 2012 at 11:08 AM, Levy, Roger <rlevy@ucsd.edu
>> <mailto:rlevy@ucsd.edu>> wrote:
>>
>> Yes -- the first column of the verbose output is the step
>> number and the second column is the deviance. If the
>> deviance was still going down and the model stopped, you
>> probably need more iterations.
>>
>> It could be useful to change the starting value of the
>> model parameters with the "start" argument of lmer and
>> see if you wind up converging to the same parameter
>> estimates regardless of starting value.
>>
>> More information about the dataset, and example code
>> output, is, of course, always helpful.
>>
>> Best
>>
>> Roger
>>
>>
>>
>> On Nov 29, 2012, at 7:03 AM PST, Laura Suttle wrote:
>>
>>> Hi Roger,
>>>
>>> Thanks for the other list suggestion, I'll cross post to
>>> there.
>>>
>>> Every variable in my data set is categorical, so I can't
>>> do that fix. I've tried playing around with the maxIter
>>> parameter before, but I'm not sure I was doing it right.
>>> Do you have any suggestions for where I can read more
>>> about how to interpret the verbose output? I found some
>>> things but they weren't very helpful.
>>>
>>> Thanks,
>>> Laura
>>>
>>>
>>> On Thu, Nov 29, 2012 at 1:34 AM, Levy, Roger
>>> <rlevy@ucsd.edu <mailto:rlevy@ucsd.edu>> wrote:
>>>
>>> Hi Laura,
>>>
>>> This is a question that might be better answered on
>>> R-sig-ME, but briefly: I would be cautious with a
>>> model that reports false convergence; in my
>>> experience with this warning (and I am by no means
>>> expert on it), it can indicate that the optimization
>>> routine that determines the best-fit model
>>> parameters got stuck at a parameter estimate that is
>>> not near a true optimum, perhaps due to numerical
>>> issues. You might try standardizing any continuous
>>> predictor variables you and rerunning the lmer()
>>> call. It would be helpful to set the msVerbose
>>> control parameter to TRUE to see what the optimizer
>>> is doing. Also, upping the maxIter and/or maxFN
>>> control parameters *might* be helpful.
>>>
>>> I do not think that this warning message alone would
>>> be justification to omit a random effect.
>>>
>>> Best & hope that this helps,
>>>
>>> Roger
>>>
>>> On Nov 28, 2012, at 8:58 PM PST, Laura Suttle wrote:
>>>
>>>> Hello all,
>>>>
>>>> I hope this question hasn't been asked before, but
>>>> the internet isn't being of much help to me.
>>>>
>>>> I am trying to run a mixed logit regression
>>>> predicting whether participants use a novel verb in
>>>> a particular construction or not depending on how
>>>> they were exposed to that novel verb. I dummy coded
>>>> the three conditions of the experiment into two
>>>> dummy variables and have added two random effects,
>>>> one for the motion used for the verb, the other for
>>>> the verb itself (since these were all
>>>> counterbalanced).
>>>>
>>>> I can get this model to run fine, the problem is
>>>> when I try to add any kind of random effect for the
>>>> subjects themselves. I then get this error message:
>>>>
>>>> Warning message:
>>>> In mer_finalize(ans) : false convergence (8)
>>>>
>>>> And all of the effects I had of the exposure type
>>>> go away.
>>>>
>>>> I've been trying to look up what this means and how
>>>> to deal with it, but there are no clear solutions
>>>> or explanations that I can find, but plenty of
>>>> warning of how I should be skeptical of any output
>>>> from a model with this warning. One suggestion I
>>>> did find was that the subjects variable may be
>>>> overfitting my data and there might be something to
>>>> this: when participants are exposed to the verb in
>>>> a certain way, they tend to only use the
>>>> construction I'm looking for, with no variance in
>>>> their responses. That said, I'm not sure that's
>>>> right and I'd love a second opinion on either how I
>>>> can fix this or whether I can use this as
>>>> justification to not include the subjects random
>>>> effect.
>>>>
>>>> Thanks in advance for any help you can give,
>>>> Laura Suttle
>>>
>>>
>>
>>
>
>
>
--
Dale Barr
Institute of Neuroscience and Psychology
University of Glasgow
58 Hillhead Street
Glasgow G12 8QB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucsd.edu/pipermail/ling-r-lang-l/attachments/20121129/3f044652/attachment-0001.html
More information about the ling-r-lang-L
mailing list