From gillespie.maureen at gmail.com Fri Aug 21 15:41:12 2009 From: gillespie.maureen at gmail.com (Maureen Gillespie) Date: Fri, 21 Aug 2009 18:41:12 -0400 Subject: [R-lang] What happens when a random effect has a st. dev. of close to zero? Message-ID: Hi everyone, I have been using weighted empircal logit linear regression (Barr, 2008) to analyze data from a number of agreement error production experiments. (Just as a side note, I have run into lots of problems trying to use logit mixed models for this data as errors are extremely rare: certain conditions produce essentially no errors and all other conditions rarely have higher than 15% error rates. If anyone has a better solution than the emp.logit please let me know!) That being said, I am running what is essentially a meta-analysis. I have data from 5 experiments and 104 different items (some of which appear in multiple experiments, some only appear in a single experiment). My model has two continuous predictors and two random effects (experiment and item). lmer(emp.logit ~ IV1 + IV2 + (1|item) + (1|exp), data, weights) When I run the model, my estimates, standard errors, and t-values all appear reasonable (i.e., comparable to other single random effect models I have run using this technique on similar data). There is no colinearity or anything else to suggest that something is wrong. But when I use pvals.fnc() to compute CIs and p values for the estimates, I find that the experiment random effect has a std. dev. of 0.0000 (5.0e-11 to be exact), and this seems to inflate the CI of the intercept estimate (t = 17, but it's only marginal significant w/ pvals from MCMC). If I run the same model excluding the experiment random effect, estimates do not change and the CIs and p values for the intercept appear normal. Strangely (or maybe not) the two models have the exact same log likelihoods. Is this just an extreme example of a random effect not being necessary? And, more on the conceptual end of things, why would a near-zero st.dev. of a random effect inflate CIs w/MCMC sampling? Thanks in advance, Maureen Gillespie Northeastern University -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlevy at ling.ucsd.edu Sat Aug 22 07:32:59 2009 From: rlevy at ling.ucsd.edu (Roger Levy) Date: Sat, 22 Aug 2009 07:32:59 -0700 Subject: [R-lang] What happens when a random effect has a st. dev. of close to zero? In-Reply-To: References: Message-ID: <185349F5-22DD-41B9-984B-E0670D20B803@ling.ucsd.edu> Hi Maureen, On Aug 21, 2009, at 3:41 PM, Maureen Gillespie wrote: > Hi everyone, > I have been using weighted empircal logit linear regression (Barr, > 2008) to analyze data from a number of agreement error production > experiments. (Just as a side note, I have run into lots of problems > trying to use logit mixed models for this data as errors are > extremely rare: certain conditions produce essentially no errors and > all other conditions rarely have higher than 15% error rates. If > anyone has a better solution than the emp.logit please let me know!) Yes, I have had similar difficulties with logit models where at least one condition is error-free. One thing you may be prone to running into in these cases is the unreliability of the Wald z-statistic. Search for "standard error is inflated" on this page: http://userwww.sfsu.edu/~efc/classes/biol710/logistic/logisticreg.htm > That being said, I am running what is essentially a meta-analysis. > I have data from 5 experiments and 104 different items (some of > which appear in multiple experiments, some only appear in a single > experiment). My model has two continuous predictors and two random > effects (experiment and item). > > lmer(emp.logit ~ IV1 + IV2 + (1|item) + (1|exp), data, weights) > > When I run the model, my estimates, standard errors, and t-values > all appear reasonable (i.e., comparable to other single random > effect models I have run using this technique on similar data). > There is no colinearity or anything else to suggest that something > is wrong. But when I use pvals.fnc() to compute CIs and p values > for the estimates, I find that the experiment random effect has a > std. dev. of 0.0000 (5.0e-11 to be exact), and this seems to inflate > the CI of the intercept estimate (t = 17, but it's only marginal > significant w/ pvals from MCMC). If I run the same model excluding > the experiment random effect, estimates do not change and the CIs > and p values for the intercept appear normal. Strangely (or maybe > not) the two models have the exact same log likelihoods. > > Is this just an extreme example of a random effect not being > necessary? > > And, more on the conceptual end of things, why would a near-zero > st.dev. of a random effect inflate CIs w/MCMC sampling? I'm not sure what you mean by inflating the CI -- do you mean making the CI on the fixed-effect intercept larger than it is in a model without the random effect of experiment? With such a small random effect of experiment, the model probably *is* telling you that you don't need it. Try comparing your model's likelihood with the likelihood of a model that doesn't have the random effect of experiment -- the likelihoods should be very similar. (Technically it's best to use restricted maximum likelihood (REML=TRUE) when doing this, but that is the default so it looks like you're doing that already. Best Roger > > > Thanks in advance, > > Maureen Gillespie > Northeastern University > _______________________________________________ > R-lang mailing list > R-lang at ling.ucsd.edu > http://pidgin.ucsd.edu/mailman/listinfo/r-lang -- Roger Levy Email: rlevy at ling.ucsd.edu Assistant Professor Phone: 858-534-7219 Department of Linguistics Fax: 858-534-4789 UC San Diego Web: http://ling.ucsd.edu/~rlevy From sloiseau at limsi.fr Mon Aug 24 23:36:34 2009 From: sloiseau at limsi.fr (sloiseau at limsi.fr) Date: Tue, 25 Aug 2009 08:36:34 +0200 (CEST) Subject: [R-lang] Problem with standard generic methods in Matrix package Message-ID: <55530.193.51.130.16.1251182194.squirrel@keo.limsi.fr> Hello, I'm puzzled by a problem with call to diag() and rowSums() on object of class "dgtMatrix", created by sparseMatrix() or spMatrix(). I use Matrix 0.999375-30. The weird thing is that I don't encounter any problem when I enter the function on the R prompt, or source()-ing a script file; I encounter problems when the code is in a package, installed with R CMD INSTALL, and loaded with library(). I have tried to reduce the package to the bare minimum in order to discover the problem, but it continue even when reduced up to a single method, no dependency other than Matrix. The method is defined as: --- getAdjacentAsSparseMatrix <- function() { adjacents <- sparseMatrix(i=1:10, j=1:10, x=1:10) print(class(adjacents)); diagonal <- as.vector(diag(adjacents)); if (any(diagonal != 0)) { diagonal[diagonal > 1] <- 1; diag(adjacents) <- diagonal; } return(adjacents); } --- And the Depends field in ./DESCRIPTION : --- Depends: R (>= 2.5.0), Matrix --- I have see two outputs: [1] "dgTMatrix" Erreur dans y[1L + 0L:(m - 1L) * (n + 1L)] <- x : types (de S4 a double) incompatibles dans l'ajustement d'affectation de type > traceback() 3: diag(adjacents) 2: as.vector(diag(adjacents)) 1: getAdjacentAsSparseMatrix(white3) The same thing happens with the method rowSums(), colnames() and rownames(). In brief, it looks as if the generic functions were called, and not the overloaded one. I can't figure out how to fix that, and will be interested in any pointers. Best, Sylvain