[R-lang] simple max margin classifier?

Kyle Gorman kylebgorman@gmail.com
Wed Oct 6 16:05:44 PDT 2010


I am predicting a binomial outcome (whether a complex linguistic structure is attested in a large corpus or not) from a single continuous variable (which is roughly the joint probability of the constituent parts of said structure). I'm uninterested in any kind of beta or p-value, just the IR stats (accuracy, F1, etc.) for the supervised prediction task compared to the same task with a baseline which predicts the most frequent outcome for every trial. As a first stab, I used logistic regression, and then linear discriminant analysis. These of course assume some variety of linearity and normality, neither of which hold, or which I care about. What seems to make sense as a better objective is something that maximizes the soft margin. 

So my wishlist is:

1. a classifier (preferably in R, though I guess I could use WEKA) that assumes monotonicity between the predictor and the binomial outcome
2. but not normality or linearity, or homoskedacity between the two classes
3. the community of practice would probably prefer it if there was no stochasticity in the estimated parameters or predictions (e.g., same answer every time)

The original support vector machine proposed by Vapnik is a linear classifier, but by default, the svm() function provided by the e1071 library provides a radial kernel. From some simulations I made up, this default setting _seems_ to satisfy #1-3. However, is it exceptionlessly true that the radial kernel SVM assumes #1? And if so, why (for example, it appears to not be true of a polynomial kernel of certain degree)? I'd also be interested in any other suggestions for non-linear, deterministic, monotonic classifiers. 

Thanks all,
Kyle Gorman
University of Pennsylvania


More information about the ling-r-lang-L mailing list