Build wrapper for Matlab’s SVM classifier

Wrapper for two classes

Notes:

  • this exercise is based on Matlab’s SVM, and requires the Matlab statistics or bioinfo toolbox.
  • an alternative is using libsvm (using cosmo classify libsvm).
  • to use either SVM (whichever is present), you can use cosmo classify svm.

Matlab has an implementation of a support vector machine classifier that supports two classes. Its implementation uses two functions: svmtrain and svmclassify. Have a look at these functions’ signatures (help svmtrain and help svmclassify) and then write a wrapper that will have the same function signature as our generic classifer, but uses matlab’s SVM inside. Below is the signature and function header for our new function.

Test your solution using the first part of run classify svm

function predicted=cosmo_classify_matlabsvm_2class(samples_train, targets_train, samples_test, opt)
% svm classifier wrapper (around svmtrain/svmclassify)
%
% predicted=cosmo_classify_matlabsvm_2class(samples_train, targets_train, samples_test, opt)
%
% Inputs:
%   samples_train      PxR training data for P samples and R features
%   targets_train      Px1 training data classes
%   samples_test       QxR test data
%   opt                struct with options. supports any option that
%                      svmtrain supports
%
% Output:
%   predicted          Qx1 predicted data classes for samples_test
%
% Notes:
%  - this function uses Matlab's builtin svmtrain function, which has
%    the same name as LIBSVM's version. Use of this function is not
%    supported when LIBSVM's svmtrain precedes in the matlab path; in
%    that case, adjust the path or use cosmo_classify_libsvm instead.
%  - Matlab's SVM classifier is rather slow, especially for multi-class
%    data (more than two classes). When classification takes a long time,
%    consider using libsvm.
%  - for a guide on svm classification, see
%      http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf
%    Note that cosmo_crossvalidate and cosmo_crossvalidation_measure
%    provide an option 'normalization' to perform data scaling
%  - As of Matlab 2017a (maybe earlier), Matlab gives the warning that
%      'svmtrain will be removed in a future release. Use fitcsvm instead.'
%    however fitcsvm gives different results than svmtrain. In this
%    function the warning message is suppressed.
%
% See also svmtrain, svmclassify, cosmo_classify_matlabsvm
%
% #   For CoSMoMVPA's copyright information and license terms,   #
% #   see the COPYING file distributed with CoSMoMVPA.           #

Hint: cosmo classify matlabsvm 2class skl

Solution: cosmo classify matlabsvm 2class / first part of run classify svm

Wrapper for multiple classes

Other classifiers (such as naive bayesian) support more than two classes. SVM classifiers can be used for multi-class problems. One approach is to classify based on all possible pairs of classes, and then take as the predicted class the one that was predicted most often. Thus, write a wrapper with the same function signature as the naive bayesian classifier but that uses the 2-class SVM classifier above. Test your solution using the second part of run classify svm.

function predicted=cosmo_classify_matlabsvm(samples_train, targets_train, samples_test, opt)
% SVM multi-classifier using matlab's SVM implementation
%
% predicted=cosmo_classify_matlabsvm(samples_train, targets_train, samples_test, opt)
%
% Inputs
%   samples_train      PxR training data for P samples and R features
%   targets_train      Px1 training data classes
%   samples_test       QxR test data
%   opt                (optional) struct with options for svm_classify
%
% Output
%   predicted          Qx1 predicted data classes for samples_test
%
% Notes:
%  - this function uses matlab's builtin svmtrain function, which has
%    the same name as LIBSVM's version. Use of this function is not
%    supported when LIBSVM's svmtrain precedes in the matlab path; in
%    that case, adjust the path or use cosmo_classify_libsvm instead.
%  - for a guide on svm classification, see
%      http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf
%    note that cosmo_crossvalidate and cosmo_crossvalidation_measure
%    provide an option 'normalization' to perform data scaling
%
% See also svmtrain, svmclassify, cosmo_classify_svm, cosmo_classify_libsvm
%
% #   For CoSMoMVPA's copyright information and license terms,   #
% #   see the COPYING file distributed with CoSMoMVPA.           #

Hint: cosmo classify matlabsvm skl

Solution: cosmo classify matlabsvm skl / run classify svm

Extra exercise: write another multi-class SVM classifier that predicts using a one-versus-all scheme.

Extra exercise: compare the results from run classify svm with an SVM classifier to those of cosmo classify naive bayes.