Previous: Learning Problems

Next: Meta-classifiers

 

Classifiers

 

The standard supervised learning problem comprises a set of training examples {(x₁,y₁),...,(x_m,y_m)} drawn identically and independently from some unknown distribution. Each example (x_i,y_i) has two components: the feature vector (x_i) and some label (y_i). The feature vector is generally a fixed-length set of numerical features (or attributes). When the label is real-valued, the problem is called regression. When the label belongs to a discreet set, then the problem is classification. The following algorithms solve the binary classification problem where the label may have one of two values.

 

 

C4.5

 

A decision tree builds an interpretable model that represents a set of rules. It is a popular tool for classification that is relatively fast to train and make predictions.

 

Features

• Requires/handles a large dataset
• Not require data preprocessing
• Handles missing data
• Avoids overfitting
• Handles contineous and normal attributes
• Several attribute selection measures
• Performs subsetting on discreet values
• Graphical Models using: graphviz

 

Tutorial: http://dms.irb.hr/tutorial/tut_dtrees.php

 

 

 

 

Tree

 

A decision tree builds an interpretable model that represents a set of rules. It is a popular tool for classification that is relatively fast to train and make predictions.

 

Features

• Not require data preprocessing
• Requires/handles a large dataset
• Handles missing attributes
• Supports weighted examples
• Computational efficient for discreet attributes
• Attribute selection: Misclassification, Gini, Entropy, TopDown, Least-squared
• Control over attribute reuse
• Graphical Models using: graphviz

 

Tutorial: http://dms.irb.hr/tutorial/tut_dtrees.php

 

 

ADTree

 

The ADTree classifier leverages a boosting procedure to build an alternating decision tree or option tree, a generalization of the decision tree. The final decision is reached summing over each decision.

 

Features

• Not require data preprocessing
• Powerful classifier, intuitive graphical model
• Computational efficient for discreet attributes
• Graphical Models using: graphviz

 

Application: Predicting Genetic Regulatory Response

 

 

 

 

LibSVM

 

The support vector machines (SVM) classifer employs the kernel trick to perform non-linear classification in a linear space. The SVM implementation used in malibu is LIBSVM.

 

Features

• Sparse solution
• Handles noise in data
• Simple model selection

 

Short Tutorial: http://www.idiom.com/~zilla/Work/Notes/svmtutorial.pdf

 

 

 

 

 

Previous: Learning Problems

Next: Meta-classifiers