Support Vector Machines (SVM)

Support Vector Machines find the decision boundary that maximizes the margin between classes, often leading to strong performance in high-dimensional spaces.

Maximum Margin Intuition

Given separable classes, SVM finds the hyperplane that maximizes the distance (margin) to the closest points, called support vectors.

Maximizing the margin typically improves generalization.
Soft-margin SVM allows some misclassifications controlled by parameter C.

Kernel Trick

SVMs can use kernels to implicitly map data into higher-dimensional spaces where it becomes linearly separable.

Linear kernel
Polynomial kernel
RBF (Gaussian) kernel

SVM with scikit-learn

RBF kernel SVM

from sklearn.svm import SVC
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

svm = SVC(
    kernel="rbf",
    C=1.0,
    gamma="scale"
)
svm.fit(X_train, y_train)

y_pred = svm.predict(X_test)
print(classification_report(y_test, y_pred))

Previous: Random Forest Next: K-Nearest Neighbours