deepbox/ml
Support Vector Machines
Find the optimal hyperplane that maximizes the margin between classes. SVMs work by mapping data into a higher-dimensional space where a linear separator exists. The margin is the perpendicular distance from the decision boundary to the nearest training points (support vectors). Larger margins lead to better generalization.
LinearSVC
Linear Support Vector Classifier. Finds the maximum-margin hyperplane wᵀx + b = 0 separating classes. The regularization parameter C controls the tradeoff between maximizing the margin and minimizing classification errors. Small C → wide margin (may underfit), large C → narrow margin (may overfit). Uses hinge loss: max(0, 1 − yᵢ(wᵀxᵢ + b)).
LinearSVR
Linear Support Vector Regression. Fits an ε-insensitive tube around the data — errors smaller than ε are ignored. Minimizes ‖w‖² + C·Σᵢ max(0, |yᵢ − (wᵀxᵢ + b)| − ε). The ε parameter controls the width of the no-penalty zone.
SVM Decision Boundary
f(x) = wᵀx + b = 0
Where:
- w = Normal vector to the hyperplane
- b = Bias (offset)
SVM Optimization (Primal)
min (1/2)‖w‖² + C Σᵢ max(0, 1 − yᵢ(wᵀxᵢ + b))
Where:
- C = Regularization (tradeoff: margin width vs. errors)
Margin Width
margin = 2 / ‖w‖
Where:
- ‖w‖ = L2 norm of weight vector
Constructor Parameters
- C: number — Regularization strength (default: 1.0). Inverse of regularization: larger C = less regularization.
- maxIter: number — Maximum iterations for the solver (default: 1000)
- tol: number — Convergence tolerance (default: 1e-4)
svm.ts
import { LinearSVC, LinearSVR } from "deepbox/ml";import { tensor } from "deepbox/ndarray";import { trainTestSplit } from "deepbox/preprocess";const X = tensor([[1, 2], [2, 3], [3, 1], [4, 3], [5, 5], [1, 0]]);const y = tensor([0, 0, 1, 1, 1, 0]);// Classificationconst svm = new LinearSVC({ C: 1.0 });svm.fit(X, y);console.log(svm.predict(tensor([[2.5, 2]]))); // [0] or [1]// Regressionconst yReg = tensor([1.5, 2.5, 2.0, 3.5, 4.0, 0.5]);const svr = new LinearSVR({ C: 1.0 });svr.fit(X, yReg);console.log(svr.predict(tensor([[3, 2]])));