deepbox/ml
Nearest Neighbors
Instance-based (lazy) learning that stores all training data and classifies new points based on the k closest stored examples. No explicit training phase — all computation happens at prediction time. KNN makes no assumptions about the data distribution, making it effective for complex, non-linear boundaries. The main drawback is O(n·d) prediction cost and sensitivity to feature scaling.
KNeighborsClassifier
K-Nearest Neighbors classifier. For each query point, finds the k training points with smallest Euclidean distance, then assigns the majority class among those neighbors (ties broken by nearest). Decision boundary is implicitly defined by Voronoi partitions. Always preprocess with StandardScaler — features on different scales will dominate the distance metric.
KNeighborsRegressor
K-Nearest Neighbors regressor. Predicts the mean (or optionally distance-weighted mean) of the k nearest training targets. Produces piecewise-constant predictions with k=1 and smoother predictions with larger k.
Euclidean Distance
d(x, x') = √(Σⱼ (xⱼ − x'ⱼ)²)
Where:
- j = Feature index
KNN Classification
ŷ = argmax_c Σᵢ∈Nₖ(x) 𝟙(yᵢ = c)
Where:
- Nₖ(x) = Set of k nearest neighbors of x
KNN Regression
ŷ = (1/k) Σᵢ∈Nₖ(x) yᵢ
Where:
- yᵢ = Target value of neighbor i
Constructor Parameters
- nNeighbors: number — Number of neighbors k (default: 5). Small k → complex boundary (may overfit), large k → smooth boundary (may underfit).
- weights: 'uniform' | 'distance' — 'uniform' gives equal weight to all neighbors; 'distance' weights by 1/d so closer neighbors have more influence (default: 'uniform').
knn.ts
import { KNeighborsClassifier, KNeighborsRegressor } from "deepbox/ml";import { StandardScaler } from "deepbox/preprocess";import { tensor } from "deepbox/ndarray";const X = tensor([[1, 2], [2, 3], [3, 1], [4, 3], [5, 2]]);const y = tensor([0, 0, 1, 1, 1]);// Always scale features firstconst scaler = new StandardScaler();const XScaled = scaler.fitTransform(X);// Classification: majority vote among k=3 nearestconst knn = new KNeighborsClassifier({ nNeighbors: 3 });knn.fit(XScaled, y);knn.predict(scaler.transform(tensor([[3, 2]]))); // [1]// Regression: average of k nearest targetsconst yReg = tensor([1.0, 2.0, 1.5, 3.0, 2.5]);const knnReg = new KNeighborsRegressor({ nNeighbors: 3, weights: 'distance' });knnReg.fit(XScaled, yReg);knnReg.predict(scaler.transform(tensor([[3, 2]])));