Skip to content

deepbox/ml

Neighbors

Nearest-neighbor classifiers and regressors, radius-based variants, and standalone KD-tree and ball-tree structures.

  • Instance-based

Overview

  • KNeighborsClassifier, KNeighborsRegressor and NearestNeighbors default to 5 neighbors and Euclidean distance ("manhattan" is also available). They scan every training row for each query, which costs O(n d) per sample.
  • NearestNeighbors.kneighbors(X?, k?) returns { distances, indices }. RadiusNeighbors* use every neighbor within radius, and NearestCentroid classifies by the closest class mean.
  • KDTree and BallTree are standalone structures. They take a flat Float64Array, a row count and a feature count, as in new KDTree(data, nSamples, nFeatures). A KD-tree is exact and pays off on low-dimensional data.
  • Neighbor methods are sensitive to feature scale, so standardize features first.

Classes

8

KNeighborsClassifier

extends KNeighborsBase

K-Nearest Neighbors Classifier.

Classification based on k nearest neighbors. Predicts class by majority vote of k nearest training samples. When classes tie, the smallest label wins, as in scikit-learn.

Algorithm: Instance-based learning 1. Store all training data 2. For each test sample, find k nearest training samples 3. Predict class by majority vote (or inverse-distance weighted vote)

Time Complexity: - Training: O(n * d) (copies the data) - Prediction: O(n * d) per sample where n=training samples, d=features

Example
import { KNeighborsClassifier } from 'deepbox/ml';import { tensor } from 'deepbox/ndarray'; const X = tensor([[0, 0], [1, 1], [2, 2], [3, 3]]);const y = tensor([0, 0, 1, 1]); const knn = new KNeighborsClassifier({ nNeighbors: 3 });knn.fit(X, y); const predictions = knn.predict(tensor([[1.5, 1.5]]));
Members (6)
  • classes: Tensor

    Sorted unique class labels seen during fit; columns of predictProba follow this order.

  • fit(X: Tensor, y: Tensor): this

    Fit the k-nearest neighbors classifier from the training set.

  • predict(X: Tensor): Tensor

    Predict class labels for samples in X.

  • predictProba(X: Tensor): Tensor

    Predict class probabilities for samples in X.

  • score(X: Tensor, y: Tensor): number

    Return the mean accuracy on the given test data and labels.

  • clone(): KNeighborsClassifier

    Create an unfitted copy with the same hyperparameters.

KNeighborsRegressor

extends KNeighborsBase

K-Nearest Neighbors Regressor.

Regression based on k nearest neighbors. Predicts value as mean (or weighted mean) of k nearest training samples.

Example
import { KNeighborsRegressor } from 'deepbox/ml';import { tensor } from 'deepbox/ndarray'; const X = tensor([[0], [1], [2], [3]]);const y = tensor([0, 1, 4, 9]); const knn = new KNeighborsRegressor({ nNeighbors: 2 });knn.fit(X, y); const predictions = knn.predict(tensor([[1.5]]));
Members (4)
  • fit(X: Tensor, y: Tensor): this

    Fit the k-nearest neighbors regressor from the training set.

  • predict(X: Tensor): Tensor

    Predict target values for samples in X.

  • score(X: Tensor, y: Tensor): number

    Return the R² score on the given test data and target values.

  • clone(): KNeighborsRegressor

    Create an unfitted copy with the same hyperparameters.

NearestNeighbors

extends KNeighborsBase

Unsupervised nearest-neighbor search.

Stores the training samples and answers k-nearest and fixed-radius queries with the chosen metric. Distances and indices are exact (brute force).

Example
import { NearestNeighbors } from 'deepbox/ml';import { tensor } from 'deepbox/ndarray'; const X = tensor([[0, 0], [1, 0], [0, 2], [5, 5]]);const nn = new NearestNeighbors({ nNeighbors: 2 }).fit(X);const { distances, indices } = nn.kneighbors(tensor([[0.2, 0]]));// indices -> [[0, 1]]
Members (7)
  • constructor(options?: { readonly nNeighbors?: number; readonly radius?: number; readonly metric?: NeighborMetric; })
  • fit(X: Tensor, _y?: Tensor): this

    Store the samples that later queries are answered against.

  • kneighbors(X?: Tensor, nNeighbors?: number): { distances: Tensor; indices: Tensor; }

    Find the k nearest training samples of each query row.

  • radiusNeighbors(X?: Tensor, radius?: number): { distances: number[][]; indices: number[][]; }

    Find all training samples within a radius of each query row.

  • getParams(): Record<string, unknown>
  • setParams(params: Record<string, unknown>): this
  • clone(): NearestNeighbors

    Create an unfitted copy with the same hyperparameters.

BallTree

Ball Tree over a row-major matrix of points.

The tree keeps a reference to data (it is not copied), so the array must not be modified while the tree is in use. All distances are Euclidean. Ties in distance are ordered by ascending point index, so results are deterministic.

Example
import { BallTree } from 'deepbox/ml'; // Three 2-D points stored row by rowconst data = new Float64Array([0, 0, 1, 0, 5, 5]);const tree = new BallTree(data, 3, 2);const { distances, indices } = tree.query(new Float64Array([0.9, 0.1]), 2);// indices -> [1, 0]
Members (6)
  • constructor(data: Float64Array, nSamples: number, nDims: number, leafSize?: number)
  • readonly nSamples: number

    Number of indexed points.

  • readonly nDims: number

    Number of coordinates per point.

  • query(point: Float64Array, k: number): { distances: number[]; indices: number[]; }

    Find the k nearest neighbors of a query point.

  • queryRadius(point: Float64Array, radius: number): { distances: number[]; indices: number[]; }

    Find all points whose distance to point is at most radius.

  • queryBatch(points: Float64Array, nPoints: number, k: number): { distances: Float64Array; indices: Int32Array; }

    Find the k nearest neighbors for several query points.

KDTree

KD-Tree over a row-major matrix of points.

The tree keeps a reference to data (it is not copied), so the array must not be modified while the tree is in use. All distances are Euclidean. Ties in distance are ordered by ascending point index, so results are deterministic.

Example
import { KDTree } from 'deepbox/ml'; // Three 2-D points stored row by rowconst data = new Float64Array([0, 0, 1, 0, 5, 5]);const tree = new KDTree(data, 3, 2);const { distances, indices } = tree.query(new Float64Array([0.9, 0.1]), 2);// indices -> [1, 0]
Members (6)
  • constructor(data: Float64Array, nSamples: number, nDims: number)
  • readonly nSamples: number

    Number of indexed points.

  • readonly nDims: number

    Number of coordinates per point.

  • query(point: Float64Array, k: number): { distances: number[]; indices: number[]; }

    Find the k nearest neighbors of a query point.

  • queryRadius(point: Float64Array, radius: number): { distances: number[]; indices: number[]; }

    Find all points whose distance to point is at most radius.

  • queryBatch(points: Float64Array, nPoints: number, k: number): { distances: Float64Array; indices: Int32Array; }

    Find the k nearest neighbors for several query points.

NearestCentroid

Nearest Centroid classifier.

Each class is represented by its centroid and a sample is assigned to the class with the nearest centroid. With metric: "euclidean" the centroid is the per-feature mean; with metric: "manhattan" it is the per-feature median.

Optionally the centroids can be shrunk towards the overall centroid (shrinkThreshold, the "nearest shrunken centroid" of Tibshirani et al., 2002), which removes features that do not separate the classes.

For predictProba and decisionFunction (Euclidean metric only) the features are scaled by the pooled within-class standard deviation and the scores are -||x - centroid||^2 + 2 log(prior), like scikit-learn.

Example
import { NearestCentroid } from 'deepbox/ml';import { tensor } from 'deepbox/ndarray'; const X = tensor([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]]);const y = tensor([1, 1, 1, 2, 2, 2]);const clf = new NearestCentroid().fit(X, y);clf.predict(tensor([[-0.8, -1]])); // [1]
Members (12)
  • constructor(options?: { readonly metric?: CentroidMetric; readonly shrinkThreshold?: number; readonly priors?: CentroidPriors; })
  • classes: Tensor

    Sorted unique class labels seen during fit.

  • centroids: Tensor

    Class centroids of shape (n_classes, n_features), after shrinking if shrinkThreshold is set.

  • classPrior: Tensor

    Class prior probabilities used by predict, predictProba and decisionFunction.

  • fit(X: Tensor, y: Tensor): this

    Compute the class centroids from the training data.

  • predict(X: Tensor): Tensor

    Predict the class of each sample.

  • predictProba(X: Tensor): Tensor

    Estimate class probabilities as the softmax of the discriminant scores (-||x - centroid||^2 + 2 log(prior) after scaling the features by the pooled within-class standard deviation), as scikit-learn does.

  • decisionFunction(X: Tensor): Tensor

    Discriminant scores of each sample.

  • score(X: Tensor, y: Tensor): number

    Return the mean accuracy on the given test data and labels.

  • getParams(): Record<string, unknown>
  • setParams(params: Record<string, unknown>): this

    Set hyperparameters.

  • clone(): NearestCentroid

    Create an unfitted copy with the same hyperparameters.

RadiusNeighborsClassifier

Classifier that votes among all training samples within a fixed radius.

Unlike k-nearest neighbors, the number of voters varies per sample. A sample with no training point inside the radius is an outlier: by default this is an error, like in scikit-learn; set outlierLabel to assign a label to such samples instead. When classes tie, the smallest label wins.

Example
import { RadiusNeighborsClassifier } from 'deepbox/ml';import { tensor } from 'deepbox/ndarray'; const X = tensor([[0, 0], [0.1, 0], [5, 5], [5.1, 5]]);const y = tensor([0, 0, 1, 1]);const clf = new RadiusNeighborsClassifier({ radius: 1, outlierLabel: -1 }).fit(X, y);clf.predict(tensor([[0.2, 0], [20, 20]])); // [0, -1]
Members (9)
  • constructor(options?: { readonly radius?: number; readonly weights?: RadiusWeights; readonly outlierLabel?: OutlierLabel; })
  • classes: Tensor

    Sorted unique class labels seen during fit.

  • fit(X: Tensor, y: Tensor): this

    Store the training data.

  • predict(X: Tensor): Tensor

    Predict the class of each sample by (weighted) vote of the training samples in its radius.

  • predictProba(X: Tensor): Tensor

    Estimate class probabilities as the (weighted) fraction of in-radius neighbors per class.

  • score(X: Tensor, y: Tensor): number

    Return the mean accuracy on the given test data and labels.

  • getParams(): Record<string, unknown>
  • setParams(params: Record<string, unknown>): this

    Set hyperparameters.

  • clone(): RadiusNeighborsClassifier

    Create an unfitted copy with the same hyperparameters.

RadiusNeighborsRegressor

Regressor that averages the targets of all training samples within a fixed radius.

A sample with no training point inside the radius gets the prediction NaN and a warning, like in scikit-learn.

Example
import { RadiusNeighborsRegressor } from 'deepbox/ml';import { tensor } from 'deepbox/ndarray'; const X = tensor([[1], [2], [3], [4]]);const y = tensor([2, 4, 6, 8]);const reg = new RadiusNeighborsRegressor({ radius: 1.1 }).fit(X, y);reg.predict(tensor([[2.5]])); // [5]
Members (7)
  • constructor(options?: { readonly radius?: number; readonly weights?: RadiusWeights; })
  • fit(X: Tensor, y: Tensor): this

    Store the training data.

  • predict(X: Tensor): Tensor

    Predict the (weighted) mean target of the training samples within the radius.

  • score(X: Tensor, y: Tensor): number

    Return the R² score on the given test data and targets.

  • getParams(): Record<string, unknown>
  • setParams(params: Record<string, unknown>): this

    Set hyperparameters.

  • clone(): RadiusNeighborsRegressor

    Create an unfitted copy with the same hyperparameters.

ml-neighbors.ts
import {  KDTree,  KNeighborsClassifier,  NearestNeighbors,  RadiusNeighborsClassifier,} from "deepbox/ml";import { tensor } from "deepbox/ndarray"; const X = tensor([[1, 2], [2, 1], [3, 3], [4, 4]]);const y = tensor([0, 0, 1, 1]);const tree = new KDTree(new Float64Array([1, 2, 2, 1, 3, 3, 4, 4]), 4, 2); console.log(new KNeighborsClassifier({ nNeighbors: 1 }).fit(X, y).predict(X).toString());console.log(new RadiusNeighborsClassifier({ radius: 2 }).fit(X, y).predict(X).toString());const { distances, indices } = new NearestNeighbors({ nNeighbors: 2 }).fit(X).kneighbors(tensor([[2, 2]]));console.log(distances.toString(), indices.toString());console.log(tree.query(new Float64Array([2, 2]), 2)); // { distances, indices } as plain arrays

Found a mistake? Open an issue.