Manifold & Projection

Isomap: nonlinear dimensionality reduction through geodesic distances.

1. Connects every sample to its nNeighbors nearest neighbors (the graph is made symmetric; separate components are joined through their closest points and a warning is issued). 2. Computes all-pairs shortest-path distances on that graph (Dijkstra). 3. Embeds the geodesic distances with classical MDS.

Unlike t-SNE, Isomap can project new samples with transform.

Classical (Torgerson) multidimensional scaling.

Computes the pairwise Euclidean distances (or takes a precomputed distance matrix), double-centres the squared distances and embeds the points with the leading eigenvectors. This is a closed-form, deterministic solution and not the iterative SMACOF stress minimisation that scikit-learn's MDS runs; for Euclidean input it equals PCA of the data.

Laplacian eigenmaps with a fully connected RBF affinity graph.

The affinity is W_ij = exp(-gamma * ||x_i - x_j||^2) with a zero diagonal. With the degree matrix D, the embedding consists of the eigenvectors of the symmetric normalized affinity D^-1/2 W D^-1/2 that belong to its largest eigenvalues (the smallest eigenvalues of the normalized Laplacian), without the trivial first one, mapped back with D^-1/2. This is the construction of scikit-learn's SpectralEmbedding(affinity="rbf").

t-Distributed Stochastic Neighbor Embedding (t-SNE).

A nonlinear dimensionality reduction technique for embedding high-dimensional data into a low-dimensional space (typically 2D or 3D) for visualization.

Algorithm: Exact t-SNE with an optional sparse approximation - Computes pairwise affinities in high-dimensional space using a Gaussian kernel whose bandwidth is calibrated per point to the requested perplexity - Computes pairwise affinities in low-dimensional space using a Student-t distribution - Minimizes the KL divergence between the two distributions by gradient descent with momentum and adaptive per-coordinate gains

Scalability Note: Exact t-SNE is O(n^2) in time and memory. With method: "approximate" the input affinities are restricted to the approximateNeighbors true nearest neighbors of every point and the repulsive forces are estimated from negativeSamples random points per sample and iteration, so every iteration costs O(n * (neighbors + negatives)). Finding the neighbors is still an O(n^2 d) brute-force pass.

t-SNE is non-parametric: it embeds exactly the samples it was fitted on and cannot project new data.

Gaussian Random Projection.

Reduces dimensionality by projecting data onto a random Gaussian matrix. Each component of the projection matrix is drawn from N(0, 1/n_components).

Based on the Johnson-Lindenstrauss lemma: random projections approximately preserve pairwise distances. With nComponents: "auto" the target dimension is the Johnson-Lindenstrauss bound for the number of training samples and eps.

Without randomState the matrix is drawn from the global generator, so setSeed() makes it reproducible; with randomState it comes from a private stream.