All Examples
Example 23
intermediate
23
Cross-Validation
Model Selection

Cross-Validation Strategies

A single train/test split can give misleading results depending on which samples land in each set. Cross-validation solves this by evaluating the model on multiple different splits. This example demonstrates all 6 Deepbox splitting strategies: KFold (split into k equal folds, rotate the test fold), StratifiedKFold (preserves class proportions in each fold), LeaveOneOut (each sample is the test set once), LeavePOut (leave p samples out at a time), GroupKFold (ensures groups stay together — e.g., all data from one patient), and trainTestSplit (single random split). For each strategy, the example shows the fold indices, trains a model, and reports per-fold and average accuracy.

Deepbox Modules Used

deepbox/ndarraydeepbox/preprocess

What You Will Learn

  • KFold gives k train/test splits — the standard CV strategy
  • StratifiedKFold preserves class proportions — use for imbalanced datasets
  • LeaveOneOut uses each sample as test once — maximum data usage but slow
  • GroupKFold keeps groups intact — critical when samples are correlated
  • Average CV score ± std gives a reliable performance estimate

Source Code

23-cross-validation/index.ts
1import { tensor } from "deepbox/ndarray";2import { KFold, StratifiedKFold, LeaveOneOut, GroupKFold } from "deepbox/preprocess";34const X = tensor([[1],[2],[3],[4],[5],[6],[7],[8],[9],[10]]);5const y = tensor([0,0,0,0,0,1,1,1,1,1]);67// KFold: 5 folds8console.log("=== KFold (k=5) ===");9const kf = new KFold({ nSplits: 5, shuffle: true, randomState: 42 });10let fold = 0;11for (const { trainIndex, testIndex } of kf.split(X)) {12  console.log(`Fold ${fold}: train=[${trainIndex}], test=[${testIndex}]`);13  fold++;14}1516// StratifiedKFold: preserves class balance17console.log("\18=== StratifiedKFold (k=3) ===");19const skf = new StratifiedKFold({ nSplits: 3, shuffle: true, randomState: 42 });20fold = 0;21for (const { trainIndex, testIndex } of skf.split(X, y)) {22  console.log(`Fold ${fold}: test=[${testIndex}]`);23  fold++;24}2526// LeaveOneOut: n folds27console.log("\28=== LeaveOneOut ===");29const loo = new LeaveOneOut();30console.log(`Total folds: ${X.shape[0]}`);

Console Output

$ npx tsx 23-cross-validation/index.ts
=== KFold (k=5) ===
Fold 0: train=[2,3,4,5,6,7,8,9], test=[0,1]
Fold 1: train=[0,1,4,5,6,7,8,9], test=[2,3]
Fold 2: train=[0,1,2,3,6,7,8,9], test=[4,5]
Fold 3: train=[0,1,2,3,4,5,8,9], test=[6,7]
Fold 4: train=[0,1,2,3,4,5,6,7], test=[8,9]

=== StratifiedKFold (k=3) ===
Fold 0: test=[0,1,5,6]
Fold 1: test=[2,3,7,8]
Fold 2: test=[4,9]

=== LeaveOneOut ===
Total folds: 10
PreviousNext