deepbox/datasets
DataLoader, Samplers & Transforms
Mini-batch loading, dataset transforms, subsets, splits, and sampling policies for iterative workloads.
Input pipelines
type DataLoaderOptions
export type DataLoaderOptions = { batchSize?: number; shuffle?: boolean; dropLast?: boolean; seed?: number; sampler?: Sampler; collateFn?: CollateFn<any>; };
Configuration options for .
type StreamingDataLoaderOptions
export type StreamingDataLoaderOptions = { /** Samples per batch. Default: 1. */ batchSize?: number; /** * Window shuffle-buffer size. When set (`>= 1`), samples are shuffled through * a bounded buffer of this many elem…
Configuration options for a constructed over a (out-of-core / lazy source).
type Sampler
export interface Sampler { … }
Dataset samplers for controlling index generation in DataLoader.
DataLoader
Data loader for batching and shuffling datasets.
SequentialSampler
Samples elements sequentially, always in the same order.
SubsetRandomSampler
Samples elements randomly from a given list of indices (subset), without replacement.
WeightedRandomSampler
Samples elements according to given weights (probabilities), with or without replacement.
Subset
A subset of a dataset defined by explicit indices.
filterDataset
export declare function filterDataset(dataset: Dataset, predicate: (data: number[], target: number) => boolean): Dataset;
Filter a dataset by a predicate function, keeping only samples where the predicate returns true.
mapDataset
export declare function mapDataset(dataset: Dataset, fn: (data: number[], target: number) => { data: number[]; target: number; }): Dataset;
Apply a transformation function to every sample in a dataset, producing a new dataset.
randomSplit
export declare function randomSplit(dataset: Dataset, lengths: readonly number[], seed?: number): Subset[];
Randomly split a dataset into non-overlapping subsets of given lengths.
datasets-dataloader.ts
import { DataLoader, SequentialSampler, randomSplit, loadIris,} from "deepbox/datasets";const dataset = loadIris();const [train, test] = randomSplit(dataset, [120, 30], 42);const loader = new DataLoader(train.data, train.target, { batchSize: 16, sampler: new SequentialSampler(train.data.shape[0]),});for (const [xBatch, yBatch] of loader) { console.log(xBatch.shape, yBatch.shape);}console.log(test.data.shape[0]);