deepbox/optim
Schedulers
Learning-rate schedulers for warmup, cosine, cyclic, sequential, plateau, and polynomial policies.
Scheduling
CosineAnnealingLR
Cosine annealing learning rate scheduler.
CosineAnnealingWarmRestarts
Cosine annealing with warm restarts.
CyclicLR
Cyclic learning rate scheduler.
ExponentialLR
Exponential learning rate scheduler.
LambdaLR
Lambda learning rate scheduler.
LinearLR
Linear learning rate scheduler.
LRScheduler
Base class for learning rate schedulers.
MultiStepLR
Multi-step learning rate scheduler.
OneCycleLR
One-cycle learning rate scheduler.
PolynomialLR
Polynomial learning rate scheduler.
ReduceLROnPlateau
Reduce learning rate on plateau.
SequentialLR
Sequential learning rate scheduler.
StepLR
Step learning rate scheduler.
WarmupLR
Warmup scheduler that wraps another scheduler.
optim-schedulers.ts
import { Linear } from "deepbox/nn";import { Adam, CosineAnnealingLR, SequentialLR, StepLR, WarmupLR } from "deepbox/optim";const layer = new Linear(4, 2);const optimizer = new Adam(layer.parameters(), { lr: 0.001 });const stepScheduler = new StepLR(optimizer, { stepSize: 10, gamma: 0.5 });const cosineScheduler = new CosineAnnealingLR(optimizer, { tMax: 20 });console.log(stepScheduler);console.log(cosineScheduler);console.log(new WarmupLR(optimizer, cosineScheduler, { warmupEpochs: 5 }));console.log(new SequentialLR(optimizer, { schedulers: [stepScheduler, cosineScheduler], milestones: [5] }));