All Examples
Example 11
intermediate
11
Trees
Ensembles
SVM

Tree-Based & Ensemble Models

Tree-based models are among the most powerful and interpretable ML algorithms. This example trains DecisionTreeClassifier/Regressor (single tree with depth control), RandomForestClassifier/Regressor (bagging ensemble of randomized trees), GradientBoostingClassifier/Regressor (sequential boosting that corrects previous errors), and LinearSVC/LinearSVR (support vector machines with linear kernel). Each model is evaluated on the Iris or Diabetes dataset with appropriate metrics. You will see how ensembles dramatically improve over single trees, and how gradient boosting trades off training speed for accuracy.

Deepbox Modules Used

deepbox/datasetsdeepbox/mldeepbox/ndarraydeepbox/metricsdeepbox/preprocess

What You Will Learn

  • Decision Trees split features recursively — control depth to prevent overfitting
  • Random Forests bag many randomized trees for variance reduction
  • Gradient Boosting builds trees sequentially, each correcting the previous
  • LinearSVC finds the maximum-margin hyperplane for classification

Source Code

11-tree-ensemble-models/index.ts
1import { loadIris } from "deepbox/datasets";2import {3  DecisionTreeClassifier, RandomForestClassifier,4  GradientBoostingClassifier, LinearSVC5} from "deepbox/ml";6import { accuracy, f1Score } from "deepbox/metrics";7import { StandardScaler, trainTestSplit } from "deepbox/preprocess";89console.log("=== Tree & Ensemble Models ===\n");1011const iris = loadIris();12const [X_tr, X_te, y_tr, y_te] = trainTestSplit(13  iris.data, iris.target, { testSize: 0.2, randomState: 42 }14);15const scaler = new StandardScaler();16scaler.fit(X_tr);17const X_train = scaler.transform(X_tr);18const X_test = scaler.transform(X_te);1920const models = [21  { name: "Decision Tree", m: new DecisionTreeClassifier({ maxDepth: 5 }) },22  { name: "Random Forest", m: new RandomForestClassifier({ nEstimators: 50 }) },23  { name: "Gradient Boost", m: new GradientBoostingClassifier({ nEstimators: 50 }) },24  { name: "Linear SVC", m: new LinearSVC() },25];2627for (const { name, m } of models) {28  m.fit(X_train, y_tr);29  const preds = m.predict(X_test);30  console.log(`${name}: acc=${accuracy(y_te, preds).toFixed(3)}, f1=${f1Score(y_te, preds, { average: "macro" }).toFixed(3)}`);31}

Console Output

$ npx tsx 11-tree-ensemble-models/index.ts
=== Tree & Ensemble Models ===

Decision Tree:   acc=0.933, f1=0.932
Random Forest:   acc=0.967, f1=0.966
Gradient Boost:  acc=0.967, f1=0.966
Linear SVC:      acc=0.967, f1=0.966
PreviousNext