Pillar 1 · Model Selection — family deep dive
Types of Classic ML
No neural networks here — and on structured business tables these methods still win on accuracy, cost and explainability. The types split along one question first: do you have labeled outcomes to learn from? Everything else follows from that.
Picking the type
The classic-ML decision tree
The types, with named models
| Type | Named models / libraries | Choose it when | Trade-off |
|---|---|---|---|
| Linear / logistic regression | scikit-learn, statsmodels; credit scorecards | Baselines, pricing, regulated risk scoring where every coefficient must be defensible | Ultimate explainability; misses non-linear patterns |
| Decision tree | CART (scikit-learn) | You need human-readable if/then rules for the business | Readable but weak alone — usually a building block |
| Random forest | scikit-learn RandomForest | Robust "just works" model with little tuning | Solid, rarely best; slower to score than boosting |
| Gradient boosting | XGBoost, LightGBM, CatBoost | The tabular default — churn, fraud, pricing, ranking; wins most tabular benchmarks | Needs feature engineering + retraining cadence for drift |
| SVM | LIBSVM / scikit-learn SVC | Small, high-dimensional datasets (mostly legacy today) | Doesn't scale to big data; boosting usually beats it |
| Clustering | k-means, DBSCAN, hierarchical (scikit-learn) | Customer segmentation, store grouping — no labels needed | "Right" cluster count is a judgment call; validate with the business |
| Anomaly detection | Isolation Forest, One-Class SVM, LOF | Fraud/fault detection when labeled fraud examples are scarce | Flags unusual, not necessarily bad — needs human triage |
| Time-series forecasting | Prophet, ARIMA/SARIMA, LightGBM with lag features; deep options: N-BEATS, TFT | Demand, capacity, cash-flow forecasting | Regime changes break history-based models — monitor and refit |
Default play for tabular data: start with gradient boosting (XGBoost/LightGBM) plus a
linear baseline. If boosting doesn't clearly beat the baseline, your features — not your model — are the
problem. Deep learning enters tabular problems only with very large data or mixed unstructured inputs.
Why this family is an architect's friend: CPU-only inference (no GPU line item),
millisecond scoring, mature explainability tooling (SHAP feature importance) for auditors — and your
existing data warehouse is the feature store.