Hyperparameter Optimization

Who this is for

• Applied Scientists shipping models where small gains compound into real business impact.
• Data Scientists and ML Engineers who need reliable, reproducible model improvements on a budget.
• Anyone moving from basic model training to systematic performance tuning.

Prerequisites

• Know basic supervised learning (train/validation/test, classification vs regression).
• Comfort with evaluation metrics (AUC, F1, accuracy, RMSE) and cross-validation.
• Basic scripting in Python or similar, and ability to run experiments (locally or in cloud).

Why this matters

Hyperparameter Optimization (HPO) turns decent models into dependable, high-performing systems. In production, a 1–3% gain in AUC/F1 can reduce costs, improve user experience, or prevent failures. HPO also helps you find simpler, faster models that meet the same target metrics—critical for latency and cost.

Real tasks you will face:

• Designing an efficient search to beat a baseline under a tight compute budget.
• Choosing the right metric and validation scheme for imbalanced or time-dependent data.
• Early stopping and pruning unpromising runs to save time and money.
• Creating reproducible tuning pipelines that can be re-run and audited.

Concept explained simply

Hyperparameters are the model settings you choose before training (like tree depth, learning rate, or regularization strength). Hyperparameter Optimization is the systematic process of picking those settings to maximize an objective (e.g., AUC on validation) within constraints (compute time, memory, latency).

Mental model

Imagine a landscape where each point is a set of hyperparameters, and the altitude is your validation score. HPO is how you explore the landscape efficiently—sampling promising areas more, avoiding cliffs (overfitting), and stopping early when a path looks bad.

Key ideas you will use

• Search spaces: define reasonable ranges (often log-scale for rates/penalties) and discrete options for categorical choices.
• Search strategies:
  • Grid search: exhaustive, reliable for very small spaces.
  • Random search: strong baseline, scales better; try more values per important dimension.
  • Bayesian optimization (e.g., TPE/GP): proposes new trials where improvement is likely.
  • Bandit methods (Hyperband/ASHA): allocate small budgets widely, promote the best.
  • Population-based training (PBT): periodically exploit/explore by mutating good configs.
• Budgeting: number of trials × training time per trial; use early stopping and smaller proxies (subset of data, fewer epochs) to explore cheaply.
• Validation design: stratified CV for imbalanced classification; time-based splits for temporal data; group-aware splits to avoid leakage.
• Objective: optimize the metric that matches your business goal (with tie-breakers like latency or model size).
• Reproducibility: fix seeds, record configs/metrics, and save the final model with its hyperparameters.

Common hyperparameters by model

• Tree ensembles (RandomForest, XGBoost/LightGBM/CatBoost): depth, number of trees, learning rate, subsampling, feature sampling, regularization (L1/L2), min child samples/leaf size.
• Linear/logistic models: regularization strength (C or alpha), penalty type (L1/L2/elastic), class weighting.
• SVM (RBF): C and gamma (search on log-scale).
• Neural nets: learning rate, batch size, optimizer choice, weight decay, dropout, layer sizes, epochs; use schedulers and early stopping.

Efficient HPO workflow

1. Pick a clear objective metric and validation scheme aligned to the use case.
2. Design search spaces with sensible bounds; use log-scale for rates and regularization.
3. Start with a quick, cheap search (random/ASHA) to map the landscape.
4. Refine with Bayesian optimization or narrowed ranges; allocate more budget to promising areas.
5. Confirm with robust evaluation (more folds/longer training) and test on a hold-out set.
6. Lock seeds, export the best config, and document the run (budget, metric, date, dataset version).

Worked examples

Quick checklist

• Objective metric matches the business goal and class balance.
• Validation split method prevents leakage (time, groups, stratification).
• Search spaces are bounded, plausible, and log-scaled where appropriate.
• Initial exploration uses random/ASHA; refinement uses Bayesian or narrowed ranges.
• Trials are reproducible (fixed seeds) and logged with configs and metrics.
• Final model re-trained on full training data and confirmed on untouched test.

Exercises to practice

These exercises mirror the tasks below in the Exercises section. You can complete them offline and compare with the provided solutions.

1. Exercise 1 (ex1): Design a search plan.

   Scenario: Binary classification with 200k rows, moderate imbalance (pos=15%). You will train LightGBM. Budget: 2 GPU-hours. Metric: AUC. Validation: stratified 5-fold CV. Propose:

   • A sensible search space for key hyperparameters.
   • A search strategy and early stopping plan.
   • A trial budget (how many runs, how to allocate time).
2. Exercise 2 (ex2): Diagnose and fix overfitting during HPO.

   Scenario: You tuned an XGBoost model. CV AUC = 0.903, but holdout AUC = 0.882. Provide likely causes and concrete fixes to your HPO setup.

• When done, open the solutions in the Exercises section below and compare.

Common mistakes and self-checks

• Leakage via random splits on temporal or grouped data.
  • Self-check: Are future timestamps or duplicate users leaking across folds? Use time-based or group-aware CV.
• Overly wide or unbounded spaces causing wasted trials.
  • Self-check: Plot best scores vs each hyperparameter. Are extremes rarely good? Tighten ranges.
• Ignoring class imbalance.
  • Self-check: Are metrics dominated by the majority class? Use stratification and appropriate metrics or class weights.
• Optimizing proxy metrics that don’t reflect business goals.
  • Self-check: If you care about precision at K, are you optimizing that exactly (or a reliable proxy)?
• No early stopping/pruning.
  • Self-check: Are many trials clearly underperforming early? Use ASHA/early stopping to reclaim budget.

Practical projects

1. Imbalanced click-through prediction: Tune a gradient boosting model to maximize PR-AUC with stratified CV and threshold tuning.
2. Time-series demand forecasting: Tune LightGBM with time-based splits and a custom objective that penalizes under-forecasting more.
3. Latency-constrained scoring: Optimize a tree ensemble for AUC with a secondary constraint that 95th percentile latency stays under X ms; report the Pareto frontier.

Mini challenge

Your baseline CatBoost model reaches AUC 0.874 on a 1M-row dataset. You have 90 minutes of compute. Propose a minimal HPO plan (space, strategy, budget, early stopping) that has a credible chance to beat the baseline by ≥0.01 AUC, and explain why.

Learning path

1. Revisit metrics and validation (stratified/time-based/group-aware splits).
2. Master search strategies: random, Bayesian, and bandit-based early stopping.
3. Design robust search spaces for your most-used models.
4. Run small pilot searches; analyze sensitivity plots; narrow ranges.
5. Confirm results with stronger validation; document and package the best config.
6. Automate periodic re-tuning with fixed seeds and clear budgets.

Next steps

• Complete the Exercises and take the Quick Test below.
• Apply HPO to one live or historical project and report both performance and cost/latency impacts.
• Create a reusable HPO template: input dataset, metric, search space, budget, and logging.