Performance Profiling And Optimization

Why this matters

In real ML/AI work, speed saves money and unlocks scale. You will:

• Speed up feature engineering and data preprocessing pipelines.
• Reduce model training time (and cost) by removing bottlenecks.
• Batch and stream data efficiently for large datasets.
• Make real-time inference meet latency SLAs.
• Diagnose memory spikes that crash notebooks or jobs.

Concept explained simply

Profiling is measuring where time and memory are spent. Optimization is changing code or data flow to make the slowest parts faster without breaking correctness.

Mental model

• Measure first: find the hottest 1–2 functions. Optimize only those.
• Pick the right tactic: algorithmic improvements > vectorization > parallelism > micro-tweaks.
• Trade-offs: speed vs memory vs readability. Document decisions.

Workflow: from slow to fast

1. Set a baseline: Wrap the full task with a timer. Record input size, runtime, memory peaks.
2. Profile: Use cProfile to find hot functions. If needed, measure within a function using perf_counter.
3. Choose a tactic: Try algorithmic changes, vectorization, batching, caching, or parallelism.
4. Optimize safely: Keep a correctness test. Optimize one change at a time.
5. Re-measure: Compare against baseline. Stop when you meet the target.

Worked examples

1) Vectorize a feature computation

import math

def feat_loop(xs):
    out = []
    for x in xs:
        y = 3*x*x + 2*x + 1
        out.append(math.sqrt(y))
    return sum(out)

xs = list(range(100000))
print(feat_loop(xs))

import numpy as np

def feat_vec(xs):
    x = np.asarray(xs)
    y = 3*x*x + 2*x + 1
    return np.sqrt(y).sum()

xs = np.arange(100000)
print(feat_vec(xs))

2) Find a bottleneck with cProfile

import cProfile, pstats, io

def slow_fn(n):
    s = 0
    for i in range(n):
        for j in range(100):
            s += (i % 7) * (j % 11)
    return s

pr = cProfile.Profile()
pr.enable()
slow_fn(20000)
pr.disable()
s = io.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats('cumtime')
ps.print_stats(10)
print(s.getvalue())  # Look for the function with the highest cumtime

import numpy as np

def faster_fn(n):
    j = np.arange(100)
    j_term = (j % 11)
    j_sum = j_term.sum()  # invariant across i
    # (i % 7) repeats every 7
    i_mod = np.arange(n) % 7
    return int((i_mod * j_sum).sum())

3) Reduce memory via generators and batching

def sum_squares_list(n):
    data = [i*i for i in range(n)]  # allocates n items
    return sum(data)

print(sum_squares_list(10_000_000))  # may stress memory

def sum_squares_gen(n):
    return sum(i*i for i in range(n))  # no large list in memory

print(sum_squares_gen(10_000_000))

Common mistakes and self-check

• Optimizing without measuring: Always keep baseline and after numbers.
• Micro-optimizing non-hot code: Focus on top 1–2 hotspots from the profiler.
• Ignoring algorithmic changes: A better algorithm beats low-level tweaks.
• Python loops over arrays: Prefer NumPy/pandas vectorized ops.
• Threads for CPU-bound work: Use multiprocessing or vectorized/native code.
• Memory bloat from temporary lists: Use generators and in-place operations.
• Not testing correctness: Add assertions comparing old vs new outputs on sample data.

Exercises

Do these in order. Measure before and after. Keep outputs identical.

Exercise ex1 — Vectorize a slow scoring loop with NumPy

Mirror of the exercise in the Exercises panel below.

import time, math

def score_loop(xs):
    s = 0.0
    for x in xs:
        s += math.sqrt(3*x*x + 2*x + 1)
    return s

N = 200_000
xs = list(range(N))

# 1) Time the loop version
start = time.perf_counter()
baseline = score_loop(xs)
loop_t = time.perf_counter() - start
print('baseline:', baseline, 'time:', round(loop_t, 4), 's')

# 2) Write a vectorized function score_vec(xs) using NumPy
# 3) Time it and compare result equality and speedup

• Expected: same result within 1e-6 tolerance.
• Target: 5x or more speedup on large N (typical).

Exercise ex2 — Remove memory bloat with generators

Mirror of the exercise in the Exercises panel below.

import time

def sum_cubes_list(n):
    data = [i*i*i for i in range(n)]
    return sum(data)

n = 8_000_000
start = time.perf_counter()
res_list = sum_cubes_list(n)
list_t = time.perf_counter() - start
print('list version time:', round(list_t, 3), 's')

# Task: Write sum_cubes_gen(n) using a generator expression.
# Compare times and confirm res_gen == res_list.

• Expected: identical sums; generator uses far less memory.
• Tip: Large n may take time; start with 2,000,000 then increase.

Who this is for

• Machine Learning Engineers who need faster data and training pipelines.
• Data Scientists transitioning to production-grade performance.
• Backend engineers adding ML inference to services.

Prerequisites

• Comfortable with Python functions, loops, and basic data structures.
• Basic NumPy or pandas understanding.
• Ability to run scripts and install common packages if needed.

Learning path

• Start: Baseline and profile with cProfile and perf_counter.
• Then: Vectorize and batch with NumPy/pandas.
• Next: Parallelize appropriately (threads for I/O, processes for CPU).
• Finally: Memory discipline (generators, chunking, avoiding copies) and caching.

Practical projects

• Speed up a feature store build job by 5x and document profiling screenshots and timings.
• Reduce a model inference pipeline’s P95 latency under 100 ms using batching and caching.
• Refactor a cross-validation script to parallel folds safely, with constant memory growth.

Next steps

• Take the Quick Test to check your understanding. Anyone can take it; sign in to save progress.
• Apply one optimization to a current project. Measure and write a short before/after note.
• Revisit this lesson in a week and try a different tactic (e.g., caching or batching).

Mini challenge

You have a preprocessing step that repeatedly parses the same 1,000 JSON templates across 5 million rows. Propose two changes to make it faster and lighter, and how you would prove the improvement with numbers.