Governance For Shared Compute

Why this matters

Shared compute is the backbone of most data platforms. Multiple teams run ELT jobs, BI dashboards, ad‑hoc analysis, and machine learning on the same finite capacity. Without clear governance, one heavy job can slow down your dashboards, costs can spike unexpectedly, and SLOs are missed. Good governance protects performance, cost, and fairness.

• Real task: Keep BI dashboards snappy during month-end loads.
• Real task: Cap spend for ad‑hoc queries without blocking critical pipelines.
• Real task: Route long-running queries away from latency-sensitive pools.

Concept explained simply

Governance for shared compute is a set of policies and guardrails that decide who can run what, where, and how much. Think: lanes on a highway. Fast lane for dashboards, a truck lane for big batch jobs, and rules to keep traffic flowing.

Core building blocks of shared compute governance

• Workload classes: Labels that describe intent (BI realtime, ELT batch, Ad‑hoc exploration, ML training).
• Resource pools/warehouses: Isolated compute groups with defined size, scaling rules, and concurrency limits.
• Admission control: Who can start a query, how many can run in parallel, and what gets queued or rejected.
• Priorities and routing: Map workloads to the right pool with appropriate priority.
• Limits and guardrails: Timeouts, memory/slot caps, row/byte scan caps, result size caps.
• Scheduling windows: When heavy jobs are allowed (e.g., nightly), and blackout periods.
• Cost governance: Quotas per team/project, daily caps, and alerts.
• Tagging and lineage: Tags for cost center, environment, data sensitivity, workload type.
• RBAC and least privilege: Who can use which pool and change its settings.
• Monitoring: Dashboards for queue times, run times, cancellations, errors, and spend; simple SLOs (e.g., p95 BI latency < 3s).

Worked examples

How to design your policy (step-by-step)

1. Inventory workloads: List top pipelines, BI dashboards, ad‑hoc users, and ML jobs. Capture SLOs and timing.
2. Define tiers (lanes): For example, BI-Fast, Adhoc-Lite, ELT-Batch, Batch-Overflow.
3. Set admission rules: Concurrency limits, queue behavior, and who can access each pool.
4. Add guardrails: Timeouts, scan/slot caps, and memory limits per workload class.
5. Plan scaling: Auto-suspend/auto-resume, min/max size, and scale-out triggers.
6. Establish cost controls: Daily quotas per pool/team and overage behavior (throttle/reroute).
7. Standardize tags: Enforce required tags on jobs and queries.
8. Create SLOs and alerts: e.g., BI p95 < 3s, ELT completes by 06:00. Alert on breaches and 80% budget use.
9. Document runbook: How to unblock queues, raise temp limits, and communicate incidents.
10. Review monthly: Adjust tiers, limits, and quotas based on usage and incidents.

Operational runbook (common tasks)

Exercises

Do these to practice. A quick test is available to everyone; only logged-in users get saved progress.

Exercise 1 — Classify and allocate

Scenario: You have three workloads: (A) Daily ELT from 01:00–05:00 needs to finish by 06:00, (B) BI dashboards 07:00–20:00 with p95 < 3s, (C) Ad‑hoc analysts weekdays 09:00–18:00 with a small budget.

• Create 3 pools with names, size/scale, concurrency, and timeouts.
• Define routing rules and daily quotas.
• Write 2 guardrails to protect BI.

Exercise 2 — Preemption and rerouting

Scenario: BI p95 latency degraded yesterday. Investigation shows several 10+ minute queries ran in the BI pool.

• Define a rule to auto-cancel or reroute long BI queries.
• Define a communication message to users.
• Propose a metric and alert to catch this early.

Common mistakes and self-checks

• Single giant pool for everything: Self-check—Do BI jobs share resources with ELT? If yes, separate lanes.
• No timeouts on BI: Self-check—Do any BI queries exceed 2–3 minutes? Add timeouts and scan caps.
• Hard budget stops: Self-check—Is work blocked mid-day? Use graceful degradation (slower pool) instead.
• Missing tags: Self-check—Can you attribute all spend to teams? If not, enforce required tags.
• Over-tuning concurrency: Self-check—Did queue times drop but errors rose? Rebalance concurrency vs stability.

Practical projects

• Project 1: Build a governance doc for your company with workload classes, pools, limits, quotas, and SLOs.
• Project 2: Create a daily dashboard showing queue time, p95 latency per pool, cancellations, and spend by tag.
• Project 3: Run a tabletop incident drill: BI slowdown during ELT. Apply your runbook and record outcomes.

Mini challenge

Design a two-tier ad‑hoc policy that encourages small, quick queries to finish instantly while pushing heavy exploration to a slower, queued pool—without blocking users. Write the rules in 5 bullet points.

Who this is for

• Data platform engineers managing multi-team warehouses.
• Analytics engineers who schedule pipelines and optimize dashboards.
• Team leads who need predictable performance and spend.

Prerequisites

• Basic understanding of data warehouses and query execution.
• Familiarity with RBAC and tagging/metadata.
• Ability to read platform metrics (latency, concurrency, queue time, spend).

Learning path

• Before: Warehouse sizing and auto-scaling fundamentals.
• This lesson: Governance for shared compute (workload classes, lanes, guardrails).
• Next: Advanced workload management and SLO monitoring.

Next steps

• Complete the exercises above.
• Take the quick test to check understanding. Everyone can take it; only logged-in users will see saved progress.
• Apply one guardrail (timeout or quota) in your environment this week and observe the impact.