Scripting Python or JS

Why this matters for Data Visualization Engineers

Scripting in Python or JavaScript turns static charts into living products: pull data, transform it, encode it into visuals, add interactions, export results, and automate refreshes. You’ll bridge raw data to stakeholder-ready insights, whether embedded in the browser (JS) or rendered on servers and notebooks (Python).

Who this is for

• Analysts and BI professionals moving into interactive visualization work.
• Frontend-leaning folks who want to connect charts to real data.
• Data Viz Engineers who need reproducible scripts and automation.

Prerequisites

• Comfort with basic programming concepts: variables, functions, conditionals, loops.
• Basics of data types (string, number, boolean, array/list, object/dict).
• Understanding of chart encodings (x/y, color, size) and when to use them.

Learning path (milestones)

1. Fetch data: Read JSON/CSV from APIs or databases. Handle auth and errors.
2. Wrangle: Clean types, filter, group, aggregate, and shape for charts.
3. Chart: Map fields to encodings. Build bars/lines/scatters and tooltips.
4. Library mastery: Learn one stack deeply (e.g., Python + matplotlib/Plotly or JS + D3/Chart.js).
5. Export/share: Save PNG/SVG/HTML; version outputs with clear naming.
6. Reusable components: Parameterize chart templates for consistency.
7. Debugging: Diagnose data issues, rendering glitches, and performance problems.
8. Automation: Script refreshes on schedule or data change.

Worked examples

Example 1 — Fetch and wrangle (Python)

import requests
import pandas as pd

url = "https://example.com/api/sales"  # placeholder
res = requests.get(url, timeout=15)
res.raise_for_status()
raw = res.json()

# Normalize and wrangle
 df = pd.json_normalize(raw)
 df["amount"] = pd.to_numeric(df["amount"], errors="coerce").fillna(0)
 df["date"] = pd.to_datetime(df["date"], errors="coerce")
 monthly = (
     df[df["date"].notna()]
     .groupby(pd.Grouper(key="date", freq="M"))["amount"]
     .sum()
     .reset_index()
)
print(monthly.head())

Example 2 — Simple bar chart (Python, matplotlib)

import matplotlib.pyplot as plt
import pandas as pd

data = pd.DataFrame({"product": ["A","B","C"], "revenue": [120, 80, 160]})
fig, ax = plt.subplots(figsize=(5,3))
ax.bar(data["product"], data["revenue"], color="#3b82f6")
ax.set_title("Revenue by Product")
ax.set_xlabel("Product"); ax.set_ylabel("Revenue")
for i, v in enumerate(data["revenue"]):
    ax.text(i, v + 3, str(v), ha='center', fontsize=9)
plt.tight_layout()
plt.savefig("revenue_by_product.png", dpi=150, bbox_inches="tight")
plt.close()

Example 3 — Fetch and render (JavaScript + Chart.js)

async function loadAndRender() {
  const res = await fetch("/data/sales.json");
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  const data = await res.json();
  const byRegion = {};
  data.forEach(r => {
    const region = r.region;
    const amt = Number(r.amount) || 0;
    byRegion[region] = (byRegion[region] || 0) + amt;
  });
  const labels = Object.keys(byRegion);
  const values = labels.map(k => byRegion[k]);
  const ctx = document.getElementById("chart").getContext("2d");
  new Chart(ctx, {
    type: "bar",
    data: { labels, datasets: [{ label: "Amount", data: values, backgroundColor: "#10b981" }] },
    options: { responsive: true, plugins: { legend: { display: false } } }
  });
}
loadAndRender().catch(console.error);

Add a canvas element with id="chart" in your HTML where this runs.

Example 4 — D3.js line chart with tooltip (JS)

// Assumes an SVG element with id="vis" exists
const svg = d3.select("#vis");
const width = 600, height = 300, margin = {top:20,right:20,bottom:30,left:40};
const innerW = width - margin.left - margin.right;
const innerH = height - margin.top - margin.bottom;
svg.attr("width", width).attr("height", height);
const g = svg.append("g").attr("transform", `translate(${margin.left},${margin.top})`);

const data = [
  {date: new Date("2025-01-01"), value: 10},
  {date: new Date("2025-02-01"), value: 15},
  {date: new Date("2025-03-01"), value: 9}
];

const x = d3.scaleTime().domain(d3.extent(data, d => d.date)).range([0, innerW]);
const y = d3.scaleLinear().domain([0, d3.max(data, d => d.value)]).nice().range([innerH, 0]);

const line = d3.line().x(d => x(d.date)).y(d => y(d.value));

g.append("g").attr("transform", `translate(0,${innerH})`).call(d3.axisBottom(x));
g.append("g").call(d3.axisLeft(y));
g.append("path").datum(data).attr("fill","none").attr("stroke","#ef4444").attr("stroke-width",2).attr("d", line);

const tooltip = d3.select("body").append("div").style("position","absolute").style("padding","4px 6px").style("background","#111").style("color","#fff").style("font-size","12px").style("border-radius","4px").style("display","none");

g.selectAll("circle").data(data).enter().append("circle")
  .attr("cx", d => x(d.date)).attr("cy", d => y(d.value)).attr("r", 3).attr("fill", "#ef4444")
  .on("mouseenter", (evt, d) => { tooltip.style("display","block").text(`${d.date.toISOString().slice(0,10)}: ${d.value}`); })
  .on("mousemove", evt => { tooltip.style("left", (evt.pageX + 8)+"px").style("top", (evt.pageY - 24)+"px"); })
  .on("mouseleave", () => tooltip.style("display","none"));

Example 5 — Lightweight automation (Python)

import time
import pathlib

# Dummy function that re-renders your charts
from my_charts import render_all  # replace with your actual module

DATA_PATH = pathlib.Path("./data/source.csv")
last_mtime = None
while True:
    try:
        mtime = DATA_PATH.stat().st_mtime
        if last_mtime is None or mtime != last_mtime:
            print("Data changed. Rebuilding visuals...")
            render_all()
            last_mtime = mtime
        time.sleep(15)  # poll for changes
    except KeyboardInterrupt:
        break
    except Exception as e:
        print("Error:", e)
        time.sleep(5)

For production, use an OS scheduler to run your script at reliable intervals.

Drills and exercises

• Fetch JSON and CSV, then validate shapes: rows, columns, and null counts.
• Convert strings to numbers/dates; assert expected ranges.
• Build one bar, one line, and one scatter chart with tooltips.
• Export the same chart as PNG, SVG, and HTML.
• Turn a one-off chart into a function with parameters (title, colors, fields).
• Add a debounce to window resize; confirm no visual jank.
• Implement a try/except (Python) or try/catch (JS) around fetching and parsing.

Common mistakes and debugging tips

• Skipping input validation: Always check types and nulls before plotting.
• Assuming API success: Verify HTTP status; handle non-2xx gracefully.
• Mixed scales: Set domain/range explicitly; use nice() for legible ticks.
• Over-encoding: Limit encodings to what the audience needs; prioritize readability.
• Stateful spaghetti: Extract pure functions; pass data and config in, return elements/figures out.
• Silent failures: Log intermediate shapes (len, head), and catch/print exceptions with context.
• Performance: Downsample large datasets for initial render; add progressive detail on demand.

Mini project: KPI trends with refreshable exports

Goal: Build a monthly KPI line chart with a 3-month moving average, export as PNG (Python) or downloadable image (JS), and make it refresh when data changes.

1. Load data from an API or CSV. Validate columns and parse dates.
2. Aggregate to monthly totals; add moving average.
3. Render line chart with tooltips and clear axes/labels.
4. Export: PNG (Python savefig) or client-side image export (JS canvas toDataURL).
5. Automation: Rebuild when data changes (polling loop or scheduled run).
6. Deliverable: A script and an output folder with timestamped images.

Next steps

• Adopt a lint/format routine and basic tests for your data transforms.
• Package your chart functions into a small internal library/module.
• Integrate with a dashboard or reporting pipeline so stakeholders can self-serve.

Subskills