Networking Vpc And Security Groups Basics

Why this matters

As a Machine Learning Engineer, you launch training machines, move data from object storage, and expose model endpoints. Getting VPCs, subnets, and security groups right means your work is reachable when needed and locked down when not. A few well-chosen rules prevent data leaks, runaway costs, and downtime.

• Spin up a secure training VM with Jupyter for teammates only.
• Train models in private subnets while still downloading packages and datasets safely.
• Expose an HTTPS inference endpoint without opening your whole network.

Note: The quick test is available to everyone; only logged-in users get saved progress.

Concept explained simply

A Virtual Private Cloud (VPC) is your private neighborhood in the cloud. Subnets are streets in that neighborhood. Security groups are your apartment door locks allowing visitors on certain ports. A route table tells cars (packets) where to go. Internet Gateways let traffic reach the internet; NAT lets private machines go out without being directly reachable from outside.

Core pieces you'll use

• VPC: An isolated network you control (IP ranges via CIDR like 10.0.0.0/16).
• Subnets: Slices of your VPC in specific availability zones. Public subnets have a route to an Internet Gateway; private subnets do not.
• Route tables: Map destinations to targets (e.g., 0.0.0.0/0 to an Internet Gateway for public subnets; to a NAT for private outbound).
• Internet Gateway (IGW): Allows inbound/outbound internet for resources with public IPs.
• NAT (Gateway/Instance): Lets private instances access the internet for downloads/updates while staying non-public.
• Security groups (SGs): Instance-level, stateful firewalls. Allow rules are enough; responses are automatic.
• Network ACLs (NACLs): Subnet-level, stateless. Optional extra guardrails; must allow both directions explicitly.
• VPC endpoints: Private links to cloud services (like object storage) without internet traversal.
• Peering/Private links: Connect VPCs or services privately across networks.

Worked examples

Example 1: Secure Jupyter on a single VM

1. Create a VPC (e.g., 10.0.0.0/16) and a public subnet (10.0.1.0/24).
2. Add an Internet Gateway and a route 0.0.0.0/0 to the IGW in the subnet’s route table.
3. Give the VM a public IP.
4. Create a security group:
   • Inbound: TCP 22 (SSH) from your office/home IP only (e.g., 203.0.113.5/32).
   • Inbound: TCP 8888 (Jupyter) from your home IP only.
   • Outbound: allow all or at least TCP 443 (HTTPS).
5. Run Jupyter—only your IP can access it. Everyone else is blocked.

Example 2: Private training cluster with data access

1. VPC 10.0.0.0/16 with:
   • Public subnet (10.0.1.0/24) for NAT.
   • Private subnets (10.0.2.0/24, 10.0.3.0/24) for training nodes.
2. Attach IGW; create a NAT in the public subnet.
3. Route tables:
   • Public subnet: 0.0.0.0/0 → IGW.
   • Private subnets: 0.0.0.0/0 → NAT.
4. Security groups for training nodes:
   • Inbound: allow only needed intra-cluster ports (e.g., TCP 22 from a bastion SG or SSM alternative; framework-specific ports from the cluster’s SG).
   • Outbound: TCP 443 for package mirrors and object storage.
5. Add a VPC endpoint for object storage to keep data pulls on private links (optional but recommended).

Example 3: Expose a model endpoint safely

1. Put inference instances in private subnets behind an internal load balancer (or in public subnets behind a public load balancer, but instances themselves can still be private).
2. Security group on load balancer:
   • Inbound: TCP 443 from 0.0.0.0/0 (public) or from your API gateway/CDN IP ranges only.
3. Security group on instances:
   • Inbound: TCP 443 from the load balancer’s security group (not from 0.0.0.0/0).
   • Outbound: minimal egress (e.g., DB/object storage endpoints).
4. No direct public IPs on instances; only the load balancer is exposed.

Exercises

Complete the two exercises below, then take the quick test. The quick test is available to everyone; only logged-in users get saved progress.

Exercise 1 — Design a security group for a training VM running Jupyter

Scenario: You will run Jupyter on port 8888 on a single VM in a public subnet. You need SSH for admin and outbound HTTPS for package installs.

• Requirement 1: Allow Jupyter (8888) only from your home IP (replace with e.g., 203.0.113.5/32).
• Requirement 2: Allow SSH (22) only from your company office CIDR (e.g., 198.51.100.0/24).
• Requirement 3: Allow outbound HTTPS (443) to anywhere.
• Everything else should be blocked by default.

Expected output: A list of inbound and outbound rules that satisfy the constraints.

Exercise 2 — Give a private subnet internet egress for training

Scenario: Your training nodes live in 10.0.2.0/24 (private). They must download packages and data from the internet but must not be reachable from the internet.

• Requirement 1: Use a NAT in a public subnet to enable outbound from private subnets.
• Requirement 2: Route 0.0.0.0/0 in private subnet route tables to the NAT.
• Requirement 3: Keep instances without public IPs.
• Requirement 4: Create SG rules to allow outbound HTTPS (443) and block inbound from public sources.

Expected output: A short architecture description covering subnets, routes, NAT, and security group directions.

Common mistakes

• Opening SSH or Jupyter to 0.0.0.0/0. Fix: restrict to your IP, office range, or a bastion/SSM method.
• Confusing NAT with IGW. NAT is for private outbound; IGW is public exposure.
• Allowing load balancer traffic from the internet directly on instance SG. Fix: allow from the load balancer’s SG only.
• Forgetting DNS egress. Fix: permit DNS to your resolver (often automatic via VPC, but confirm).
• Overusing NACLs when SGs suffice. Keep NACLs simple (default allow) unless you need subnet-level deny rules.

Who this is for

• ML Engineers deploying training or inference on cloud VMs/containers.
• Data Scientists experimenting with notebooks who want safe access patterns.
• MLOps/Platform engineers standardizing secure network defaults.

Prerequisites

• Basic cloud account familiarity (instances, regions/zones).
• Comfort with IP/CIDR basics (what 10.0.0.0/16 means).
• Knowing your team’s source IP ranges for access control.

Learning path

1. Understand VPC, subnets, route tables.
2. Add IGW for public subnets and NAT for private egress.
3. Create least-privilege security groups for admin, training, inference.
4. Use endpoints/peering for private data access.
5. Harden with monitoring and periodic rule reviews.

Practical projects

• One-click secure Jupyter: script a VM + SG that only you can reach on 8888.
• Private training cluster: autoscaling workers in private subnets with NAT egress.
• Hardened inference: instances behind a load balancer; SG allows 443 from LB SG only; outbound to storage/DB endpoints.

Next steps

• Automate these patterns with IaC (e.g., templates) to make secure-by-default your team’s norm.
• Add monitoring: alerts when SGs include 0.0.0.0/0 on admin ports.
• Rotate and audit: periodically review rules and tighten where possible.

Mini challenge

Sketch a small VPC for a team of 3 ML engineers: one public subnet for a bastion or NAT, two private subnets for training and inference, an endpoint for object storage, and SGs that keep admin and app traffic separated. Write down each rule in one line and state why it exists.