Why this matters
Private connectivity and peering keep your systems fast, secure, and predictable. As a Platform Engineer, you will:
- Connect on-prem networks to cloud VPC/VNets using VPN or dedicated circuits.
- Peer networks across accounts/projects for internal service-to-service traffic.
- Expose managed services privately (e.g., databases, SaaS) without public IPs.
- Design for high availability, failover, and predictable routing with BGP.
- Troubleshoot outages: asymmetric routing, overlapping CIDRs, MTU issues.
Real tasks you might own
- Plan addressing and non-overlapping CIDRs for multiple environments.
- Deploy dual VPN tunnels with BGP and test failover.
- Set up PrivateLink/Private Service Connect equivalents to reach SaaS over private paths.
- Implement hub-and-spoke with transit to avoid N^2 peering.
- Document runbooks, test procedures, and rollback plans.
Concept explained simply
Private connectivity lets two networks talk without traversing the public internet. Peering allows two private networks to exchange traffic directly. In cloud, this often means:
- Site-to-site VPN (IPsec) over the internet but encrypted.
- Dedicated circuits via a provider backbone (e.g., direct connections/interconnects) for stable latency and higher SLAs.
- VPC/VNet peering for private routing within and across accounts/projects.
- Private service endpoints (e.g., PrivateLink/Private Service Connect) to access specific services privately.
Jargon buster
- BGP: Dynamic routing protocol to exchange prefixes and path preferences.
- ASN: Autonomous System Number used by BGP to identify routing domains.
- CIDR: IP addressing blocks (e.g., 10.0.0.0/16).
- Transit: A hub that forwards traffic between spokes and external networks.
- MTU: Maximum Transmission Unit; mismatches cause fragmentation or drops.
Mental model
Imagine offices (networks) connected by private hallways (peering) or a leased private tunnel (dedicated circuit). BGP is the receptionist updating who can reach whom. Transit hubs are central lobbies that connect many offices without everyone building hallways to everyone else.
Core building blocks and decisions
- Address planning: Non-overlapping CIDRs across environments and clouds.
- Connectivity type: VPN (quick, flexible) vs dedicated circuit (stable, higher throughput, SLA).
- Routing: Static for simple cases; BGP for resilience, scale, and controlled advertisement.
- Topology: Peering mesh vs hub-and-spoke via transit gateway/router.
- HA patterns: Dual devices, dual tunnels/ports, dual locations, fast detection (e.g., BFD).
- Security: Private endpoints, security groups/firewalls, route filters, least privilege prefixes.
- Performance: Throughput per tunnel/port, MTU settings, jumbo frames consistency.
- Operations: Health checks, runbooks, change windows, testing failover regularly.
Worked examples
Example 1: On-prem to cloud with redundant VPN and BGP
- Plan: Choose two cloud VPN gateways in different zones. Allocate non-overlapping CIDRs. Assign ASN for on-prem and cloud.
- Configure: Establish two IPsec tunnels, enable BGP on both. Advertise on-prem prefixes; accept cloud prefixes.
- Prefer path: Use BGP local-pref/AS-path or metrics so tunnel A is primary, B is standby.
- Test: Ping across, then disable tunnel A and confirm failover to B within seconds.
- Document: Record timers, expected convergence, and rollback steps.
Outcome: Encrypted, resilient private path with dynamic routing.
Example 2: VPC/VNet peering across accounts
- Check CIDRs: Ensure no overlap between 10.10.0.0/16 and 10.20.0.0/16.
- Create peering: Request/accept peering in both accounts/projects.
- Route tables: Add routes to send 10.20.0.0/16 via peering in VPC A and vice versa.
- Security: Update firewall rules/security groups to allow specific ports between subnets.
- Validate: Traceroute shows direct peering path, not NAT or internet.
Note: Many cloud peerings are non-transitive. Traffic cannot automatically hop through a peered network to a third network without transit.
Example 3: Private access to a managed database or SaaS
- Choose private endpoint: Use the provider’s private service feature to expose the service inside your VPC/VNet.
- DNS: Create a private DNS zone that maps service name to the private endpoint.
- Restrict: Only allow required subnets to reach the endpoint; block public access.
- Test: From an app subnet, resolve DNS to a private IP and connect. Confirm no egress to public internet.
Outcome: Internal-only access to the service, reducing exposure.
Hands-on practice
Use these cloud-agnostic steps with any provider or lab environment.
- Plan addressing: Write down three CIDRs for dev, stage, prod with no overlap.
- Design topology: Sketch hub-and-spoke with a transit hub and two spokes. Mark routes you expect to see.
- Define routing: Specify which prefixes each side should advertise via BGP (or static routes if BGP unavailable).
- HA strategy: Decide dual tunnels/ports and failover timers. Write your failover test plan.
- DNS plan: Define private zones for internal service names and how clients resolve them.
Practice checklist
- Non-overlapping CIDRs documented.
- Primary and secondary paths identified.
- Explicit list of advertised prefixes.
- Firewall/security group rules mapped to flows.
- Failover test steps written and time-to-recover target set.
- Roll-back steps prepared.
Exercises
Do the exercises below. Open the hints if you get stuck, then compare with the sample solution.
Exercise 1: Design a resilient private connectivity plan
See the Exercises panel for full instructions. Deliverables: diagram, route advertisements, HA plan, and test procedure.
Exercise 2: Troubleshoot asymmetric routing in peering
See the Exercises panel for full instructions. Deliverables: root cause, corrected routes/rules, and verification steps.
Common mistakes and self-check
- Overlapping CIDRs: Causes blackholes. Self-check: List all CIDRs; ensure uniqueness.
- Assuming transitive peering: Many peerings are non-transitive. Self-check: Trace paths; ensure transit is configured if needed.
- No HA: Single tunnel/port becomes a SPOF. Self-check: Simulate failure and measure recovery.
- Missing return routes: Forward path works, return path fails. Self-check: Verify routes in both directions.
- Firewall gaps: Routes exist, but ports blocked. Self-check: Document required ports; test with simple allow rules first.
- MTU mismatch: Drops for large packets. Self-check: Path MTU discovery or set consistent MTU/Jumbo frames end-to-end.
- DNS not private: Clients resolve public endpoints. Self-check: Confirm private DNS zones and resolution order.
Self-check mini-quiz
- Can you explain which prefixes are advertised on each link?
- Do you know your failover time target and how to test it?
- Which security controls restrict lateral movement across peers?
Practical projects
- Build a hub-and-spoke lab: two spokes, one hub, with simulated BGP and failover test plan.
- Private service exposure: set up a private endpoint to a managed DB and verify no internet path is used.
- Runbook: write a troubleshooting guide for "cannot reach service over peering" with commands, checks, and escalation steps.
Mini challenge
Your company acquires a startup using 10.0.0.0/16, which overlaps with your 10.0.0.0/16. You must connect them privately within two weeks. Propose a plan that avoids renumbering now but allows eventual migration. Include NAT domains, route filters, and staged cutover steps.
What a solid answer includes
- Short-term NAT at the edge for overlapping subnets with strict prefix filters.
- Documented mapping for critical services only (not full mesh).
- Plan to migrate one side to a new CIDR with phased DNS updates.
- Rollback and risk notes.
Who this is for
- Platform and DevOps engineers connecting on-prem, cloud, and managed services.
- Backend engineers needing reliable, private service-to-service connectivity.
- SREs responsible for performance, reliability, and incident response.
Prerequisites
- Basic IP networking (subnets, routing, firewalls).
- Comfort with at least one cloud’s VPC/VNet and route tables.
- Familiarity with change management and testing.
Learning path
- Before: IP addressing, routing basics, security groups/firewalls.
- Now: Private connectivity and peering design, HA, DNS, and troubleshooting.
- Next: Transit architectures, network policy as code, multi-region design.
Next steps
- Complete the exercises and the quick test below.
- Automate your lab with IaC (modules for VPN/peering, routes, and DNS).
- Schedule quarterly failover drills and MTU validation.
Quick Test
Anyone can take the test. Logged-in learners get progress saved automatically.