backend / networking / 09_vpc_cloud_networking.md

VPC and Cloud Networking

6 interview angles 6 min read source

VPC and Cloud Networking

A VPC (Virtual Private Cloud) is a logically isolated network within a cloud provider — your private slice of their datacenters with its own IP range, subnets, route tables, and gateways. AWS has VPC, Azure has VNet, GCP has VPC. The concepts are the same; the names of components differ.

The pieces

VPC 10.0.0.0/16
├── Subnets — partitioned IP ranges, each tied to one AZ
├── Route Tables — "to reach X, send via Y"
├── Internet Gateway (IGW) — door to the public Internet
├── NAT Gateway — egress for private subnets without giving them public IPs
├── Security Groups — instance-level stateful firewall (see 07_firewalls_security_groups.md)
├── NACLs — subnet-level stateless firewall
├── VPC Endpoints — private connection to AWS services (S3, DynamoDB, ...)
├── VPC Peering — connect two VPCs (must have non-overlapping CIDRs)
└── Transit Gateway — hub that connects many VPCs + on-prem

Public vs private subnets — the distinction

A subnet is “public” if its route table has a route 0.0.0.0/0 → IGW. That’s it. There’s no flag, no checkbox.

Public subnet Private subnet
Default route IGW NAT Gateway (or none)
Instances can have public IPs? yes no (or yes-but-useless without IGW)
Reachable from Internet? yes (if SG allows) no
Used for load balancers, NAT GWs, bastion app servers, databases, queues

Typical 3-tier VPC

VPC 10.0.0.0/16
├── Public  10.0.1.0/24   AZ-a   ← ALB, NAT-GW-a
├── Public  10.0.2.0/24   AZ-b   ← ALB, NAT-GW-b
├── App     10.0.10.0/24  AZ-a   ← EC2/ECS/EKS, default route → NAT-GW-a
├── App     10.0.11.0/24  AZ-b   ← default route → NAT-GW-b
├── Data    10.0.20.0/24  AZ-a   ← RDS, Elasticache, no default route
└── Data    10.0.21.0/24  AZ-b

Route tables:

Subnet Routes
Public 10.0.0.0/16 local, 0.0.0.0/0 → IGW
App 10.0.0.0/16 local, 0.0.0.0/0 → NAT-GW (this AZ)
Data 10.0.0.0/16 local (no default route — no Internet at all)

Security groups handle the rest (sg-web → sg-app → sg-db).

Why per-AZ NAT GWs? Cross-AZ traffic costs $$ and an AZ outage shouldn’t take down the other AZ’s egress.

Internet Gateway (IGW)

A horizontally-scaled, highly-available VPC component that allows public-IP traffic to/from the Internet. Free; one per VPC. Without an IGW attached and a route to it, no instance can reach the Internet — not even ones with public IPs.

NAT Gateway

Lets private-subnet instances make outbound connections to the Internet without exposing inbound. Sits in a public subnet, has an Elastic IP, SNATs traffic. See 04_nat.md.

Cost: ~$0.045/hr per gateway + $0.045/GB processed. For a service that pulls 1 TB from the Internet daily, that’s ~$45/day — meaningful at scale.

VPC Endpoints — bypass NAT for AWS services

Two types:

Type What Use case
Gateway endpoint route entry, free S3, DynamoDB
Interface endpoint (PrivateLink) ENI in your subnet, ~$0.01/hr + $0.01/GB most other AWS services (Secrets Manager, ECR, Lambda, …)

Adding s3 gateway endpoint means private subnets can aws s3 cp without traversing NAT GW. For an app pulling lots of data from S3, this saves the NAT cost entirely.

VPC Peering

Connect two VPCs so their instances see each other on private IPs.

Constraints:

  • CIDRs cannot overlap. This is the #1 reason peering fails. Plan unique ranges day 1.
  • Not transitive: A peered with B and B peered with C does not let A reach C. Use Transit Gateway for hub-and-spoke.
  • Cross-region peering exists but adds latency.

Transit Gateway (TGW)

Hub-and-spoke replacement for many peerings. Up to ~5,000 VPCs attached, plus VPN/Direct Connect, with central route tables.

                 ┌── VPC-prod
TGW ──────┬──── VPC-staging
          ├──── VPC-shared-services
          └──── on-prem (via VPN or Direct Connect)

Costs more than peering but scales. Use TGW once you have >3 VPCs to interconnect.

DNS inside a VPC

AWS provides a default resolver at VPC_CIDR_BASE+2 (e.g. 10.0.0.2) that can resolve:

  • Internal hostnames you create with Route 53 private hosted zones.
  • Public DNS via the standard chain.

EC2 instances get an automatic internal name like ip-10-0-1-42.ec2.internal. RDS endpoints, ELB endpoints, etc. are public DNS names that resolve to private IPs when queried from inside the VPC.

Connectivity to on-prem / other clouds

Option Throughput Use
Site-to-Site VPN up to ~1.25 Gbps per tunnel quick, encrypted, fine for low-throughput
Direct Connect 1, 10, 100 Gbps dedicated high-throughput, lower latency, no Internet
Direct Connect + VPN combo encryption over the dedicated link

See 08_vpn.md.

VPC Flow Logs

Per-flow records of accepted/rejected traffic. Goes to CloudWatch / S3 / Kinesis. Schema includes source/dest IP+port, protocol, action (ACCEPT/REJECT), bytes.

Use for:

  • Debugging “why can’t A talk to B” — find the REJECT.
  • Security forensics — what did the compromised instance contact.
  • Cost analysis — who’s pulling the most cross-AZ traffic.

Other clouds

AWS Azure GCP
VPC Virtual Network (VNet) VPC
Subnet (per-AZ) Subnet (per-region; AZ-aware) Subnet (regional)
Internet Gateway implicit implicit
NAT Gateway NAT Gateway Cloud NAT
Security Group Network Security Group (NSG) Firewall Rule
Route Table Route Table Route
VPC Peering VNet Peering VPC Network Peering
Transit Gateway Virtual WAN Network Connectivity Center

GCP VPCs are global by default (subnets in different regions, one VPC) — different mental model from AWS’s regional VPCs.

Common interview confusions

  • “My EC2 has a public IP, why can’t it reach the Internet?” — public IP without an IGW route is useless. Check route table has 0.0.0.0/0 → igw-....
  • “My private-subnet instance can reach the NAT GW but still can’t reach the Internet.” — NAT GW without IGW in its public subnet doesn’t help. NAT GW uses IGW.
  • “I’ll just peer all my VPCs.” — peering doesn’t transit. Three VPCs need three peerings (a triangle); ten VPCs need 45. Use TGW.
  • “Subnets are AZ-redundant.” — a subnet lives in one AZ. For HA, create one subnet per AZ and put your workload in both.

Interview angle

  • “What’s a VPC and why do you need one?” — isolated virtual network in the cloud with its own IP range, subnets, routing, and firewall — your private boundary inside the provider’s shared infrastructure.
  • “Public vs private subnet?” — public has a default route to the IGW (instances can have a public-Internet path); private doesn’t (egress via NAT GW or none).
  • “How do you make the VPC HA?” — multiple AZs, a subnet per AZ per tier, NAT GW per AZ, multi-AZ for managed services (RDS, ElastiCache).
  • “When would you use VPC Endpoints?” — to reach AWS services (S3, DynamoDB) without traversing the NAT Gateway / Internet — saves cost, improves security, lower latency.
  • “VPC peering vs Transit Gateway?” — peering is 1:1 and non-transitive (good for 2-3 VPCs); TGW is hub-and-spoke that scales to thousands of attachments and supports VPN/Direct Connect.
  • “Walk me through troubleshooting ‘app server can’t reach RDS’.” — SG ingress on DB → SG egress on app → NACLs both directions → route table → DB listening + auth. Use VPC Flow Logs to find the REJECT.