Question

Difficulty: Very hardModernizing Workloads using Containers (ECS and EKS)

VeloDynamics Manufacturing is modernizing its transaction-clearing microservices by migrating them from on-premises virtual machines to Amazon EKS in a multi-account AWS environment. The EKS worker nodes must be deployed in a highly constrained VPC where the primary subnet allocated for nodes is a single /24/24 block (10.100.1.0/2410.100.1.0/24), which cannot be expanded due to tight integration with the corporate Transit Gateway mesh. The microservices must scale up to 1,0001,000 concurrent pods during peak clearing cycles. The pods must communicate with on-premises mainframe databases over AWS Direct Connect via an AWS Transit Gateway. The on-premises firewalls and routing tables are strictly managed and will only route traffic originating from the node primary subnet (10.100.1.0/2410.100.1.0/24); adding routes for new CIDR blocks to the on-premises network is prohibited. Additionally, client applications in other VPCs must connect to the services via an Application Load Balancer (ALB) with direct routing to pods (minimal network hops) to meet latency SLAs. Which of the following architectures meets these requirements with the least operational overhead?

  1. Associate a secondary CIDR block with the VPC and create subnets in this range. Set the AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG environment variable to true in the VPC CNI DaemonSet, and define ENIConfig custom resources mapping to the secondary subnets to allocate pod IPs. Keep the AWS_VPC_K8S_CNI_EXTERNALSNAT environment variable set to false to ensure the CNI performs source NAT using the worker nodes' primary IPs for outbound traffic. Deploy the AWS Load Balancer Controller and configure the Ingress with the target type set to ip.Answer
  2. B
    Associate a secondary CIDR block with the VPC and create subnets in this range. Configure the EKS cluster with Calico in overlay mode using VXLAN encapsulation to assign pod IPs from the secondary range. Set up the AWS Load Balancer Controller with the target type set to ip to enable direct routing. Configure AWS Transit Gateway to route the overlay network CIDR block to the Direct Connect gateway, and configure the on-premises routers to accept the encapsulated packet headers.
  3. C
    Associate a secondary CIDR block with the VPC and create subnets in this range. Set the AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG environment variable to true in the VPC CNI DaemonSet, and define ENIConfig custom resources mapping to the secondary subnets. Set the AWS_VPC_K8S_CNI_EXTERNALSNAT environment variable to true to allow the pods to bypass NAT and communicate directly with on-premises databases using their secondary IPs. Deploy the AWS Load Balancer Controller and configure the Ingress with the target type set to instance.
  4. D
    Deploy EKS worker nodes on AWS Outposts using the AWS Fargate launch type to scale the pods. Configure the Fargate pod execution role to inherit permissions from the node group. Configure the tasks to use host networking mode to share the underlying host network namespace, bypassing the primary subnet's IP limitations. Deploy an Application Load Balancer and configure target groups using instance mode.

Answer

Associate a secondary CIDR block with the VPC and create subnets in this range. Set the AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG environment variable to true in the VPC CNI DaemonSet, and define ENIConfig custom resources mapping to the secondary subnets to allocate pod IPs. Keep the AWS_VPC_K8S_CNI_EXTERNALSNAT environment variable set to false to ensure the CNI performs source NAT using the worker nodes' primary IPs for outbound traffic. Deploy the AWS Load Balancer Controller and configure the Ingress with the target type set to ip.
The correct architecture uses Amazon VPC CNI Custom Networking. This separates node IP allocation from pod IP allocation. EKS nodes run in the primary 10.100.1.0/2410.100.1.0/24 subnet, while pods receive IPs from a secondary VPC CIDR block. Because the secondary CIDR block resides within the VPC, the pod IPs are fully routable inside AWS, permitting the AWS Load Balancer Controller to route traffic directly to the pods using the 'ip' target type, satisfying the latency SLA. By keeping 'AWS_VPC_K8S_CNI_EXTERNALSNAT' set to 'false', the CNI translates the source IP of any traffic leaving the VPC to the primary IP of the host node. This ensures that all traffic targeting the on-premises mainframe appears to originate from the whitelisted 10.100.1.0/2410.100.1.0/24 range, complying with firewall constraints.

Step-by-Step Solution

1
Address the IP exhaustion in the primary /24/24 subnet by associating a secondary CIDR block with the VPC.
The VPC has a new, larger CIDR block (e.g., 100.64.0.0/16100.64.0.0/16) dedicated for pod allocations.
The primary node subnet 10.100.1.0/2410.100.1.0/24 has only 251 usable IP addresses, which cannot accommodate the 1,0001,000 pods required during peak times.
2
Configure the Amazon VPC CNI plugin to use custom networking by setting AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG to true and creating ENIConfig custom resources for the secondary subnets.
EKS worker nodes are provisioned in the primary subnet (10.100.1.0/2410.100.1.0/24), but their pods are assigned IPs from the secondary CIDR block.
This decouples pod IP allocation from the host node subnets, enabling scaling without exhausting the primary subnet's IPs.
3
Configure AWS_VPC_K8S_CNI_EXTERNALSNAT to false (default behavior) in the CNI daemonset.
The VPC CNI automatically performs Source NAT (SNAT) for any outbound traffic originating from pods destined for addresses outside the VPC CIDR.
This translates the pod's secondary source IP (e.g., 100.64.x.y100.64.x.y) to the node's primary IP (10.100.1.x10.100.1.x) when routing over the Transit Gateway to the on-premises mainframe, matching the firewall whitelist.
4
Deploy the AWS Load Balancer Controller and configure target groups with target-type set to ip.
The Application Load Balancer routes traffic directly to the pod IP addresses within the VPC secondary CIDR block.
Since the secondary CIDR is native VPC space, the ALB can target the pods directly, minimizing network hops and routing latency.

Key Concept

Amazon EKS VPC CNI custom networking allows worker nodes to reside in a primary subnet while pods are allocated IPs from a secondary CIDR block. By keeping external SNAT disabled, outbound traffic to external networks (like on-premises) is translated to the node's IP, bypassing on-premises routing restrictions while maintaining VPC-routable pod IPs for direct ALB integration.
Rate this question