Terraform and CI/CD

I am proficient in Terraform, and find it to be the best declarative language for infrastructure, and much simpler than the likes of Cloudformation and Azure Resource Manager templates. A personal project I created in Terraform to deploy a kubernetes stack in AWS (as part of training for my CKA) can be found in GitHub. I decided to build it using only EC2 services, rather than using EKS, so I could better learn to bootstrap a cluster. I also used Packer to create a Debian 12 image, as there was not a publically (and officially) image available at the time. I chose Debian simply because it is my favourite Linux distribution for personal use, and I am most familiar with it. For enterprise solutions, I would likely choose a RHEL-based distro, like Rocky Linux, or RHEL itself.

A packer build in progress. apt update is run to update the image to its packages to the latest version available in the debian 12 repositories.

To write the terraform, first I had to explore the best methods for bootstrapping a Kubernetes cluster. I decided to use kubeadm, as it is something I must know about for the CKA certification, but also because it is an easy tool to use and still allows fairly comprehensive configuration of the built cluster.

The Terraform does the following:

Creates a launch template specifying the debian 12 image and the cloud-config files, which contain commands to set up the instances and install tools required to join or initialise the cluster. I did this instead of installing the tools onto the AMI because it is much quicker to make changes, which I found valuable as it allowed me to "tinker" with the cluster more quickly (without needing to rebuild the AMI and then deploy it).

Deploys the autoscaling groups for the control plane and the worker nodes.

Sets security groups on these nodes to ensure they can only communicate internally.

Deploys a load balancer to balance traffic between control plane nodes.

Deploys an Internet Gateway to allow communication with the outside world, for kubernetes API access (secured by X.509 using a certificate, I am working on OIDC authentication!).

A completed Terraform apply; the cluster is now deployed.

After the cluster is deployed, I use some simple (bash) commands to configure kubectl with credentials retreived via SSH from a control plane node.

#!/bin/bash ip=$(for id in $(aws autoscaling describe-auto-scaling-instances | yq '.AutoScalingInstances[] | select(.AutoScalingGroupName == "control_plane_nodes") | .InstanceId'); do aws ec2 describe-instances --instance-ids $id | yq '.Reservations[].Instances[].PublicIpAddress'; break done) ssh -i ~/.ssh/jumpbox_rsa admin@$ip "sudo cat /etc/kubernetes/admin.conf" > ~/.kube/config

Systems & Networking

Hobbies & Interests