Monday, April 6, 2020

AWS AMI vs Launch Templates

Very often I get asked about the differences between AMI (Amazon Machine Images) and EC2 Launch Templates, although both of them have different purpose. This blog is about getting these concepts clear.

What is AMI?


There might be a requirement where we need to install softwares and applications on hundreds of EC2s. It's not practically possible to login to each of the EC2 and perform the task as it is time consuming and also prone to errors. We can automate these tasks using AMIs. Below is the workflow to start working with the AMIs.


Once an EC2 has been created, the appropriate software/applications have to be installed along with the configurations and then an AMI has to be created. The AMI has to OS and all the software/applications on top of it. The AMI usually is a couple of GB, based on the original EC2 which was created and is immutable (no changes can be made to it).

With this AMI, additional EC2s can be created and each one of the EC2 will have the original software automatically installed as in the case of Apache2. This makes getting the EC2 ready much easier/quicker.

Different ways of launching EC2 Instance


AWS provides multiple ways of launching EC2 instances via the SDK, CLI, CloudFormation and the EC2 Management Console (Web UI). From the EC2 Management Console again there are two ways, one is is using the Launch Instance wizard and other is via the EC2 Launch Templates as shown below.


Both these approaches lead to launching an EC2 and takes the required parameters for the same like the AMI which was mentioned above, the EBS volume size/type, SecurityGroup, KeyPair to be used and a few other details.


In the EC2 creation via the Wizard approach we need to select each and every time the AMI, Instance Type, Network Settings, Storage, Security Groups, pricing model etc from the different options. In the below screen the AMI has to be selected from the available ones. This is OK when we launch an EC2 a few times, but it's more of a routine task and is also time consuming.


This is where the EC2 Launch Templates come to the rescue. We can create a template and predefine all the EC2 properties and reuse the same to launch an EC2 instance. This way we don't need to select the EC2 properties again and again. Below is the template with the AMI, instance type, KeyPair, SecurityGroup predefined. Using this template we should be able to create an EC2 instance using Option (6).


CloudFormation Templates vs EC2 Launch Templates


While the EC2 Launch Templates can be used for the automation of the EC2 Instance creation. CloudFormation Templates are much more than that. It's possible to create many of the AWS resources via the CloudFormation Templates and connect them together, watch the drift (changes to the AWS resources) and much more. Here is the list of AWS Resources that can be created by CloudFormation.

Tuesday, March 31, 2020

Debugging K8S applications with Ephemeral Containers

It's always CRITICAL to pack a Container image with the minimal binaries required as this makes the surface area of attack minimal, upgrading the image and testing also becomes easier as there are less variables to be addressed. Distroless Docker images can be used for the same. In the above diagram Container (A) has only the application and the dependent binaries and nothing more. So, if there are no debugging tools in the Container (A) nor any way to check the status of the process then how do we debug any problem in the application? Once a pod is created, it's even not possible to add Containers to it for additional debugging tools.

That's where the Ephemeral Containers come into picture as in the Container (B) in the above picture. These Containers are temporary that can be included in the Pod dynamically with additional debugging tools. Once a Ephemeral Container has been created, we can connect to it as usual using the kubectl attach, kubectl exec and kubectl logs commands.

Here are the steps for creating an Ephemeral Container. The assumption is that kubeadm has been used to create the K8S Cluster and the Cluster is using K8S 1.18 version or newer which is the latest as of this writing.

Step 1) The Ephemeral Containers feature has to be enabled before creating it. More about enabling K8S features here.

Edit the below files to add '- --feature-gates=EphemeralContainers=true' in the command section. there is no need to restart the Pods as the kubelet process continuously monitors these files and restarts the Pods automatically. Run the 'kubectl get pods --all-namespaces' and notice the changes to the AGE column.

/etc/kubernetes/manifests/kube-apiserver.yaml
/etc/kubernetes/manifests/kube-scheduler.yaml

Step 2) Edit the '/etc/systemd/system/kubelet.service.d/10-kubeadm.conf' to pass the  '--feature-gates=EphemeralContainers=true' parameter to kubelet as shown below.

ExecStart=/usr/bin/kubelet --feature-gates=EphemeralContainers=true $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

Step 3) Restart the kubelet service for the above changes to take place into effect.

sudo systemctl daemon-reload
sudo systemctl restart kubelet

Note that Step 2 and 3 have to be run on all the Control Plane Nodes and the Worker Nodes.

Step 4) Now that the feature has been enabled, lets create a Pod with an application. For sake of simplicity, a pause Pod is created using the below command.

kubectl run ephemeral-demo --image=k8s.gcr.io/pause --restart=Never

There is no shell or any other debugging tools in this container, so the below command will fail.

kubectl exec -it ephemeral-demo -- sh

Step 5) Let's start an Ephemeral Container with the below command. Note that this particular feature is in alpha release of K8S, so the parameter alpha in the kubectl command. Once the feature graduates to stable, we don't need this any more. The below command opens a terminal to the Ephemeral Container and from where we can start debugging the main application.

kubectl alpha debug -it ephemeral-demo --image=busybox --target=ephemeral-demo

Step 6) Open a new terminal and run the below command to get the details of the Ephemeral Container.

kubectl describe pod ephemeral-demo

Hope you had fun learning about Ephemeral Containers, more about here and here.

Monday, January 13, 2020

Prajval in '32nd South Zone Aquatic Championship – 2019'

My son SS Prajval got Silver Medal in the finals of ‘4x50 mts Medley Relay – Group III – Boys’ 32nd South Zone Aquatic Championship– 2019. This was the first time Prajval represented in the finals of the South Zone Competitions and he was able to grab a Silver Medal in the same.

The District and the State level competitions were held at ‘B V Gurumurthy Memorial MCH Swimming Pool, Secunderabad, while the finals were held at ‘GMC Balayogi Athletic Stadium, Gachibowli, Hyderabad’. These events were conducted by the Telangana Swimming Association. The event was well covered by all the major media outlets.


GMC Balayogi Athletic Stadium, Gachibowli, Hyderabad

In the finals Swimmers from Andhra Pradesh, Telangana, Kerala, Karnataka, Tamil Nadu and Puducherry (Southern States of India) represented. It was nice to see kids really motivated with the Swimming. The final events were held on 3rd/4th/5th of January 2020. We had been to the events on the all three days and it was an enriching and motivating experience for Prajval and us.

Receiving the Silver Medal

With his Swimming Coach Mr Satish Balasubramanian (NIS Coach)

With his buddies - BEFORE the competition

With his buddies - AFTER the competetion

Sakshi

Deccan Chronicle

Sakshi
It's a tough balancing act between studies, swimming and peer pressures. But it's good to see that the kids are able to balance and prioritize them. Wish him luck for all the future competitions.