tag:blogger.com,1999:blog-87165310897194200132024-03-16T06:39:47.902+05:30Big Data and Cloud TipsRant about Big Data, Cloud and related technologies.Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.comBlogger289125tag:blogger.com,1999:blog-8716531089719420013.post-36603831895943980822021-06-09T11:04:00.002+05:302021-06-09T11:04:35.710+05:30Using MFA with AWS CLI<p><span style="font-family: verdana;">Lets say that the AWS account credentials get compromised, the hacker should be able to access the account and do a lot of damage. This is where the <a href="https://aws.amazon.com/iam/features/mfa/">AWS MFA</a> comes into play. But, the MFA doesn’t apply the CLI/SDK operations by default and some additional work has to be done.</span></p>
<ol>
<li>
<p><span style="font-family: verdana;">Create an IAM Policy with one of the the below JSON. Both have the same effect of giving the access to all the S3 operations using the short term access keys authenticated via the MFA.</span></p>
<pre><code><span style="font-family: verdana;">{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Action": "s3:*",
"Resource": "*",
"Condition": {
"BoolIfExists": {
"aws:MultiFactorAuthPresent": "false"
}
}
}
]
}
</span></code></pre>
<pre><code><span style="font-family: verdana;">{
"Version": "2012-10-17",
"Id": "123",
"Statement": [
{
"Effect": "Deny",
"Resource": "*",
"Action": "s3:*",
"Condition": {
"Null": {
"aws:MultiFactorAuthAge": true
}
}
}
]
}
</span></code></pre>
</li>
<li>
<p><span style="font-family: verdana;">Create an IAM User and get the access keys for this user.</span></p>
</li>
<li>
<p><span style="font-family: verdana;">Attach the above Policy and the AmazonS3FullAccess Policy to the IAM user.</span></p>
</li>
<li>
<p><span style="font-family: verdana;">Enable MFA for the user.</span></p>
</li>
<li>
<p><span style="font-family: verdana;">Install the AWS CLI.</span></p>
</li>
<li>
<p><span style="font-family: verdana;">Set the above access keys using the <code>aws configure</code> command. Specify the appropriate Region code.</span></p>
</li>
<li>
<p><span style="font-family: verdana;">Get the short term credentials for the same IAM User using the below command. Make sure to replace the arn-of-the-mfa-device and code-from-token in the command.</span></p>
<blockquote>
<p><span style="font-family: verdana;">aws sts get-session-token --serial-number arn-of-the-mfa-device --token-code code-from-token</span></p>
</blockquote>
</li>
<li>
<p><span style="font-family: verdana;">In the <code>.aws/credendtials</code> file create a named profile with the below content. Make sure to replace the access key, secret access key and the session token from the previous command.</span></p>
<blockquote>
<p><span style="font-family: verdana;">[mfa]<br />
aws_access_key_id = example-access-key-as-in-returned-output<br />
aws_secret_access_key = example-secret-access-key-as-in-returned-output<br />
aws_session_token = example-session-Token-as-in-returned-output</span></p>
</blockquote>
</li>
<li>
<p><span style="font-family: verdana;">The below command uses the default named profile and so the long term credentials and so should fail.</span></p>
<blockquote>
<p><span style="font-family: verdana;">aws s3 ls</span></p>
</blockquote>
</li>
<li>
<p><span style="font-family: verdana;">The below command uses the long term credentials authenticated via MFA and should return the list of buckets in S3.</span></p>
<blockquote>
<p><span style="font-family: verdana;">aws s3 ls --profile mfa</span></p>
</blockquote>
</li>
</ol>
<h1 id="further-reading"><span style="font-family: verdana;">Further Reading</span></h1>
<ol>
<li>
<p><span style="font-family: verdana;">AWS CLI and MFA</span></p>
<ul>
<li><a href="https://stackoverflow.com/a/67886052/614157"><span style="font-family: verdana;">https://stackoverflow.com/a/67886052/614157</span></a></li>
</ul>
</li>
<li>
<p><span style="font-family: verdana;">How do I use an MFA token to authenticate access to my AWS resources through the AWS CLI?</span></p>
<ul>
<li><a href="https://aws.amazon.com/premiumsupport/knowledge-center/authenticate-mfa-cli/"><span style="font-family: verdana;">https://aws.amazon.com/premiumsupport/knowledge-center/authenticate-mfa-cli/</span></a></li>
</ul>
</li>
<li>
<p><span style="font-family: verdana;">How can I enforce MFA authentication for IAM users that use the AWS CLI?</span></p>
<ul>
<li><a href="https://aws.amazon.com/premiumsupport/knowledge-center/mfa-iam-user-aws-cli/"><span style="font-family: verdana;">https://aws.amazon.com/premiumsupport/knowledge-center/mfa-iam-user-aws-cli/</span></a></li>
</ul>
</li>
<li>
<p><span style="font-family: verdana;">Adding a bucket policy to require MFA</span></p>
<ul>
<li><a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html#example-bucket-policies-use-case-7"><span style="font-family: verdana;">https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html#example-bucket-policies-use-case-7</span></a></li>
</ul>
</li>
<li>
<p><span style="font-family: verdana;">aws:MultiFactorAuthPresent</span></p>
<ul>
<li><a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-multifactorauthpresent"><span style="font-family: verdana;">https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-multifactorauthpresent</span></a></li>
</ul>
</li>
<li>
<p><span style="font-family: verdana;">Boolean Condition</span></p>
<ul>
<li><a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html#Conditions_Boolean"><span style="font-family: verdana;">https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html#Conditions_Boolean</span></a></li>
</ul>
</li>
<li>
<p><span style="font-family: verdana;">IfExists condition</span></p>
<ul>
<li><a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html#Conditions_IfExists"><span style="font-family: verdana;">https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html#Conditions_IfExists</span></a></li>
</ul>
</li>
</ol>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-77585149833482018452021-04-20T11:39:00.004+05:302021-04-20T11:40:28.554+05:30Installing K8S on AWS EC2 and connecting via Lens<p><span style="font-family: verdana;">There are tons of ways of setting up K8S on AWS. Today we will see one of the easiest way to get started with K8S on AWS. The good thing is that we would be using t2.micro instance type, which falls under the <a href="https://aws.amazon.com/free/" target="_blank">AWS free tier</a>. This configuration is good enough to get started with K8S and not for production setup. It's with the assumption that the reader is familiar with the basic concepts of AWS.</span></p><p><span style="font-family: verdana;"><b>Step 1</b>: Create a SecurityGroup which allows all the traffic inbound and outbound as shown below. This is not a good practice, but is OK for the sake of demo and practicing K8S. Also, make sure to create a KeyPair.</span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-AoHA7FbGIRk/X2hTYla3mNI/AAAAAAABMXU/l5d9g2wGoxQzg7SM8OgnpvEdunXKZOeFQCLcBGAsYHQ/s1920/001-SecurityGroup-Inbound.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-AoHA7FbGIRk/X2hTYla3mNI/AAAAAAABMXU/l5d9g2wGoxQzg7SM8OgnpvEdunXKZOeFQCLcBGAsYHQ/w400-h199/001-SecurityGroup-Inbound.jpg" width="400" /></span></a></div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-OOVutW6eUA0/X2hTYfNyrvI/AAAAAAABMXM/iv1Mnde5VZwCR6yZnRZv_i08vRbt2PF0gCLcBGAsYHQ/s1920/002-SecurityGroup-Outbound.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-OOVutW6eUA0/X2hTYfNyrvI/AAAAAAABMXM/iv1Mnde5VZwCR6yZnRZv_i08vRbt2PF0gCLcBGAsYHQ/w400-h199/002-SecurityGroup-Outbound.jpg" width="400" /></span></a></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;"><b>Step 2</b>: Create 3 Ubuntu instances with t2.micro as the instance type. Make sure to attach the above created SecurityGroup and to attach the KeyPair for connecting to the EC2 instances later. Name the instances as ControlPlane, Worker1 and Worker2 to avoid any confusion.</span></div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-LqHXWGvWJq8/X2hTYSOC9uI/AAAAAAABMXQ/IirwXQ9Uqj8_Tshjozv7apuzevLXH3LdgCLcBGAsYHQ/s1920/100-EC2-Instances.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-LqHXWGvWJq8/X2hTYSOC9uI/AAAAAAABMXQ/IirwXQ9Uqj8_Tshjozv7apuzevLXH3LdgCLcBGAsYHQ/w400-h199/100-EC2-Instances.jpg" width="400" /></span></a></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;"><b>Step 3</b>: Create an Elastic IP and assign it to the Control Plane EC2 instance or else the external IP address of the EC2 might change on reboot and we won't be able to connect from our laptop which is outside the VPC.</span></div><div><span style="font-family: verdana;"><br /></span></div><span style="font-family: verdana;"><b>Step 4</b>: Connect to the EC2 instances using Putty or some other SSH Client. Here I had setup <a href="https://en.wikipedia.org/wiki/Tmux" target="_blank">tmux</a> panes for the 3 EC2 instances. The left pane is for the Control Plane and the right side panes for the Worker EC2 instances. tmux has a cool feature "synchronize-panes" as mentioned in the StackOverflow response here (<a href="https://stackoverflow.com/a/23704472/614157" target="_blank">1</a>). Enter the command in one of the pane and it will be automatically be played in the other panes. If not comfortable with tmux, then simply open multiple Putty sessions to the EC2 instances.</span><div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-9tLOi1jgv8A/X2hTZCoKRkI/AAAAAAABMXY/aZ-k1XaEtegav2t63hQtoi9yaFCfmthZACLcBGAsYHQ/s1920/110-tmux-multiple-panes-sync.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="1040" data-original-width="1920" height="216" src="https://1.bp.blogspot.com/-9tLOi1jgv8A/X2hTZCoKRkI/AAAAAAABMXY/aZ-k1XaEtegav2t63hQtoi9yaFCfmthZACLcBGAsYHQ/w400-h216/110-tmux-multiple-panes-sync.jpg" width="400" /></span></a></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;"><b>Step 5</b>: On the Control Plane and the Worker Instances execute the below commands. This is where the above mentioned tmux feature comes handy.</span></div><div><span style="font-family: verdana;"><br /></span></div><div><div><span style="font-family: verdana;">#Update Ubuntu</span></div><div><span style="font-family: verdana;">sudo su</span></div><div><span style="font-family: verdana;">apt-get update</span></div><div><span style="font-family: verdana;">apt-get dist-upgrade -y</span></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;">#Install Docker</span></div><div><span style="font-family: verdana;">apt install docker.io -y</span></div><div><span style="font-family: verdana;">systemctl enable docker</span></div><div><span style="font-family: verdana;">usermod -a -G docker ubuntu</span></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;">#Add Google K8S repo and install kubeadm</span></div><div><span style="font-family: verdana;">curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add</span></div><div><span style="font-family: verdana;">apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"</span></div><div><span style="font-family: verdana;">apt install kubeadm -y</span></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;">#Pull K8S Docker images (makes the installation faster later)</span></div><div><span style="font-family: verdana;">kubeadm config images pull</span></div></div><div><span style="font-family: verdana;"><br /></span></div><span style="font-family: verdana;"><b>Step 6</b>: On the Control Plane instance execute the below commands.</span></div><div><span style="font-family: verdana;"><br /></span></div><div><div><span style="font-family: verdana;">#Initialize the Control Plane. It will take a few minutes.</span></div><div><span style="font-family: verdana;">#note down the complete "kubeadm join ....." command from the output</span></div><div><span style="font-family: verdana;">kubeadm init --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=NumCPU</span></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;">#Setup the K8S configiguration for Ubuntu user</span></div><div><span style="font-family: verdana;">exit</span></div><div><span style="font-family: verdana;">mkdir -p $HOME/.kube</span></div><div><span style="font-family: verdana;">sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config</span></div><div><span style="font-family: verdana;">sudo chown $(id -u):$(id -g) $HOME/.kube/config</span></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;">#Install the Flanner overlay network</span></div><div><span style="font-family: verdana;">kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml</span></div></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;"><b>Step 7</b>: On both the Worker EC2 instances execute the "kubeadm join ....." command for the Worker EC2 instances to be part of the K8S Cluster.</span></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;"><b>Step 8</b>: Go back to the Control Plane and execute the below commands to make sure the Cluster is setup properly.</span></div><div><span style="font-family: verdana;"><br /></span></div><div><div><span style="font-family: verdana;">#Make sure all the nodes are in a Ready state</span></div><div><span style="font-family: verdana;">kubectl get nodes</span></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;">#Make sure all the pods are in a running state</span></div><div><span style="font-family: verdana;">kubectl get pods --all-namespaces</span></div></div><div><span style="font-family: verdana;"><br /></span></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-QsivHP26c9M/X2hTZTmjxVI/AAAAAAABMXc/3ayL7xRqgfY-op-nrk9MBtaCKbEIQG7kACLcBGAsYHQ/s1911/111-cluster-installation-done.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="1042" data-original-width="1911" height="217" src="https://1.bp.blogspot.com/-QsivHP26c9M/X2hTZTmjxVI/AAAAAAABMXc/3ayL7xRqgfY-op-nrk9MBtaCKbEIQG7kACLcBGAsYHQ/w400-h217/111-cluster-installation-done.jpg" width="400" /></span></a></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;"><b>Step 9</b>: On the Control Plane create dep.yaml file with the below yaml content and create a deployment with the "kubectl apply -f dep.yaml" command. Get the status of the deployment/pods using the "kubectl get deployments" and "kubectl get pods" commands.</span></div><div><span style="font-family: verdana;"><br /></span></div><div><div><span style="font-family: verdana;">apiVersion: apps/v1</span></div><div><span style="font-family: verdana;">kind: Deployment</span></div><div><span style="font-family: verdana;">metadata:</span></div><div><span style="font-family: verdana;"> name: nginx-deployment</span></div><div><span style="font-family: verdana;"> labels:</span></div><div><span style="font-family: verdana;"> app: nginx</span></div><div><span style="font-family: verdana;">spec:</span></div><div><span style="font-family: verdana;"> replicas: 2</span></div><div><span style="font-family: verdana;"> selector:</span></div><div><span style="font-family: verdana;"> matchLabels:</span></div><div><span style="font-family: verdana;"> app: nginx</span></div><div><span style="font-family: verdana;"> template:</span></div><div><span style="font-family: verdana;"> metadata:</span></div><div><span style="font-family: verdana;"> labels:</span></div><div><span style="font-family: verdana;"> app: nginx</span></div><div><span style="font-family: verdana;"> spec:</span></div><div><span style="font-family: verdana;"> containers:</span></div><div><span style="font-family: verdana;"> - name: nginx</span></div><div><span style="font-family: verdana;"> image: nginx</span></div><div><span style="font-family: verdana;"> ports:</span></div><div><span style="font-family: verdana;"> - containerPort: 80</span></div></div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-QQ-uHCdtDWY/X2hTZvTeG8I/AAAAAAABMXg/mpcgTM6hxYotJea5xaET-jNGwNJIHOXNgCLcBGAsYHQ/s1917/112-deployement-yaml.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="1037" data-original-width="1917" height="216" src="https://1.bp.blogspot.com/-QQ-uHCdtDWY/X2hTZvTeG8I/AAAAAAABMXg/mpcgTM6hxYotJea5xaET-jNGwNJIHOXNgCLcBGAsYHQ/w400-h216/112-deployement-yaml.jpg" width="400" /></span></a></div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-0b9NNmvX9Xw/X2hTZ23JW5I/AAAAAAABMXk/RC1S5P_MH_MdloFYbWxU9grnh2YU-ZcUACLcBGAsYHQ/s1913/113-deployment-pods-status.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="1040" data-original-width="1913" height="217" src="https://1.bp.blogspot.com/-0b9NNmvX9Xw/X2hTZ23JW5I/AAAAAAABMXk/RC1S5P_MH_MdloFYbWxU9grnh2YU-ZcUACLcBGAsYHQ/w400-h217/113-deployment-pods-status.jpg" width="400" /></span></a></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;"><b>Step 10</b>: Now lets try connect Lens to the K8S Cluster. Connect to the Control Plane EC2 instance and execute the below commands to generate the certificates again. Make sure to replace the IP addresses with the Public and Private IP address of the Control Plane EC2 instance.</span></div><div><span style="font-family: verdana;"><br /></span></div><div><div><span style="font-family: verdana;">sudo su</span></div><div><span style="font-family: verdana;">rm /etc/kubernetes/pki/apiserver.*</span></div><div><span style="font-family: verdana;">kubeadm init phase certs all --apiserver-advertise-address=0.0.0.0 --apiserver-cert-extra-sans=<b>35.173.255.229,172.31.46.112</b></span></div><div><span style="font-family: verdana;">docker rm -f `docker ps -q -f 'name=k8s_kube-apiserver*'`</span></div><div><span style="font-family: verdana;">systemctl restart kubelet</span></div></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;"><b>Step 11</b>: Copy the content of the ".kube/config" file from the Control Plane to the laptop and save it as a file. This file has all the details to connect to the K8S Cluster. Replace server IP with the external IP of the master EC2</span></div><div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;"><b>Step 12</b>: Download Lens from <a href="https://k8slens.dev/" target="_blank">here</a> and install it. Add a Cluster by pointing to the K8S config file created earlier. Go to Cluster properties in Lens and install Metrics.</span></div></div><div><span style="font-family: verdana;"><br /></span></div><div><span style="font-family: verdana;">In a few seconds, Lens would be gathering the details and metrics from the K8S Cluster on AWS. Note that as of now there is not too much pressure on the EC2 instances.</span></div><div><span style="font-family: verdana;"><br /></span></div><div style="text-align: center;"><span style="font-family: verdana;">(<b>List of nodes</b>)</span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-b5nnVe9T0v0/X2hTaIyE52I/AAAAAAABMXo/cOyOQuocm9Qgk-zmknzOdBcQxonY_DhFACLcBGAsYHQ/s1920/199-nodes.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="1045" data-original-width="1920" height="217" src="https://1.bp.blogspot.com/-b5nnVe9T0v0/X2hTaIyE52I/AAAAAAABMXo/cOyOQuocm9Qgk-zmknzOdBcQxonY_DhFACLcBGAsYHQ/w400-h217/199-nodes.jpg" width="400" /></span></a></div><div><span style="font-family: verdana;"><br /></span></div><div style="text-align: center;"><span style="font-family: verdana;">(<b>Deployment which was created earlier</b>)</span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-eF_LyiCjvys/X2hTaDbQn9I/AAAAAAABMXs/cVQTKqCu7A4493lVQ9MTxnr4A_lmz9YCACLcBGAsYHQ/s1916/200-deployments.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="1041" data-original-width="1916" height="217" src="https://1.bp.blogspot.com/-eF_LyiCjvys/X2hTaDbQn9I/AAAAAAABMXs/cVQTKqCu7A4493lVQ9MTxnr4A_lmz9YCACLcBGAsYHQ/w400-h217/200-deployments.jpg" width="400" /></span></a></div><div><span style="font-family: verdana;"><br /></span></div></div><div><div style="text-align: center;"><span style="font-family: verdana;">(<b>Pods which were created earlier</b>)</span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-cDgzLwE46zE/X2hTaRNH7FI/AAAAAAABMXw/KfQ5rA62DBcoTc-Qxxu-LWyKf78fzg_2QCLcBGAsYHQ/s1920/201-pods.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="1041" data-original-width="1920" height="217" src="https://1.bp.blogspot.com/-cDgzLwE46zE/X2hTaRNH7FI/AAAAAAABMXw/KfQ5rA62DBcoTc-Qxxu-LWyKf78fzg_2QCLcBGAsYHQ/w400-h217/201-pods.jpg" width="400" /></span></a></div><div class="separator" style="clear: both; text-align: center;"><span style="font-family: verdana;"><br /></span></div><div style="text-align: center;"><span style="font-family: verdana;">(<b>kubectl commands via Lens on the Control Plane</b>)</span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-SrdALElL0no/X2hTawTq7DI/AAAAAAABMX0/Q7PGt6D5rlw-GEs235L9Hc2hx86VY38DwCLcBGAsYHQ/s1917/202-kubectl-commands.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="1042" data-original-width="1917" height="217" src="https://1.bp.blogspot.com/-SrdALElL0no/X2hTawTq7DI/AAAAAAABMX0/Q7PGt6D5rlw-GEs235L9Hc2hx86VY38DwCLcBGAsYHQ/w400-h217/202-kubectl-commands.jpg" width="400" /></span></a></div><div class="separator" style="clear: both; text-align: center;"><span style="font-family: verdana;"><br /></span></div></div><div class="separator" style="clear: both; text-align: left;"><b><span style="font-family: verdana;">Conclusion</span></b></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: verdana;"><br /></span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: verdana;">It's not that difficult to setup K8S on AWS using <a href="https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/">kubeadm</a>. We haven't really considered security and performance though, this setup is good enough to get started with K8S on AWS. Since, we are installing K8S manually, we are responsible for HA, Scalability, Upgradation etc. This is where managed services like <a href="https://aws.amazon.com/eks/" target="_blank">AWS EKS</a> come into play.</span></div>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-35253127024834159972021-04-16T18:43:00.001+05:302021-04-16T18:45:26.157+05:30Bicycling - my new hobby<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-xLQtTv2-_pk/YHmN1YGq1qI/AAAAAAABUA4/L6qAN4RrVJY8Ui2509EpQi_zL6xrR7UxwCLcBGAsYHQ/s1920/IMG_20210416_071742%2B%25281%2529.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="887" data-original-width="1920" height="185" src="https://1.bp.blogspot.com/-xLQtTv2-_pk/YHmN1YGq1qI/AAAAAAABUA4/L6qAN4RrVJY8Ui2509EpQi_zL6xrR7UxwCLcBGAsYHQ/w400-h185/IMG_20210416_071742%2B%25281%2529.jpg" width="400" /></a></div><p><span style="font-family: verdana;">It had been quite some time I blogged here. Lately I had been a bit busy with personal and professional work and didn't get much of a chance to post here. This blog is not about technology, but about a new passion which I got bitten into lately.</span></p><p><span style="font-family: verdana;">With the pandemic it had been tough to hit the gym and get some exercise and travelling has come to stand still. And so, I got into the habit of bicycling with my son. I bought a BTWIN Riverside 120 a hybrid bike three months back from Decathlon and really getting the kick out of it. Initially it was about 15 km round trip, but occasionally we had been riding about 40 km also. We had been exploring new routes on a regular basis and I never knew that so many nice places existed around me. We get started early before it gets too hot. We load up with lots of water to keep us hydrated and energy bars to push us. </span></p><p><span style="font-family: verdana;">Fortunately, we have <a href="https://en.wikipedia.org/wiki/Mahavir_Harina_Vanasthali_National_Park" target="_blank">Mahavir Harina Vanasthali National Park</a> a few km drive from our house and we had been hitting it quite often. And also, for whatever reason it has not become a concrete jungle as of now. Fingers crossed, we hope it remains the same for ever.</span></p><p></p><p style="-webkit-text-stroke-width: 0px; color: black; font-size: medium; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"></p><p></p><p style="orphans: 2; text-align: left; text-decoration-color: initial; text-decoration-style: initial; text-decoration-thickness: initial; text-indent: 0px; widows: 2;"><span style="font-family: verdana;">First few days I used to use MapMyRide on my mobile to keep track of the route, time and distance bicycled. But, it had become more of a distraction than something useful. So, stopped using it and started enjoying the places around me and be in the movement. Over time I had become averse of gadgets for some reason and try to keep it as simple as possible. I could have bought a GoPro, Garmin device, but charging, transferring the data etc. nah nah nah.</span></p><p><span style="font-family: verdana;">It' a bit tough to carry a DSLR on the bicycle, but I had been using my mobile phone to take a few pictures here and there. Below are few of the pictures from my trip. Planning to buy a car rack, so that we can explore a bit far away places, but for now we are exploring places which are within 10 to 20 km radius from where we stay.</span></p><p><span style="font-family: verdana;">Hope you like the pictures, I will try to keep the blog updated with pictures of any good locations I come across. Meanwhile, stay healthy and keep safe.</span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-j8j56ln-5rg/YHl1LO5otaI/AAAAAAABUAY/GXKM4-CCjy8dd4rI4Fn3aCAMSl3cl6zAgCLcBGAsYHQ/s1920/IMG_20210416_071735.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="887" data-original-width="1920" height="185" src="https://1.bp.blogspot.com/-j8j56ln-5rg/YHl1LO5otaI/AAAAAAABUAY/GXKM4-CCjy8dd4rI4Fn3aCAMSl3cl6zAgCLcBGAsYHQ/w400-h185/IMG_20210416_071735.jpg" width="400" /></span></a></div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-U-PA8Gt7TiA/YHl1LG_CmSI/AAAAAAABUAU/JbwJRKysNEo1qZ7RKZNptgYtbjmWhSg4gCLcBGAsYHQ/s1920/IMG_20210416_071725.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="887" data-original-width="1920" height="185" src="https://1.bp.blogspot.com/-U-PA8Gt7TiA/YHl1LG_CmSI/AAAAAAABUAU/JbwJRKysNEo1qZ7RKZNptgYtbjmWhSg4gCLcBGAsYHQ/w400-h185/IMG_20210416_071725.jpg" width="400" /></span></a></div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-DwUodhhPpNw/YHl1K77CW-I/AAAAAAABUAQ/Osl3GzD4HTM7xgkwN2KoIZhCJw_kumbsACLcBGAsYHQ/s1920/IMG_20210415_065512.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="887" data-original-width="1920" height="185" src="https://1.bp.blogspot.com/-DwUodhhPpNw/YHl1K77CW-I/AAAAAAABUAQ/Osl3GzD4HTM7xgkwN2KoIZhCJw_kumbsACLcBGAsYHQ/w400-h185/IMG_20210415_065512.jpg" width="400" /></span></a></div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-wVV1S_YCQb0/YHl1KQi0yQI/AAAAAAABUAI/br-J0n3yOl0E2J3-BoVuWkcONVElm5rVACLcBGAsYHQ/s1920/IMG_20210409_065608.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="885" data-original-width="1920" height="185" src="https://1.bp.blogspot.com/-wVV1S_YCQb0/YHl1KQi0yQI/AAAAAAABUAI/br-J0n3yOl0E2J3-BoVuWkcONVElm5rVACLcBGAsYHQ/w400-h185/IMG_20210409_065608.jpg" width="400" /></span></a></div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-AJ1dOrNlxKE/YHl1KbBbREI/AAAAAAABUAE/iiaEURTxNX0ytLHoC-ahfRzgRxZ3kKy2QCLcBGAsYHQ/s1920/IMG_20210409_065514.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="887" data-original-width="1920" height="185" src="https://1.bp.blogspot.com/-AJ1dOrNlxKE/YHl1KbBbREI/AAAAAAABUAE/iiaEURTxNX0ytLHoC-ahfRzgRxZ3kKy2QCLcBGAsYHQ/w400-h185/IMG_20210409_065514.jpg" width="400" /></span></a></div><span style="font-family: verdana;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-6M9RKWHSe0k/YHl1KYq7MNI/AAAAAAABUAM/9-68Xcic1RkVXVheQqFaUHJ3iylTLGxNACLcBGAsYHQ/s1920/IMG_20210318_065305.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: verdana;"><img border="0" data-original-height="887" data-original-width="1920" height="185" src="https://1.bp.blogspot.com/-6M9RKWHSe0k/YHl1KYq7MNI/AAAAAAABUAM/9-68Xcic1RkVXVheQqFaUHJ3iylTLGxNACLcBGAsYHQ/w400-h185/IMG_20210318_065305.jpg" width="400" /></span></a></div><br /><p></p>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-21549787438605039862020-10-14T13:25:00.002+05:302020-10-15T07:20:41.973+05:30Applications around the intersection of Big Data / Machine Learning and AWS<p><span style="font-family: inherit;">As many of the readers of this blog know I am a big fan of Big Data and the AWS Cloud, especially I am interested in the intersection of these two. But, Big Data processing requires huge number of machines, to process huge amounts of data and do some complex processing as in the case of Machine Learning.</span></p><div class="separator" style="clear: both; text-align: center;"><span style="font-family: inherit;"><a href="https://lh3.googleusercontent.com/-rywZ7jqLkjM/X4akA9g9ofI/AAAAAAABNOs/DBU0HQi7AQMSSI4z8Em6bQt-xvFmZxsHgCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="571" data-original-width="542" height="320" src="https://lh3.googleusercontent.com/-rywZ7jqLkjM/X4akA9g9ofI/AAAAAAABNOs/DBU0HQi7AQMSSI4z8Em6bQt-xvFmZxsHgCLcBGAsYHQ/w304-h320/image.png" width="304" /></a></span></div><p></p><p><span style="font-family: inherit;">Cloud has democratized the usage of Big Data, there is no need to buy any machines, we can spin a number of EC2 instances, do the Big Data processing and once done we can terminate the EC2 instances. AWS and other vendors are doing a lot of hardware and software innovations in this space, below are a few hardware innovations from AWS. They do require a lot of investment in the R&D and building them, which is usually possible at the scale Cloud operates.</span></p><p><a href="https://aws.amazon.com/ec2/nitro/" target="_blank">AWS Nitro Systems</a> : Some of the virtualization responsibilities have been shifted from the CPU to the dedicated hardware and software.</p><p><a href="https://aws.amazon.com/ec2/graviton/" target="_blank">AWS Graviton Processor</a> : The Graviton processor uses ARM based architecture, similar to the once used on mobile phones. Now we can spin EC2 with Graviton Processor.</p><p><a href="https://aws.amazon.com/nvidia/" target="_blank">AWS and Nvidia</a> : They bring very high end GPU to the Cloud with the EC2 instances for Machine Learning modelling.</p><p><a href="https://aws.amazon.com/machine-learning/inferentia/" target="_blank">AWS Inferentia</a> : Once the Machine Learning model has been created, the next step is inference which takes most of the CPU cycles. Inferentia is a custom chip from AWS for the same.</p><p><a href="https://aws.amazon.com/ec2/instance-types/f1/" target="_blank">F1 Instances</a> : Hardware acceleration on the EC2 using FPGA.</p><p>Coming back to the subject of this blog, AWS provides a few open data sets via S3 for free for us to do the processing in the Cloud and get some meaningful insights out of it. The data sets can be found <a href="https://registry.opendata.aws/" target="_blank">here</a>. For those who are familiar with either AWS or Big Data, the challenge is how to figure out how the intersection of these work together. For this AWS has published a bunch of blogs/articles <a href="https://registry.opendata.aws/usage-examples/" target="_blank">here</a> on the intersection of AWS and Big Data /Machine Learning for different domains. Below is a sample application around the intersection of Big Data and AWS around Genome data. Note that AWS has been highlighted, look out for more of them.</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-sUrrCgfm_Bg/X4aqTYSD9lI/AAAAAAABNO4/YCVMhSIvovMeimDfMVYvUiRHskfKYXkYgCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="235" data-original-width="660" height="143" src="https://lh3.googleusercontent.com/-sUrrCgfm_Bg/X4aqTYSD9lI/AAAAAAABNO4/YCVMhSIvovMeimDfMVYvUiRHskfKYXkYgCLcBGAsYHQ/w400-h143/image.png" width="400" /></a></div><br /><b>Summary</b><p></p><div>The intersection of Big Data / Machine Learning and AWS is very interesting. Cloud with the pricing democratizes the usage of Big Data / Machine Learning, but each one is a beast on its own to learn and there is a lot of innovation happening in this space and it's tough to keep in pace. <a href="https://registry.opendata.aws/usage-examples/" target="_blank">Here</a> are a few applications around these to get started. Good Luck !!!</div>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-79347365723039601822020-10-08T11:37:00.000+05:302020-10-08T11:37:23.218+05:30Setting up additional EC2 users with username/password and Keypair authentication<p><span style="font-family: inherit;">When an Ubuntu EC2 instances is created in the AWS Cloud, we should be able to connect to the EC2 using the username/password and the Keypairs. In the case of the Ubuntu AMI provided by AWS, only the Keypair authentication is enabled while the username/password authentication is disabled. Very often I get the query "How to create additional users for the Ubuntu EC2 with Keypair for authentication", so is the blog. At the end of the day, Linux is a Linux weather we run it in the Cloud, Laptop or in On-Premise, so the instructions apply everywhere.</span></p><p><span style="font-family: inherit;"></span></p><div class="separator" style="clear: both; text-align: center;"><span style="font-family: inherit;"><a href="https://1.bp.blogspot.com/-jVr6btrMt24/XovlfspjPiI/AAAAAAABF-w/fVyNw2rB3bkCBoUs7BwFTSGxKQPZMtZFACLcBGAsYHQ/s786/001-different-ec2-authentications.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="301" data-original-width="786" height="153" src="https://1.bp.blogspot.com/-jVr6btrMt24/XovlfspjPiI/AAAAAAABF-w/fVyNw2rB3bkCBoUs7BwFTSGxKQPZMtZFACLcBGAsYHQ/w400-h153/001-different-ec2-authentications.png" width="400" /></a></span></div><h2 style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /><br /><span>Setting up an EC2 user with username/password authentication</span></span></h2><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><b>Step 1:</b> Create an Ubuntu EC2 instance and connect to it</span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><b>Step 2:</b> Add user "praveen" using the below command</span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;">#Enter the password and other details</span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;">sudo adduser praveen<br /><br /></span></div><div class="separator" style="clear: both;"><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 3:</b> Open the "/etc/ssh/sshd_config" file and set "PasswordAuthentication" to yes</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 4:</b> Restart the ssh service</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;">sudo service ssh restart</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 5:</b> Connect to the EC2 as the user "praveen" via Putty or some other software by specifying the password</span></div><br /><br /><h2 style="clear: both; text-align: left;"><span style="font-family: inherit;">Setting up an EC2 user with Keypair authentication</span></h2><div class="separator" style="clear: both;"><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 1:</b> Add user "sripati" and disable the the password authentication</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;">#as we would be using the Keypair for authentication</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;">sudo adduser sripati --disabled-password</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 2:</b> Switch as the user</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;">sudo su - sripati</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 3:</b> Generate the keys. They would be in the .ssh folder</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;">ssh-keygen</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 4:</b> Copy the public key to the authorized_keys file in the .ssh folder</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;">cat .ssh/id_rsa.pub >> .ssh/authorized_keys</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 5:</b> Copy the private key in the </span>~/.ssh/id_rsa <span style="font-family: inherit;">to a file sripati.pem on your local machine</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;">cat ~/.ssh/id_rsa</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 6:</b> Using <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html" target="_blank">PuttyGen</a> convert the pem file to ppk. "Load" the pem file and "Save private key" in the ppk format.</span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both;"><span style="font-family: inherit;"><b>Step 7:</b> Now connect via Putty via the username as "sripati", the public IP of the EC2 instance and private key in the ppk format. There is no need to specify the password.</span></div></div></div>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-88221944383996043432020-10-06T09:04:00.001+05:302020-10-06T09:04:29.681+05:30Provisioning AWS infrastructure using Ansible<div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;">Cloud infrastructure provision can be automated using code. The main advantage is that the process can be repeated with consistent output and the code can be version controlled in github, bitbucket or something else.</span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;">AWS comes with CloudFormation for automation of the provisioning of the AWS infrastructure, the main disadvantage is that CloudFormation template (code) is very specific to AWS and takes a lot of effort to migrate to some other Cloud. In this blog we will look at <a href="https://www.ansible.com/" target="_blank">Ansible</a> using which infrastructure can be provisioned for multiple Clouds and also migrating code to provision code to some Cloud doesn't take as much effort as with CloudFormation.</span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;">We would installing Ansible on an Ubuntu EC2 instance for provisioning of the AWS infrastructure. Ansible can be setup on Windows also, but as we install more and more softwares on Windows (host OS) directly, it becomes slow over time. So, I prefer to launch an EC2, try a few things and tear it down once done with it. Anyway, lets look at setting up Ansible and create AWS infrastructure on it.</span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"> <b>Step 1</b>: Create an Ubuntu instances (t2.micro) and connect to it.</span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-PzyyPBe6wMI/X2tasXR7a6I/AAAAAAABMiA/nRMZYdDUOV8dfPchiSYu6-2UBF6LV5-LwCLcBGAsYHQ/s1920/001-ec2-control-node.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-PzyyPBe6wMI/X2tasXR7a6I/AAAAAAABMiA/nRMZYdDUOV8dfPchiSYu6-2UBF6LV5-LwCLcBGAsYHQ/w400-h199/001-ec2-control-node.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 2</b>: Install Python and boto (AWS SDK for Python) on the EC2 instance using the below commands.</span></div><div><span style="font-family: inherit;"><br /></span></div><div><div><span style="font-family: inherit;"> sudo apt-get update</span></div><div><span style="font-family: inherit;"> sudo apt-get install python2.7 python-pip -y</span></div><div><span style="font-family: inherit;"> pip install boto</span></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 3</b>: Install Ansible using the below command.</span></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"> sudo apt install software-properties-common -y</span></div><div><span style="font-family: inherit;"> sudo apt-add-repository --yes --update ppa:ansible/ansible</span></div><div><span style="font-family: inherit;"> sudo apt install ansible -y</span></div></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 4</b>: Go to the IAM Management Console here (<a href="https://console.aws.amazon.com/iam/home?region=us-east-1#/security_credentials" target="_blank">1</a>) and create the Access Keys. Note them down.</span></div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-txziYcvCKIk/X2tasH5XVMI/AAAAAAABMh8/QLXlEIMZzBovTYY4qNuz7xDd7YbDooZaACLcBGAsYHQ/s1920/002-iam-access-keys.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-txziYcvCKIk/X2tasH5XVMI/AAAAAAABMh8/QLXlEIMZzBovTYY4qNuz7xDd7YbDooZaACLcBGAsYHQ/w400-h199/002-iam-access-keys.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 5</b>: Export the Access Keys using the below commands. Make sure to replace 'ABC' and 'DEF' with the Access Keys which have been generated in the previous step.</span></div><div><span style="font-family: inherit;"><br /></span></div><div><div><span style="font-family: inherit;">export AWS_ACCESS_KEY_ID='ABC'</span></div><div><span style="font-family: inherit;">export AWS_SECRET_ACCESS_KEY='DEF'</span></div></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 6</b>: Create a file called "launch-ec2.yaml" with the below content. Make sure to replace the highlighted sections.</span></div><div><span style="font-family: inherit;"><br /></span></div><div><div><span style="font-family: inherit;">- name: Provision a set of instances</span></div><div><span style="font-family: inherit;"> hosts: localhost</span></div><div><span style="font-family: inherit;"> tasks:</span></div><div><span style="font-family: inherit;"> - name: Provision a set of instances</span></div><div><span style="font-family: inherit;"> ec2:</span></div><div><span style="font-family: inherit;"> key_name: <b>my-keypair</b></span></div><div><span style="font-family: inherit;"> region: us-east-1</span></div><div><span style="font-family: inherit;"> group_id:</span></div><div><span style="font-family: inherit;"> - <b>sg-0fa7df1dab4d7ebcb</b></span></div><div><span style="font-family: inherit;"> - <b>sg-040f6c6ef9932dbb5</b></span></div><div><span style="font-family: inherit;"> instance_type: t2.micro</span></div><div><span style="font-family: inherit;"> image: <b>ami-0bcc094591f354be2</b></span></div><div><span style="font-family: inherit;"> wait: yes</span></div><div><span style="font-family: inherit;"> instance_tags:</span></div><div><span style="font-family: inherit;"> Name: Demo</span></div><div><span style="font-family: inherit;"> exact_count: 1</span></div><div><span style="font-family: inherit;"> count_tag: Name</span></div><div><span style="font-family: inherit;"> assign_public_ip: yes</span></div><div><span style="font-family: inherit;"> vpc_subnet_id: <b>subnet-59120577</b></span></div></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 7</b>: Execute the below command to launch an EC2 instance.</span></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;">ansible-playbook launch-ec2.yaml<br /></span></div><div class="separator" style="clear: both; text-align: center;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-5zDQyz47QME/X2tas-zrU1I/AAAAAAABMiE/JrrJif1xsWQwxQp8WI2iwMRolvL9AgACQCLcBGAsYHQ/s1457/004-ansible-launch-ec2-command.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="877" data-original-width="1457" height="241" src="https://1.bp.blogspot.com/-5zDQyz47QME/X2tas-zrU1I/AAAAAAABMiE/JrrJif1xsWQwxQp8WI2iwMRolvL9AgACQCLcBGAsYHQ/w400-h241/004-ansible-launch-ec2-command.jpg" width="400" /></span></a></div><span style="font-family: inherit;"><br /><b>Step 8</b>: Go to the EC2 Management Console and notice a new EC2 instance has been launched with the Name:Demo tag. Make sure to note down the "Instance ID" of the newly created EC2 instance.<br /><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-SJCGO7hnxjI/X2tateHldXI/AAAAAAABMiI/zZLuJNJsUVIFwQtGxQW_hF8kcTFObew9gCLcBGAsYHQ/s1920/005-console-ec2-instance-launched.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-SJCGO7hnxjI/X2tateHldXI/AAAAAAABMiI/zZLuJNJsUVIFwQtGxQW_hF8kcTFObew9gCLcBGAsYHQ/w400-h199/005-console-ec2-instance-launched.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 9</b>: Create a file called "terminate-ec2.yaml" with the below content. Make sure to replace the highlighted section with the Instance ID of the EC2 got from the previous step.</span></div><div><span style="font-family: inherit;"><br /></span></div><div><div><span style="font-family: inherit;">- name: Terminate instances</span></div><div><span style="font-family: inherit;"> hosts: localhost</span></div><div><span style="font-family: inherit;"> tasks:</span></div><div><span style="font-family: inherit;"> - name: Terminate instances</span></div><div><span style="font-family: inherit;"> ec2:</span></div><div><span style="font-family: inherit;"> state: "absent"</span></div><div><span style="font-family: inherit;"> instance_ids: "<b>i-08ef0942aabbc45d7</b>"</span></div><div><span style="font-family: inherit;"> region: us-east-1</span></div><div><span style="font-family: inherit;"> wait: true</span></div></div><div><span style="font-family: inherit;"><br /></span></div><div><div><span style="font-family: inherit;"><b>Step 10</b>: Execute the below command to launch an EC2 instance.</span></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;">ansible-playbook terminate-ec2.yaml</span></div></div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-C6Pk5SPZK5U/X2tat9m270I/AAAAAAABMiM/ptPqnmg_Zwkrz_Rll0JF5r_TgNZzjVMbgCLcBGAsYHQ/s1450/006-ansible-terminate-ec2-command.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="865" data-original-width="1450" height="239" src="https://1.bp.blogspot.com/-C6Pk5SPZK5U/X2tat9m270I/AAAAAAABMiM/ptPqnmg_Zwkrz_Rll0JF5r_TgNZzjVMbgCLcBGAsYHQ/w400-h239/006-ansible-terminate-ec2-command.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 11</b>: Go back to the EC2 Management Console and notice that the EC2 which was created by Ansible will be in a terminated status within a few minutes.</span></div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-XGnBi9fcOEk/X2tat9aWucI/AAAAAAABMiQ/JEvc20RK9cow2Gj2I51OjCXLRP24lH1ZgCLcBGAsYHQ/s1920/007-console-ec2-instance-terminated.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-XGnBi9fcOEk/X2tat9aWucI/AAAAAAABMiQ/JEvc20RK9cow2Gj2I51OjCXLRP24lH1ZgCLcBGAsYHQ/w400-h199/007-console-ec2-instance-terminated.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><b><span style="font-family: inherit;">Conclusion</span></b><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;">By using YAML code, we were able to launch and terminate instance. Ansible allows to do lot of complicated things than this, this is something to start with. As mentioned earlier Ansible allows easy migration to some other Cloud vendor when compared to AWS CloudFormation. BTW, Ansible has been bought by Red Hat which has been bought by IBM. So, Ansible is part of IBM now.</span></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;">For reference, here is the yaml code for launching and terminating the EC2 instances, the screen has been split horizontally using tmux.</span></div><div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-_T4f2V45Lg4/X2tauOGxvQI/AAAAAAABMiU/0yHWz9jhad4lDAWLuRb8m_JI57T88k3EgCLcBGAsYHQ/s1448/008-yaml-on-control-node.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="867" data-original-width="1448" height="240" src="https://1.bp.blogspot.com/-_T4f2V45Lg4/X2tauOGxvQI/AAAAAAABMiU/0yHWz9jhad4lDAWLuRb8m_JI57T88k3EgCLcBGAsYHQ/w400-h240/008-yaml-on-control-node.jpg" width="400" /></span></a></div></div>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-82734476014937855412020-10-01T12:05:00.001+05:302020-10-01T12:07:03.250+05:30Automating EC2 or Linux tasks using "tmux"<div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;">A lot of times we do create multiple EC2 instances and install the same software on each one of them manually, this can be for trying out a Load Balancer feature or to test routing with High Availability across different Regions and Availability Zones. One way to avoid this manual process is to create an AMI, but they are immutable and a new <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html" target="_blank">AMI</a> has to be created for even small changes. This is where <a href="https://github.com/tmux/tmux/wiki/Getting-Started" target="_blank">tmux</a> (Terminal Multiplexer) comes into play.</span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-iRwS5QnmrLg/X2oOcGbFChI/AAAAAAABMeE/6wG0KYT2NOExzIdJKXxCnVJ2GSYPr3uAACLcBGAsYHQ/s1040/001-elb-ec2.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="580" data-original-width="1040" height="223" src="https://1.bp.blogspot.com/-iRwS5QnmrLg/X2oOcGbFChI/AAAAAAABMeE/6wG0KYT2NOExzIdJKXxCnVJ2GSYPr3uAACLcBGAsYHQ/w400-h223/001-elb-ec2.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><span style="font-family: inherit;">Here the assumptions is that we want three EC2 instances as shown above and they are fronted by an ELB, which will load balance the traffic across these EC2 instances. On each of these instances we would like to install Apache2 and create webpages. For this, we would be using one of the EC2 as the <a href="https://en.wikipedia.org/wiki/Bastion_host" target="_blank">jump or bastion box</a> and connect to the other two EC2 instances from here as shown below.</span><div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-egYVs8VkBRg/X2oOcfXaJvI/AAAAAAABMeI/GPvXsH4Arw4IdS7fPRZcE_UTR7seitDFQCLcBGAsYHQ/s835/002-as-bastion-jum-box.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="377" data-original-width="835" height="180" src="https://1.bp.blogspot.com/-egYVs8VkBRg/X2oOcfXaJvI/AAAAAAABMeI/GPvXsH4Arw4IdS7fPRZcE_UTR7seitDFQCLcBGAsYHQ/w400-h180/002-as-bastion-jum-box.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><span style="font-family: inherit;"><b>Step 1</b>: Start three EC2 Ubuntu instances and name them as "WS1/Jump/BastionBox", "WS2" and "WS3".</span><div><span style="font-family: inherit;"> </span><div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-enDIBTAKYok/X2oOcc9TEgI/AAAAAAABMeM/JiciLOafxTYrOAoJ1OIuUHPh5Po9AO4CgCLcBGAsYHQ/s1920/003-ec2-instances.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-enDIBTAKYok/X2oOcc9TEgI/AAAAAAABMeM/JiciLOafxTYrOAoJ1OIuUHPh5Po9AO4CgCLcBGAsYHQ/w400-h199/003-ec2-instances.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 2</b>: Download pagent.exe from here (<a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html" target="_blank">1</a>) and click on "Add Key" and point to the Private Key in the ppk format. Close the window.</span></div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-8kQJU7rmio8/X2oOdNmzqlI/AAAAAAABMeQ/irNw7tHzyxEferXG-EEcIMbkchAypX7YwCLcBGAsYHQ/s690/004-pagent.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="472" data-original-width="690" height="274" src="https://1.bp.blogspot.com/-8kQJU7rmio8/X2oOdNmzqlI/AAAAAAABMeQ/irNw7tHzyxEferXG-EEcIMbkchAypX7YwCLcBGAsYHQ/w400-h274/004-pagent.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 3</b>: Connect to the EC2 instance names as "WS1/Jump/BastionBox" via Putty. In the "Host Name (or IP address)" specify the username and the IP as show below.<br /></span><div class="separator" style="clear: both; text-align: center;"></div></div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-p2FZA9UaZng/X2oOdHxsUAI/AAAAAAABMeU/My75w1Ca85wXEn4J8XXSdnQNzV5AzOlUwCLcBGAsYHQ/s628/005-Putty-Session.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="566" data-original-width="628" height="360" src="https://1.bp.blogspot.com/-p2FZA9UaZng/X2oOdHxsUAI/AAAAAAABMeU/My75w1Ca85wXEn4J8XXSdnQNzV5AzOlUwCLcBGAsYHQ/w400-h360/005-Putty-Session.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;">Go to "Connection --> SSH --> Auth" and make sure to select "Allow agent forwarding". This makes it easy to connect to the EC2 instances, as there is no need to specify the Private Key, it would be picked from pagent.exe. Click on "Open" to connect to the EC2 instance.</span></div><div><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-B_xhNTXAtbo/X22I-Y0qMfI/AAAAAAABMlk/iLe58ODDRFspHeWnbPgkIorUAh0JDIJHgCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img alt="" data-original-height="571" data-original-width="628" height="364" src="https://lh3.googleusercontent.com/-B_xhNTXAtbo/X22I-Y0qMfI/AAAAAAABMlk/iLe58ODDRFspHeWnbPgkIorUAh0JDIJHgCLcBGAsYHQ/w400-h364/image.png" width="400" /></span></a></div></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 4</b>: Execute the tmux command to start it.</span></div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-MxVZwkg1RJQ/X2oOd4-zWDI/AAAAAAABMec/y7oBQLlO5iwoIHzBrIDHG6fMx1OrI3mTwCLcBGAsYHQ/s1733/007-putty-to-bastion.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="950" data-original-width="1733" height="219" src="https://1.bp.blogspot.com/-MxVZwkg1RJQ/X2oOd4-zWDI/AAAAAAABMec/y7oBQLlO5iwoIHzBrIDHG6fMx1OrI3mTwCLcBGAsYHQ/w400-h219/007-putty-to-bastion.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 5</b>: Enter "Ctrl + B" and "%" to split the panes horizontally. Again enter "Ctrl + B" and "Double Quotes" to split the panes vertically. Now we should see three panes as shown below. Use the "Ctrl + B" and the arrow buttons to navigate the panes.</span></div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-xzMj_JqyHaY/X2oOePKZAQI/AAAAAAABMeg/8jWYSETWm2sNwYDhTzhNUctwAVxJ2Vu9wCLcBGAsYHQ/s1730/008-tmux-panes.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="945" data-original-width="1730" height="219" src="https://1.bp.blogspot.com/-xzMj_JqyHaY/X2oOePKZAQI/AAAAAAABMeg/8jWYSETWm2sNwYDhTzhNUctwAVxJ2Vu9wCLcBGAsYHQ/w400-h219/008-tmux-panes.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 6</b>: On the right side upper and bottom panes execute the "ssh ubuntu@ip" command to login to the EC2 instances. Make sure to replace the IP address of WS2 and WS3 EC2 instances in the command. </span></div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-2x6GBtT90-Q/X2oOedUl4oI/AAAAAAABMek/Gifo8atNttUQwMrdbeEvk5ndoUMxfyNUgCLcBGAsYHQ/s1732/009-ssh.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="942" data-original-width="1732" height="217" src="https://1.bp.blogspot.com/-2x6GBtT90-Q/X2oOedUl4oI/AAAAAAABMek/Gifo8atNttUQwMrdbeEvk5ndoUMxfyNUgCLcBGAsYHQ/w400-h217/009-ssh.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 7</b>: Now we are connected to three EC2 instances as shown below. Execute the "ifconfig" command on all the panes and note that the IP address should be different. This is to make sure we are connected to different EC2 instances.</span></div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-gU67VPNNMSQ/X2oOe02AE7I/AAAAAAABMeo/VGIiCmA3Ekgvbsc4xRZlp7QGIEw4aalkgCLcBGAsYHQ/s1732/010-tmux-panes-multiple-ec2.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="941" data-original-width="1732" height="217" src="https://1.bp.blogspot.com/-gU67VPNNMSQ/X2oOe02AE7I/AAAAAAABMeo/VGIiCmA3Ekgvbsc4xRZlp7QGIEw4aalkgCLcBGAsYHQ/w400-h217/010-tmux-panes-multiple-ec2.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;"><b>Step 8</b>: Now we will turn on the synchronization across the panes, this way any command executed on the panes will be automatically executed on the other panes also automatically. For synchronization to happen enter "Ctrl-B " and ":" and "setw synchronize-panes on" and "Enter Button". Use the setw command with "off" options to turn off the synchronization across the panes.<br /><br /></span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-qSUFU_I6uBI/X2oOfJngeFI/AAAAAAABMes/KgZ2M7se5kY0YVVGI10sAxQpMP6mp4ZBgCLcBGAsYHQ/s1728/011-turning-sync-on.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="941" data-original-width="1728" height="217" src="https://1.bp.blogspot.com/-qSUFU_I6uBI/X2oOfJngeFI/AAAAAAABMes/KgZ2M7se5kY0YVVGI10sAxQpMP6mp4ZBgCLcBGAsYHQ/w400-h217/011-turning-sync-on.jpg" width="400" /></span></a></div><div><span style="font-family: inherit;"><br /></span></div><span style="font-family: inherit;"><b>Step 9</b>: Navigate to one of the pane and notice that any command executed in one of the pane would get executed in the other panes. <b>Ain't it neat !!!</b></span></div><div><span style="font-family: inherit;"><br /></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-lguXkN4qn5g/X2oOfT-GOAI/AAAAAAABMew/dOf4kUexL5gugRlLsKtq0m4lSzNmUFhHwCLcBGAsYHQ/s1727/012-running-commands.jpg" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: inherit;"><img border="0" data-original-height="945" data-original-width="1727" height="219" src="https://1.bp.blogspot.com/-lguXkN4qn5g/X2oOfT-GOAI/AAAAAAABMew/dOf4kUexL5gugRlLsKtq0m4lSzNmUFhHwCLcBGAsYHQ/w400-h219/012-running-commands.jpg" width="400" /></span></a></div></div></div><div class="separator" style="clear: both; text-align: center;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: left;"><b><span style="font-family: inherit;">Conclusion</span></b></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;">When we want to automate tasks AWS provides a few means like <a href="https://aws.amazon.com/systems-manager/" target="_blank">SSM</a>, <a href="https://aws.amazon.com/opsworks/" target="_blank">OpsWorks</a>, AMI and so on. But, there are good for automating on the long run, but not good when we want to try different things in an iterative approach or we are really not sure what we want to do.</span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;"><br /></span></div><div class="separator" style="clear: both; text-align: left;"><span style="font-family: inherit;">This is where tmux with the synchronization feature comes handy. There is lot more to tmux, but hope this blog articles helps you to get started with tmux and builds the curiosity around it.</span></div></div>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-63148552278088767182020-09-29T11:38:00.006+05:302020-10-06T09:00:21.662+05:30Using the same Keypair across AWS Regions<div dir="ltr" style="text-align: left;" trbidi="on"><div><span style="font-family: inherit;"><div class="separator" style="clear: both; text-align: left;"><span>In one of the previous blog (<a href="https://www.thecloudavenue.com/2020/04/linux-ec2-authentication-key-pair.html" target="_blank">1</a>), we looked what happens behind the scenes when we use a Keypair for authentications against Linux. </span><span style="font-family: inherit;">This blog post is more about productivity. I do create and connect to EC2 instances quite often and so I have created </span><a href="https://documentation.help/PuTTY/config-saving.html" style="font-family: inherit;" target="_blank">Sessions</a><span style="font-family: inherit;"> in Putty for most of my regularly connected Linux instances. One of the Session is for AWS which automatically populates the username and the keypair as shown below. When I would like to connect to an EC2 instance all I need to specify the Public IP address of the EC2 instance.</span></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-dtp2oEWPHiE/X2mPpbPw8GI/AAAAAAABMcc/fR2J-js7Gg4caCMg6gTRa4TRO2-IdpkNQCLcBGAsYHQ/s632/001-Putty_SavedSessions.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="576" data-original-width="632" height="365" src="https://1.bp.blogspot.com/-dtp2oEWPHiE/X2mPpbPw8GI/AAAAAAABMcc/fR2J-js7Gg4caCMg6gTRa4TRO2-IdpkNQCLcBGAsYHQ/w400-h365/001-Putty_SavedSessions.jpg" width="400" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-c0eIvWLuQbw/X2mPpZWvXfI/AAAAAAABMcg/AzL-73K5WLcjFxY2MKUBWG1-UpYG2uWAQCLcBGAsYHQ/s627/002-Authorization.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="571" data-original-width="627" height="364" src="https://1.bp.blogspot.com/-c0eIvWLuQbw/X2mPpZWvXfI/AAAAAAABMcg/AzL-73K5WLcjFxY2MKUBWG1-UpYG2uWAQCLcBGAsYHQ/w400-h364/002-Authorization.jpg" width="400" /></a></div><div><span><br /></span></div><div><span>It all looks fine and dandy, the only problem is when I create EC2 instances in different AWS regions to test High Availability or some other features and try to connect to them. With the above approach since the Keypairs have regional scope, when I connect to EC2 instances in different regions, I need to change the keypairs in Putty. It would be good to use the same Keypair across regions, this way I don't need to change when connecting to the EC2 in different regions when using Putty saved sessions feature. Let's look at how to.</span></div><div><span><br /></span></div><div><span><b>Step 1</b>: Download putty.exe and puttygen.exe from here (<a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html" target="_blank">1</a>). There is no need to install it, just downloading should be good enough.</span></div><div><span><br /></span></div><div><span><b>Step 2</b>: Go to the EC2 Management Console and create a Keypair. Generate the Keypair by selecting the pem or ppk format. </span></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-tHYbrwruCss/X2mPpZrZUpI/AAAAAAABMcY/i6qWm6leYic-KSiXxXe6zgRZXHpYwRgRgCLcBGAsYHQ/s1920/003-keypair-creation.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-tHYbrwruCss/X2mPpZrZUpI/AAAAAAABMcY/i6qWm6leYic-KSiXxXe6zgRZXHpYwRgRgCLcBGAsYHQ/w400-h199/003-keypair-creation.jpg" width="400" /></a></div><div><span><br /></span></div><div><span><b>Step 3</b>: When prompted store the private key.</span></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-jWb0Bo3eYbU/X2mPqG_mTaI/AAAAAAABMck/pCIvhcATTo82b-z3E_DmoRbA5AWxu1xogCLcBGAsYHQ/s812/004-keypair-save.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="621" data-original-width="812" height="306" src="https://1.bp.blogspot.com/-jWb0Bo3eYbU/X2mPqG_mTaI/AAAAAAABMck/pCIvhcATTo82b-z3E_DmoRbA5AWxu1xogCLcBGAsYHQ/w400-h306/004-keypair-save.jpg" width="400" /></a></div><div><span><br /></span></div><div class="separator" style="clear: both; text-align: left;">The Keypair should be created as shown below.</div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-q8-0csKiJtM/X2mPqqTJXvI/AAAAAAABMco/QfV2CXEg8hk2Swl6oM2G5uwNxyWd_FVIACLcBGAsYHQ/s1920/005-keypair-created.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-q8-0csKiJtM/X2mPqqTJXvI/AAAAAAABMco/QfV2CXEg8hk2Swl6oM2G5uwNxyWd_FVIACLcBGAsYHQ/w400-h199/005-keypair-created.jpg" width="400" /></a></div><div><span><br /></span></div><div><span><b>Step 4</b>: Start PuttyGen and click on Load.</span></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Ce8ms6BO93o/X2mPrNAgFnI/AAAAAAABMcs/p5urtvnp50QBiJu4zalH7JF-0hRVq6nIQCLcBGAsYHQ/s660/050-PuttyGen.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="605" data-original-width="660" height="366" src="https://1.bp.blogspot.com/-Ce8ms6BO93o/X2mPrNAgFnI/AAAAAAABMcs/p5urtvnp50QBiJu4zalH7JF-0hRVq6nIQCLcBGAsYHQ/w400-h366/050-PuttyGen.jpg" width="400" /></a></div><div><span><br /></span></div><div><span><b>Step 5</b>: Point to the private key which has been downloaded earlier. If the file is not visible then remove the filter and select "All files (*.*)". Click on Open and click on OK.</span></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-YuOgrIRK8gw/X2mPr84bFaI/AAAAAAABMcw/IL632LqSaVkzyCe1KpDONChOYViU_dHxACLcBGAsYHQ/s967/051-PuttyGen-Load.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="621" data-original-width="967" height="258" src="https://1.bp.blogspot.com/-YuOgrIRK8gw/X2mPr84bFaI/AAAAAAABMcw/IL632LqSaVkzyCe1KpDONChOYViU_dHxACLcBGAsYHQ/w400-h258/051-PuttyGen-Load.jpg" width="400" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-9WN1q1MYIHI/X2mPsNQFkWI/AAAAAAABMc0/H3mVYjXuUhkrSC4dUo0kHXk0jL6cgVt6wCLcBGAsYHQ/s663/052-PuttyGen-Load.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="607" data-original-width="663" height="366" src="https://1.bp.blogspot.com/-9WN1q1MYIHI/X2mPsNQFkWI/AAAAAAABMc0/H3mVYjXuUhkrSC4dUo0kHXk0jL6cgVt6wCLcBGAsYHQ/w400-h366/052-PuttyGen-Load.jpg" width="400" /></a></div><div><span><br /></span></div><div><span><b>Step 6</b>: Click on "Save public key" and specify the same file name but with a pub extension as shown below.</span></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/--8yhUhXHHIU/X2mPsXE8fWI/AAAAAAABMc4/Ra9l8jLnv2wSEes4jMrQ9ndCPAta6LF_gCLcBGAsYHQ/s972/053-PuttyGen-SavePublicKey.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="618" data-original-width="972" height="254" src="https://1.bp.blogspot.com/--8yhUhXHHIU/X2mPsXE8fWI/AAAAAAABMc4/Ra9l8jLnv2wSEes4jMrQ9ndCPAta6LF_gCLcBGAsYHQ/w400-h254/053-PuttyGen-SavePublicKey.jpg" width="400" /></a></div><div><span><br /></span></div><div><span><b>Step 7</b>: Go to the EC2 Management Console for some other region and navigate to the Keypair tab. Click on Actions and then "Import key pair".</span></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-3ZwLngZZWcM/X2mPsir3d7I/AAAAAAABMc8/rSMiumJZQH8uZFTImOWAVNA1KiXhFLOugCLcBGAsYHQ/s1920/100-some-other-region.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-3ZwLngZZWcM/X2mPsir3d7I/AAAAAAABMc8/rSMiumJZQH8uZFTImOWAVNA1KiXhFLOugCLcBGAsYHQ/w400-h199/100-some-other-region.jpg" width="400" /></a></div><div><span><br /></span></div><div><span><b>Step 8</b>: Click on "Choose file" and point to the pub file which was created earlier. Finally click on Import to create the Keypair.</span></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-WBMI2EbcxEw/X2mPswEAwbI/AAAAAAABMdA/RkD635OTcFkO1XR8TbWGiU3tGekn7yg6gCLcBGAsYHQ/s871/101-import-keypair.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="515" data-original-width="871" height="236" src="https://1.bp.blogspot.com/-WBMI2EbcxEw/X2mPswEAwbI/AAAAAAABMdA/RkD635OTcFkO1XR8TbWGiU3tGekn7yg6gCLcBGAsYHQ/w400-h236/101-import-keypair.jpg" width="400" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-UGr7-dZ8vpg/X2mPtFTOY1I/AAAAAAABMdE/8MjFzeftkWckDJUUQKwwJCEaR2_H9E9jgCLcBGAsYHQ/s1920/102-keypair-created.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="954" data-original-width="1920" height="199" src="https://1.bp.blogspot.com/-UGr7-dZ8vpg/X2mPtFTOY1I/AAAAAAABMdE/8MjFzeftkWckDJUUQKwwJCEaR2_H9E9jgCLcBGAsYHQ/w400-h199/102-keypair-created.jpg" width="400" /></a></div><br /><span><b>Conclusion</b></span></span></div><div><span><span style="font-family: inherit;"><br /></span></span></div><div><span><span><span style="font-family: inherit;">Now we have created a Keypair in two regions. And both the regions have keypairs which have the same public/private key. So, we would be able to use the same Putty session when connecting to the EC2 instances in different regions. It's not a life saving hack, but it something interesting to know and saves a few seconds/clicks here and there.</span></span></span></div><div><span><span><span style="font-family: inherit;"><br /></span></span></span></div><div><span><span><span style="font-family: inherit;">Note that this approach is not recommended for production and sensitive setup as we are using the same Keypair across regions, but can definitely used when we are trying to learn AWS.</span></span></span></div>
</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-70826332493429011522020-09-22T10:32:00.000+05:302020-09-22T10:32:48.898+05:30Connecting Lens IDE to K8S Cluster using port forwarding<p>In the previous blogs (<a href="https://www.thecloudavenue.com/2018/09/k8s-cluster-on-laptop.html" target="_blank">1</a>, <a href="https://www.thecloudavenue.com/2020/09/virtualbox-network-settings-k8s.html" target="_blank">2</a>), I mentioned about setting up K8S Cluster on laptop for the sake of experimenting. We should be able to connect to the Control Plane/Master instance and execute the kubectl commands to interact with the K8S cluster. For those who are new to K8S or not from technology background it might be a bit intimidating using the different options with kubectl, this is where Lens (K8S IDE) (<a href="https://k8slens.dev/" target="_blank">1</a>, <a href="https://github.com/lensapp/lens" target="_blank">2</a>) comes into play.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Zq-e4u0jfZw/X2TBRWqDU_I/AAAAAAABMJs/IzAAkwy5mIYoukT9ahxXK7oqskPklY_hgCLcBGAsYHQ/s1127/008-starting-vm.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="660" data-original-width="1127" height="235" src="https://1.bp.blogspot.com/-Zq-e4u0jfZw/X2TBRWqDU_I/AAAAAAABMJs/IzAAkwy5mIYoukT9ahxXK7oqskPklY_hgCLcBGAsYHQ/w400-h235/008-starting-vm.jpg" width="400" /></a></div><p>Lens is dubbed as K8S IDE and is a FOSS and can be integrated with multiple K8S Clusters at a time. Depending on the permissions, both Read and Write Operations are allowed on the K8S Cluster. As shown below, I had configured Virtual Machines for the K8S Cluster on the Laptop using VirtualBox.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-LPTVWN8D5ZI/X2boiNjxAPI/AAAAAAABML4/AjR8sxA7masW8qUFoPv0UnEhnjtgxAO-gCLcBGAsYHQ/s652/013-virtualbox-virtual-machines.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="460" data-original-width="652" height="283" src="https://1.bp.blogspot.com/-LPTVWN8D5ZI/X2boiNjxAPI/AAAAAAABML4/AjR8sxA7masW8qUFoPv0UnEhnjtgxAO-gCLcBGAsYHQ/w400-h283/013-virtualbox-virtual-machines.jpg" width="400" /></a></div><div><br /></div>'NAT Networking' was used for the VirtualBox networking as this allows to work in the offline mode, network communication across Virtual Machines and also access to the internet. The only caveat is that there is no direct connectivity form the Host Machines to the Guest Virtual Machines, port forwarding has to be used as mentioned in the documentation here (<a href="https://docs.oracle.com/en/virtualization/virtualbox/6.1/user/network_nat.html#natforward" target="_blank">1</a>).<div><br /></div><div>Below is how the port forwarding has been configured in the VirtualBox global settings. The Host IP had been left out and will default to localhost. The port 27 from the localhost is pointing to the port 6443 on which K8S API Server is listening to. This is required for the Lens to connect to the K8S Cluster, rest of the rules are for connecting to the Virtual Machin Instances via SSH.<div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-aMGryoZrRwY/X2bq1tmPEuI/AAAAAAABMMM/MXH18_fLs24QTv14cdvGuFcGHuL-eDOzACLcBGAsYHQ/s700/012-virtualbox-port-forwarding.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="517" data-original-width="700" height="295" src="https://1.bp.blogspot.com/-aMGryoZrRwY/X2bq1tmPEuI/AAAAAAABMMM/MXH18_fLs24QTv14cdvGuFcGHuL-eDOzACLcBGAsYHQ/w400-h295/012-virtualbox-port-forwarding.jpg" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div></div><div class="separator" style="clear: both; text-align: left;">In the Lens, the ".kube/config" file from the K8S Control Plane/Master must be imported during setting up the Cluster. The ".kube/config" file didn't work as-is because port forwarding has been used and the X509 certificates are not valid for the localhost/127.0.0.1 IP address. Had to do two things.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><b>(a)</b> Generate the certificates on the Control Plane/Master as root using the below commands. Note that 10.0.2.101 is the IP address of the K8S Control Plane/Master on which the API Server is running. Thanks to the StackOverflow solution here (<a href="https://stackoverflow.com/questions/46360361/invalid-x509-certificate-for-kubernetes-master/47284147#47284147" target="_blank">1</a>).</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><div class="separator" style="clear: both;">rm /etc/kubernetes/pki/apiserver.*</div><div class="separator" style="clear: both;">kubeadm init phase certs all --apiserver-advertise-address=0.0.0.0 --apiserver-cert-extra-sans=<b>10.0.2.101</b>,127.0.0.1</div><div class="separator" style="clear: both;">docker rm -f `docker ps -q -f 'name=k8s_kube-apiserver*'`</div><div class="separator" style="clear: both;">systemctl restart kubelet</div><div><br /></div><div><b>(b)</b> And then modify the config file to point to 127.0.0.1:27, before importing and creating a K8S Cluster in Lens. Note that 27 is the port number configured in the VirtualBox port forwarding rules for the API Server.</div></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-gwMllbwUebY/X2brwvk9pII/AAAAAAABMMU/aQMH6V0sY6IZUfnAgOUBqIGPdiNuGn5KwCLcBGAsYHQ/s437/011-kube-config.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="98" data-original-width="437" height="90" src="https://1.bp.blogspot.com/-gwMllbwUebY/X2brwvk9pII/AAAAAAABMMU/aQMH6V0sY6IZUfnAgOUBqIGPdiNuGn5KwCLcBGAsYHQ/w400-h90/011-kube-config.jpg" width="400" /></a></div><div><br /></div><div>Completing the above two steps allowed a connection from the Lens to the K8S API Server which is the single point of interface to the K8S Cluster. It took some time to figure it out, but it was interesting and fun. Below are some of the screens from the Lens around various dimensions.</div><div><br /></div><div style="text-align: center;">(<b>Details of the Control Plane/Master</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-UiIrA2kuBLE/X2bs4p9BwEI/AAAAAAABMMg/vRdyDaIpieMgMyGsN-6ev4hxWnueKrEBQCLcBGAsYHQ/s1918/001-cluster-master.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1042" data-original-width="1918" height="217" src="https://1.bp.blogspot.com/-UiIrA2kuBLE/X2bs4p9BwEI/AAAAAAABMMg/vRdyDaIpieMgMyGsN-6ev4hxWnueKrEBQCLcBGAsYHQ/w400-h217/001-cluster-master.jpg" width="400" /></a></div><div><br /></div><span style="text-align: center;"><div style="text-align: center;">(<b>Details of the Slave</b>)</div></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-GfwqC50KOb4/X2bs4umFoBI/AAAAAAABMMc/5tExlIyo1TAxl3-sl9lB6kuy6pVFJCN4ACLcBGAsYHQ/s1920/002-cluster-worker.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1013" data-original-width="1920" height="211" src="https://1.bp.blogspot.com/-GfwqC50KOb4/X2bs4umFoBI/AAAAAAABMMc/5tExlIyo1TAxl3-sl9lB6kuy6pVFJCN4ACLcBGAsYHQ/w400-h211/002-cluster-worker.jpg" width="400" /></a></div><span> <br /><div style="text-align: center;">(<b>Details of the nodes in the Cluster</b>)</div></span><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-JzOZrPGo8nA/X2bs4tgttBI/AAAAAAABMMk/crsjtHhtsSwwAOTdbhmPN-DIX7RrHJjpACLcBGAsYHQ/s1920/003-nodes.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1043" data-original-width="1920" height="217" src="https://1.bp.blogspot.com/-JzOZrPGo8nA/X2bs4tgttBI/AAAAAAABMMk/crsjtHhtsSwwAOTdbhmPN-DIX7RrHJjpACLcBGAsYHQ/w400-h217/003-nodes.jpg" width="400" /></a></div><div><br /></div><div style="text-align: center;">(<b>Details of the Control Plane/Master</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-KoVkb_iwujk/X2bs5exYNhI/AAAAAAABMMo/nwvkege8qQ0plOwTAGE_RBGFt4KzN9ukQCLcBGAsYHQ/s1920/004-nodes.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1045" data-original-width="1920" height="217" src="https://1.bp.blogspot.com/-KoVkb_iwujk/X2bs5exYNhI/AAAAAAABMMo/nwvkege8qQ0plOwTAGE_RBGFt4KzN9ukQCLcBGAsYHQ/w400-h217/004-nodes.jpg" width="400" /></a></div><div><br /></div><div style="text-align: center;">(<b>Overview of the workloads on the Cluster</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-l9xY_w1JdOo/X2bs5ZmUiEI/AAAAAAABMMs/2n4zQCLNWuI6kbaTveKTKRom2VishZv5QCLcBGAsYHQ/s1920/005-workload-overview.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1045" data-original-width="1920" height="217" src="https://1.bp.blogspot.com/-l9xY_w1JdOo/X2bs5ZmUiEI/AAAAAAABMMs/2n4zQCLNWuI6kbaTveKTKRom2VishZv5QCLcBGAsYHQ/w400-h217/005-workload-overview.jpg" width="400" /></a></div><div><br /></div><div style="text-align: center;">(<b>Pods on the Cluster</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-djR2Wm8HHX0/X2bs55btP9I/AAAAAAABMMw/mc-fu6mxYhU9Su1Yzme8izK7HdZIdXSAgCLcBGAsYHQ/s1920/006-workload-pods.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1041" data-original-width="1920" height="217" src="https://1.bp.blogspot.com/-djR2Wm8HHX0/X2bs55btP9I/AAAAAAABMMw/mc-fu6mxYhU9Su1Yzme8izK7HdZIdXSAgCLcBGAsYHQ/w400-h217/006-workload-pods.jpg" width="400" /></a></div><div><br /></div><div style="text-align: center;">(<b>DeamonSets on the Cluster</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-LLbn3WB290I/X2bs5x2PXRI/AAAAAAABMM0/zgbQwfv48qsabBWmDZPGnYM5AF4IYqJRACLcBGAsYHQ/s1920/007-workload-daemonsets.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1042" data-original-width="1920" height="217" src="https://1.bp.blogspot.com/-LLbn3WB290I/X2bs5x2PXRI/AAAAAAABMM0/zgbQwfv48qsabBWmDZPGnYM5AF4IYqJRACLcBGAsYHQ/w400-h217/007-workload-daemonsets.jpg" width="400" /></a></div><div><br /></div><div style="text-align: center;">(<b>Services on the Cluster</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-zlYCQBREeRY/X2bs6QRIxUI/AAAAAAABMM4/9dLdPbsQF0onVUyyDs345eYdXnEAfQtEwCLcBGAsYHQ/s1920/008-network-services.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1045" data-original-width="1920" height="217" src="https://1.bp.blogspot.com/-zlYCQBREeRY/X2bs6QRIxUI/AAAAAAABMM4/9dLdPbsQF0onVUyyDs345eYdXnEAfQtEwCLcBGAsYHQ/w400-h217/008-network-services.jpg" width="400" /></a></div><div><br /></div><div style="text-align: center;">(<b>Endpoints on the Cluster</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Pt5wU8pssOk/X2bs6hGchtI/AAAAAAABMM8/Dza87n0ObV8lhfxo3eIlNLYLW0ZSgfq_QCLcBGAsYHQ/s1917/009-network-endpoints.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1043" data-original-width="1917" height="217" src="https://1.bp.blogspot.com/-Pt5wU8pssOk/X2bs6hGchtI/AAAAAAABMM8/Dza87n0ObV8lhfxo3eIlNLYLW0ZSgfq_QCLcBGAsYHQ/w400-h217/009-network-endpoints.jpg" width="400" /></a></div><div><br /></div><div style="text-align: center;">(<b>Namespaces in the Cluster</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Pd20HvHngBA/X2bs6ztbPJI/AAAAAAABMNA/hvz-hRlw7TQxneLAUx4LWei2yj1X3EVUACLcBGAsYHQ/s1920/010-namespaces.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1037" data-original-width="1920" height="216" src="https://1.bp.blogspot.com/-Pd20HvHngBA/X2bs6ztbPJI/AAAAAAABMNA/hvz-hRlw7TQxneLAUx4LWei2yj1X3EVUACLcBGAsYHQ/w400-h216/010-namespaces.jpg" width="400" /></a></div><br /><div class="separator" style="clear: both; text-align: left;">Likewise, it's possible to connect to multiple K8S Cluster from Lens and operate on them. Lens is context aware and automatically downloads the correct version of the kubectl from the Google K8S repository.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><b>Conclusion</b></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">Lens is a nice K8S IDE and it's a nice way to get started with K8S and also very useful for those who are not that technology savvy to browse around different components in K8S. But, those who are familiar with K8S or have spent good amount of time it's a hinderance and would prefer executing the kubectl command. It's very much like using "vi" vs "notepad" for editing files. With the recent acquisition of Lens by Mirantis (<a href="https://www.mirantis.com/blog/lens-the-worlds-most-popular-kubernetes-ide-at-mirantis/" target="_blank">1</a>), we need to wait how Lens adds to the productivity.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">Also, don't get too used to Lens, <a href="https://www.cncf.io/certification/cka/" target="_blank">CKA</a> and <a href="https://www.cncf.io/certification/ckad/" target="_blank">CKAD</a> certifications don't allow the usage of Lens. Everything has be performed from the command line and one needs to be very familiar with vi/tmux and bunch of command line tools.</div>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-6101758849622703182020-09-19T09:58:00.000+05:302020-09-19T09:58:34.873+05:30Optimal VirtualBox network setting for K8S on Laptop<p>In one of the <a href="https://www.thecloudavenue.com/2018/09/k8s-cluster-on-laptop.html" target="_blank">previous blog</a> we looked at setting up K8S on a laptop. The advantages of this setup is the freedom to try out of different things and it is very quick to get started. On my laptop it takes about 5 minutes for the Virtual Machines to start including the K8S in them. The downside is it's mainly for learning things and doesn't take much load.</p><p>Recently I bought a new <a href="https://www.amazon.in/gp/product/B086R8418N/" target="_blank">Lenovo ThinkPad</a> and so had to go with the entire exercise of setting up K8S on it. BTW, pretty happy with the Laptop. The only gripe is that it comes with 8GB of RAM, need to upgrade it to 16GB, the maximum RAM it supports. The Laptop is very light and I can snug into any corner of the house to work with concentration easily.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-oz3NAgVwvnk/X2TBPxEeBoI/AAAAAAABMJc/_n29RNV342sh3UCMYGU6oK0Nt2kCEZx2QCLcBGAsYHQ/s953/001-laptop.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="591" data-original-width="953" height="248" src="https://1.bp.blogspot.com/-oz3NAgVwvnk/X2TBPxEeBoI/AAAAAAABMJc/_n29RNV342sh3UCMYGU6oK0Nt2kCEZx2QCLcBGAsYHQ/w400-h248/001-laptop.jpg" width="400" /></a></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">Above is the setup on my previous Laptop, with one Control Plane (master) and two slaves. There had been a few problems with the VirtualBox networking. Different types of networking are supported by VirtualBox (<a href="https://www.virtualbox.org/manual/ch06.html#networkingmodes" target="_blank">1</a>) and Bridged Networking was used. With Bridged Networking everything was working fine with the below problems.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">- Had to be always connected to the network. Won't be able to work in the offline mode.</div><div class="separator" style="clear: both; text-align: left;">- Also, switching between the different networks will change the IP of the master and K8S would stop working.</div><div><br /></div>As mentioned above there is more than one way of configuring the network in VirtualBox. The same can be seen in the Virtual Machine settings under Network tab.<div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-NsSj5C_g9h0/X2TBP-5ejoI/AAAAAAABMJY/DXpXFkUSdQgmZ4O9_GLnjgQ7CRkj_MRSgCLcBGAsYHQ/s721/002-vbox-networking.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="605" data-original-width="721" height="336" src="https://1.bp.blogspot.com/-NsSj5C_g9h0/X2TBP-5ejoI/AAAAAAABMJY/DXpXFkUSdQgmZ4O9_GLnjgQ7CRkj_MRSgCLcBGAsYHQ/w400-h336/002-vbox-networking.jpg" width="400" /></a></div><div><br /></div><div>Here(<a href="https://www.nakivo.com/blog/virtualbox-network-setting-guide/" target="_blank">1</a>) is a good article on the different types of networking in VirtualBox and details about them. On the Y-Axis we have the different types of networking and on the X-Axis the features they support. Let's narrow down to the type of networking we would like to use with VirtualBox by identifying the required features for having a K8S Cluster on the Laptop.</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-F03zd1WqO44/X2TBP0W9ZGI/AAAAAAABMJg/29mpIQ4vXII9fNeM2yijCNr7ZO6bZx93QCLcBGAsYHQ/s987/003-vbox-networking.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="352" data-original-width="987" height="143" src="https://1.bp.blogspot.com/-F03zd1WqO44/X2TBP0W9ZGI/AAAAAAABMJg/29mpIQ4vXII9fNeM2yijCNr7ZO6bZx93QCLcBGAsYHQ/w400-h143/003-vbox-networking.jpg" width="400" /></a></div><div><br /></div><div>-- "VM <--> VM" -- <b>Required</b> for communicating across VM instances.</div><div>-- "VM <- Host" -- <b>Required</b> as we need to connect from the Host OS to the Guest for debugging etc.</div><div>-- "VM --> LAN" -- <b>Required</b> for the internet connection to download the different softwares</div><div>-- "VM --> Host" -- Is <b>optional</b> for connecting from the Virtual Machine to Host</div><div>-- "VM <-- LAN" -- Is <b>optional</b> for accessing the K8S Cluster from outside the Laptop</div><div><br /></div><div>From the feature matrix and the required features, the only options left around the VirtualBox networking are NAT Network and Bridged Networking. The problem with the Bridged networking is that as mentioned above, it always requires connection to the network and switching to a different network changes the IP of the K8S master and breaks down the entire setup. The certificates during the K8S setup are tied to a specific IP and need to generated again each time the IP address of the master changes (<a href="https://github.com/kubernetes/kubeadm/issues/338" target="_blank">1</a>). This is not impossible, but is tedious every time we change the network and the IP address of the master changes. So, the only optimal option left is to use the NAT Network.</div><div><br /></div><div>With the combination of the NAT Network in VirtualBox and using static IP address in guest Virtual Machines, we don't need to worry about changing from one network to another as the VirtualBox NAT Network has a DHCP component and an IP address from it can be configured as Static IP for the Guest Virtual Machines. Also, a Virtual Switch would be used for the communication across the different guest Virtual Machines and there is no need to be connected to the network. This ensures that we can work in the offline mode with K8S on the laptop even we are on the move. Below are the different components while using the VirtualBox NAT Network and how the network communication happens. Highlighted in the red is how the network communication happens.</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-r5lj8IV4w9o/X2TBRI327VI/AAAAAAABMJk/go9khD_RqMoSJwZ-ot5FZTajqROa6v7SgCLcBGAsYHQ/s1221/004-vbox-networking.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="830" data-original-width="1221" height="272" src="https://1.bp.blogspot.com/-r5lj8IV4w9o/X2TBRI327VI/AAAAAAABMJk/go9khD_RqMoSJwZ-ot5FZTajqROa6v7SgCLcBGAsYHQ/w400-h272/004-vbox-networking.jpg" width="400" /></a></div><div><br /></div><div>The only catch with the NAT Network is that we won't be able to connect to the guest Virtual Machines directly without doing any port forwarding as mentioned in the VirtualBox documentation here (<a href="https://docs.oracle.com/en/virtualization/virtualbox/6.1/user/network_nat.html#natforward" target="_blank">1</a>). The documentation mentions NAT, but the same applies to the NAT Network also. This is a not a big issue, but is a matter of configuring the VirtualBox with "Port Forwarding Rules" before connecting to the guest Virtual Machines.</div><div><br /></div><div><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-V5Bw9snP41M/X2WGJNp0M5I/AAAAAAABMK0/Y3ppNGmlHWM-qcCo7r02X7F2cZlbatjygCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="513" data-original-width="700" height="235" src="https://lh3.googleusercontent.com/-V5Bw9snP41M/X2WGJNp0M5I/AAAAAAABMK0/Y3ppNGmlHWM-qcCo7r02X7F2cZlbatjygCLcBGAsYHQ/image.png" width="320" /></a></div><br /></div><div>In a future blog, I will provide the binaries and the steps to easily setup K8S on a laptop. But, for now I took a screenshot of the Memory usage before and after starting the Virtual Machines on the laptop.</div><div><br /></div><div style="text-align: center;">(<b>Before</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-wyWVeuxB6SU/X2TBRWeux7I/AAAAAAABMJo/A1fLSkRcf2szdQZ0_Qkr1BwinKy67mL4gCLcBGAsYHQ/s860/007-memory-before.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="767" data-original-width="860" height="356" src="https://1.bp.blogspot.com/-wyWVeuxB6SU/X2TBRWeux7I/AAAAAAABMJo/A1fLSkRcf2szdQZ0_Qkr1BwinKy67mL4gCLcBGAsYHQ/w400-h356/007-memory-before.jpg" width="400" /></a></div><div><br /></div><div style="text-align: center;">(<b>Starting the Virtual Machines with K8S</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Zq-e4u0jfZw/X2TBRWqDU_I/AAAAAAABMJs/IzAAkwy5mIYoukT9ahxXK7oqskPklY_hgCLcBGAsYHQ/s1127/008-starting-vm.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="660" data-original-width="1127" height="234" src="https://1.bp.blogspot.com/-Zq-e4u0jfZw/X2TBRWqDU_I/AAAAAAABMJs/IzAAkwy5mIYoukT9ahxXK7oqskPklY_hgCLcBGAsYHQ/w400-h234/008-starting-vm.jpg" width="400" /></a></div><div><br /></div><div style="text-align: center;">(<b>After</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-ebhBTsAod2k/X2TBRiLz0mI/AAAAAAABMJw/TJWJhS7_PeMHQxb9RCX5wkePhHRSj-dvACLcBGAsYHQ/s863/009-memory-after.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="767" data-original-width="863" height="355" src="https://1.bp.blogspot.com/-ebhBTsAod2k/X2TBRiLz0mI/AAAAAAABMJw/TJWJhS7_PeMHQxb9RCX5wkePhHRSj-dvACLcBGAsYHQ/w400-h355/009-memory-after.jpg" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;">(<b>Laptop CPU and RAM</b>)</div><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-fR2eoyj2Sug/X2TrBtg99oI/AAAAAAABMKU/lNe6uL19ccocW64VpF4S1nrfPKmuWwVswCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="82" data-original-width="566" height="46" src="https://lh3.googleusercontent.com/-fR2eoyj2Sug/X2TrBtg99oI/AAAAAAABMKU/lNe6uL19ccocW64VpF4S1nrfPKmuWwVswCLcBGAsYHQ/image.png" width="320" /></a></div><div><br /></div>Within 4 to 5 minutes, I was able to login to the K8S master and able to get the list of nodes and the pods using the kubectl command.</div><div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-pchjmO6NAkA/X2TBR7Lw-YI/AAAAAAABMJ0/dy88qiii6dMPMpV4gDXr0eNik0SB7EWlACLcBGAsYHQ/s1451/010-kubectl-command.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="867" data-original-width="1451" height="239" src="https://1.bp.blogspot.com/-pchjmO6NAkA/X2TBR7Lw-YI/AAAAAAABMJ0/dy88qiii6dMPMpV4gDXr0eNik0SB7EWlACLcBGAsYHQ/w400-h239/010-kubectl-command.jpg" width="400" /></a></div></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;"><b>Conclusion</b></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">To conclude setting up K8S is not a hard task, but requires a bit of patience for the installation of the OS, softwares, configurations and finally cloning the Virtual Machines, so as to avoid repetition of tasks and saving time. Also "VirtualBox NAT Network" is the best option in the network type as this enables to work in the offline mode and doesn't break the K8S setup while switching between networks.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">As mentioned I would be uploading the Virtual Machines Images and would be detailing the procedure for setting up K8S on a Laptop. But, I need to zip and uploads huge files, so it might take some time.</div>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-89384692294483474972020-08-12T15:14:00.002+05:302020-08-12T18:21:48.763+05:30Connecting to S3 Service via VPC Gateway Endpoint<p></p><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: left;">Lets say we are building a image processing application using ML which gets the images from S3 and identifies the action performed (sitting, standing etc) in those images . By default the network data flows from the application to S3 over the internet as shown in the left image which is not really that efficient and secure. AWS provides VPC Gateway Endpoint feature and this all the data will be within the AWS network only.</div><div class="separator" style="clear: both; text-align: left;"></div><p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-k-rLdQ1M9gA/XzOlABJcdvI/AAAAAAABKzA/qGW6EuXtIkUcus_s1qhv8PtTUnKV83gMACLcBGAsYHQ/s1081/050.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="364" data-original-width="1081" height="173" src="https://1.bp.blogspot.com/-k-rLdQ1M9gA/XzOlABJcdvI/AAAAAAABKzA/qGW6EuXtIkUcus_s1qhv8PtTUnKV83gMACLcBGAsYHQ/w513-h173/050.jpg" width="513" /> </a></div><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: left;"> </div><div class="separator" style="clear: both; text-align: left;">VPC Endpoints provides Gateway Endpoints for S3 and DynamoDB services, while Interface Endpoints are for the rest of the services. In this blog we will explore the VPC Gateway Endpoints.<br /></div><div class="separator" style="clear: both; text-align: left;"> </div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-K5N5fEEiOx0/XzOlAK63XzI/AAAAAAABKzE/Wh1coYUQ0M05sRQDyFEZKiw6A9pPRF3kACLcBGAsYHQ/s949/051.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="546" data-original-width="949" height="294" src="https://1.bp.blogspot.com/-K5N5fEEiOx0/XzOlAK63XzI/AAAAAAABKzE/Wh1coYUQ0M05sRQDyFEZKiw6A9pPRF3kACLcBGAsYHQ/w512-h294/051.jpg" width="512" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><b>Step 1:</b> Create a VPC as mentioned in the <a href="https://www.thecloudavenue.com/2020/08/connecting-ec2-private-subnet.html" target="_blank">previous blog</a> and connect to the EC2 in the Private Subnet. By default the route table for the Private Subnet has route for 0.0.0.0/0 and so any EC2 in the Private Subnet will have internet connection.<br /><br /><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-gwd25tr-lts/XzOftrDY4mI/AAAAAAABKxY/oAeTgj19XK8V5QEmHIP6Ydilis9Y4DvqgCLcBGAsYHQ/s1920/100.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-gwd25tr-lts/XzOftrDY4mI/AAAAAAABKxY/oAeTgj19XK8V5QEmHIP6Ydilis9Y4DvqgCLcBGAsYHQ/w512-h248/100.jpg" width="512" /> </a></div><div class="separator" style="clear: both; text-align: left;"> </div><div class="separator" style="clear: both; text-align: left;">The same can be verified by pinging google.com or some other host.<br /></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-jcLImRgq1Pg/XzOftuwo-wI/AAAAAAABKxU/dUFwhWyR7d85EoGXNsf5db4u4lRzDoFGgCLcBGAsYHQ/s1323/101.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="912" data-original-width="1323" height="353" src="https://1.bp.blogspot.com/-jcLImRgq1Pg/XzOftuwo-wI/AAAAAAABKxU/dUFwhWyR7d85EoGXNsf5db4u4lRzDoFGgCLcBGAsYHQ/w512-h353/101.jpg" width="512" /></a></div><p></p><p><b>Step 2:</b> In the Putty execute the below commands to install the AWS CLI.<br /></p><p>sudo apt-get update<br />sudo apt-get install python2.7 python-pip -y<br />pip install awscli --upgrade<br />export PATH="$PATH:/home/ubuntu/.local/bin/" <br /></p><p><b>Step 3:</b> Create an IAM Role with AmazonS3ReadOnlyAccess and attach it to the EC2 instance in the Private Subnet. <br /></p><p><b>Step 4:</b> Lets remove the internet connection for the EC2 instance in the Private Subnet. For this select the Routing Table for the Private Subnet and click on Edit Routes. Delete the route for 0.0.0.0/0 and click on "Save routes".<br /></p><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-caT_pz6qgY4/XzOfuKFCSSI/AAAAAAABKxc/SNoNBn9EJt0YjD8M3PgptzKsCPmtJ8F8wCLcBGAsYHQ/s1920/102.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-caT_pz6qgY4/XzOfuKFCSSI/AAAAAAABKxc/SNoNBn9EJt0YjD8M3PgptzKsCPmtJ8F8wCLcBGAsYHQ/w512-h248/102.jpg" width="512" /> </a></div><div class="separator" style="clear: both; text-align: center;"> </div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-jDvTN5T6xjs/XzOfuXKFPPI/AAAAAAABKxg/EbZmmW0rrisADLojkWu0OVm-zTXHRcCzACLcBGAsYHQ/s1920/103.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-jDvTN5T6xjs/XzOfuXKFPPI/AAAAAAABKxg/EbZmmW0rrisADLojkWu0OVm-zTXHRcCzACLcBGAsYHQ/w512-h248/103.jpg" width="512" /></a></div><p></p><p>The Route Table will be updated as shown below.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-uTVg68ChhW0/XzOfuaQFUgI/AAAAAAABKxk/EL9WWSNmfZ0Gjzs7Emxa9BufgAiVzHcZQCLcBGAsYHQ/s1920/104.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-uTVg68ChhW0/XzOfuaQFUgI/AAAAAAABKxk/EL9WWSNmfZ0Gjzs7Emxa9BufgAiVzHcZQCLcBGAsYHQ/w512-h248/104.jpg" width="512" /> </a></div><div class="separator" style="clear: both; text-align: left;"> </div><div class="separator" style="clear: both; text-align: left;"><b>Step 5:</b> There is no need for NAT Gateway and the ElasticIP as we have removed the internet connection for the EC2 in the Private Subnet. Make sure to remove them. You can also keep it, but there is a cost associated with it. <br /></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-BkNR0gHUIKQ/XzOfu4nfGRI/AAAAAAABKxo/vIfiF4q_UuQU3bSRbG6Agr5IfRwdBMS4QCLcBGAsYHQ/s1920/110.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-BkNR0gHUIKQ/XzOfu4nfGRI/AAAAAAABKxo/vIfiF4q_UuQU3bSRbG6Agr5IfRwdBMS4QCLcBGAsYHQ/w512-h248/110.jpg" width="512" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-_EPYF1Lb0TU/XzOfu6hBylI/AAAAAAABKxs/wDe3AUb_rp8Le-Ffd3EIhYyg5QfZiDbiACLcBGAsYHQ/s1920/111.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-_EPYF1Lb0TU/XzOfu6hBylI/AAAAAAABKxs/wDe3AUb_rp8Le-Ffd3EIhYyg5QfZiDbiACLcBGAsYHQ/w512-h248/111.jpg" width="512" /></a></div><p></p><p>Test out the internet connectivity (ping google.com") and also try to get the list of files in S3 (aws s3 ls). Both the commands should fail. Press Ctrl+C to come out of the commands.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-qv0LI_lr11Q/XzOfvOLSysI/AAAAAAABKxw/e3eJSWJtCFQ90b9GE2PyCB-jUV_OqovnwCLcBGAsYHQ/s1324/120.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="907" data-original-width="1324" height="350" src="https://1.bp.blogspot.com/-qv0LI_lr11Q/XzOfvOLSysI/AAAAAAABKxw/e3eJSWJtCFQ90b9GE2PyCB-jUV_OqovnwCLcBGAsYHQ/w512-h350/120.jpg" width="512" /></a></div><p></p><p><b>Step 6:</b> In the VPC Management Console, go to "Endpoints" and click on "Create Endpoint".<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-OnIdNpJBmRM/XzOfv6_ZSBI/AAAAAAABKx0/DDpDK5HjF9YkHrYOQbY8Db_5IHqa_sRkACLcBGAsYHQ/s1920/200.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-OnIdNpJBmRM/XzOfv6_ZSBI/AAAAAAABKx0/DDpDK5HjF9YkHrYOQbY8Db_5IHqa_sRkACLcBGAsYHQ/w512-h248/200.jpg" width="512" /></a></div><p></p><p>Search for S3 in the Service Name and select "com.amazonaws.us-east-1.s3". Make sure to select the VPC which was created in the previous step and select the Private Subnet.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-6b2x5fzsynI/XzOfv613PpI/AAAAAAABKx4/P4mBRUl_YXQd7nBkMJKZKMw4fKDk_xS7wCLcBGAsYHQ/s1920/201.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-6b2x5fzsynI/XzOfv613PpI/AAAAAAABKx4/P4mBRUl_YXQd7nBkMJKZKMw4fKDk_xS7wCLcBGAsYHQ/w512-h248/201.jpg" width="512" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-P-MzDMqSK20/XzOfwJinBEI/AAAAAAABKx8/zbGHRB86uWIpYP_VD6RHKI_CiBqhgiSYQCLcBGAsYHQ/s1920/202.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-P-MzDMqSK20/XzOfwJinBEI/AAAAAAABKx8/zbGHRB86uWIpYP_VD6RHKI_CiBqhgiSYQCLcBGAsYHQ/w512-h248/202.jpg" width="512" /></a></div><p></p><p>Rest of the default options are good enough. Click on "Create endpoint" and the Endpoint will be created in a few minutes.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-yTiYTZr2I3s/XzOfwvk9qLI/AAAAAAABKyE/JIlzkG9mVTIVG6g0ptHhfPVTHwQMnuqVgCLcBGAsYHQ/s1920/203.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-yTiYTZr2I3s/XzOfwvk9qLI/AAAAAAABKyE/JIlzkG9mVTIVG6g0ptHhfPVTHwQMnuqVgCLcBGAsYHQ/w512-h248/203.jpg" width="512" /></a></div><p></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Uf0B3iQBlas/XzOfwlfjk2I/AAAAAAABKyA/oZzoVrYyfhgzkA2XlStEeSi0h12HAUhpACLcBGAsYHQ/s1920/204.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-Uf0B3iQBlas/XzOfwlfjk2I/AAAAAAABKyA/oZzoVrYyfhgzkA2XlStEeSi0h12HAUhpACLcBGAsYHQ/w512-h248/204.jpg" width="512" /></a></div><p></p><p><b>Step 7:</b> Go back to the Route Table of the Private Subnet and note that a Route has been automatically added to the VPC Endpoint.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-HKib-6LBQ2o/XzOfwx8N4OI/AAAAAAABKyI/pQIGAx5jEVwtWOmbXoygRBDsPaz29TSCQCLcBGAsYHQ/s1920/205.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-HKib-6LBQ2o/XzOfwx8N4OI/AAAAAAABKyI/pQIGAx5jEVwtWOmbXoygRBDsPaz29TSCQCLcBGAsYHQ/w512-h248/205.jpg" width="512" /></a></div><div class="separator" style="clear: both; text-align: left;"> </div><div class="separator" style="clear: both; text-align: left;"><b>Step 8:</b> Go back to the Putty session and execute the below commands. Notice that there is no internet connection, but still we are able to get the number of buckets in the AWS S3. This is because we have setup the AWS Gateway Endpoint and all the traffic remains with the AWS network only.<br /></div><div class="separator" style="clear: both; text-align: left;"> </div><div class="separator" style="clear: both; text-align: left;">ping google.com</div><div class="separator" style="clear: both; text-align: left;">aws s3 ls | wc -l </div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-i-R2-G0C4-M/XzOfxjrVuAI/AAAAAAABKyM/PMDQsXMG8rUshSHLQ2vjiaEbKFmY5xpxACLcBGAsYHQ/s1315/206.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="913" data-original-width="1315" height="355" src="https://1.bp.blogspot.com/-i-R2-G0C4-M/XzOfxjrVuAI/AAAAAAABKyM/PMDQsXMG8rUshSHLQ2vjiaEbKFmY5xpxACLcBGAsYHQ/w512-h355/206.jpg" width="512" /></a></div><h1 style="text-align: left;">Conclusion</h1><p>By default when we consume any AWS service from an EC2 instance the network traffic goes through the internet, which is not really secure. And there is an additional cost for NAT and Internet Gateway. By using the VPC Endpoint Gateway we noticed that the network traffic remains within the AWS network only. This makes it easy for migrating the applications to AWS and also make sure they are compliant.<br /></p><p></p>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com1tag:blogger.com,1999:blog-8716531089719420013.post-11599863453813764222020-08-11T23:17:00.003+05:302020-08-11T23:17:51.787+05:30Creating a VPC and connecting to the EC2 in the Private SubnetAn <a href="https://aws.amazon.com/vpn/" target="_blank">AWS VPC</a> (Virtual Private Cloud) is a logically isolated network for isolating different environments like Production, QA, Development. VPC can also be used to isolate applications like CRM, HR and others. Applications is one VPC by default won't be able to communicate with applications in another VPC. A <a href="https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html" target="_blank">VPC Peering Connection</a> has to be explicitly setup for communication to happen between two VPC.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-X8FwbOlpBF0/XzJh_Ow8RNI/AAAAAAABKmg/ssPCQxsOqsow8L6EApQix6anCR70XA7MACLcBGAsYHQ/s1050/025.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="559" data-original-width="1050" height="273" src="https://1.bp.blogspot.com/-X8FwbOlpBF0/XzJh_Ow8RNI/AAAAAAABKmg/ssPCQxsOqsow8L6EApQix6anCR70XA7MACLcBGAsYHQ/w512-h273/025.jpg" width="512" /></a><br /></div><p>In this blog we will setup a new VPC. In this VPC, we will create a Public and a Private Subnet. The way we will configure them is that any EC2 in the Public Subnet will have a Public and Private IP address, while any EC2 in the Private Subnet will have only a Private IP address associated with it. We can connect to the EC2 in the Public Subnet as it has a Public IP, but how do we connect to the EC2 in the Private Subnet as it doesn't have a Public IP? This might be required for making any changes to that particular EC2 instances to perform tasks like installing/upgrading databases etc.<br /></p><p>One way to it to setup a VPN connection between the Laptop and the VPC as mentioned in the <a href="https://www.thecloudavenue.com/2020/05/aws-client-vpn.html" target="_blank">previous blog</a>. This way the Laptop and the EC2 in the Private Subnet will appear as though they are in the same network and so we would be able connect from the Laptop to the EC2 in the Private Subnet. Another way is to connect to the EC2 in the Public Subnet using the Public IP and from there connect to the EC2 in the Private Subnet using the Private IP. This is what we would be exploring in this blog.<br /></p><p><b>Step 1:</b> Go to the EC2 Management Console and make sure the "New EC2" experience is selected.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-8kRWEQWEnJ4/XzJh-7ZhGQI/AAAAAAABKmc/rsa2PUTjBdIXb-sGOJG1cUC2Eyhl3H4ewCLcBGAsYHQ/s274/050.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="204" data-original-width="274" height="130" src="https://1.bp.blogspot.com/-8kRWEQWEnJ4/XzJh-7ZhGQI/AAAAAAABKmc/rsa2PUTjBdIXb-sGOJG1cUC2Eyhl3H4ewCLcBGAsYHQ/w175-h130/050.jpg" width="175" /></a></div><br /><b>Step 2:</b> Click on the "Key Pairs" and click on "Create key pair". Enter the Key pair name and make sure the ppk format is selected and click on "Create key pair". <br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Hb7lEGK3C-I/XzJh_E9Ui_I/AAAAAAABKmk/RXeNAxhacMwBvfCAPmZz0MqhG5Bw9E7UACLcBGAsYHQ/s1920/101.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-Hb7lEGK3C-I/XzJh_E9Ui_I/AAAAAAABKmk/RXeNAxhacMwBvfCAPmZz0MqhG5Bw9E7UACLcBGAsYHQ/w512-h248/101.jpg" width="512" /></a></div><p></p><p>The Key pair would be created as shown below.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Eqp0aUR2T-E/XzJh_rQRHEI/AAAAAAABKmo/qaYYGUOvR_gpSWOcs6Mtl9fPxIfVdaAawCLcBGAsYHQ/s1920/103.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-Eqp0aUR2T-E/XzJh_rQRHEI/AAAAAAABKmo/qaYYGUOvR_gpSWOcs6Mtl9fPxIfVdaAawCLcBGAsYHQ/w512-h248/103.jpg" width="512" /></a></div><br /><b>Step 3:</b> Click on "Elastic IPs", click on "Allocate Elastic IP address" and finally click on Allocate. The Elastic IP address is required for NAT instance which will be automatically created while creating the VPC later.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-HNZA-EPont8/XzJh_0D5O3I/AAAAAAABKms/_HABFbEoDLE7Q_AY7HmfTWVVvElzX9eewCLcBGAsYHQ/s1920/151.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-HNZA-EPont8/XzJh_0D5O3I/AAAAAAABKms/_HABFbEoDLE7Q_AY7HmfTWVVvElzX9eewCLcBGAsYHQ/w512-h248/151.jpg" width="512" /></a></div><p></p><p>An Elastic IP address will be created as shown below.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-H-EYvFhz0zA/XzJiAL4jqSI/AAAAAAABKmw/vkZN-QtXs-oZu7x5jKTTwTvZRslI0kWsQCLcBGAsYHQ/s1920/152.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-H-EYvFhz0zA/XzJiAL4jqSI/AAAAAAABKmw/vkZN-QtXs-oZu7x5jKTTwTvZRslI0kWsQCLcBGAsYHQ/w512-h248/152.jpg" width="512" /></a></div><p></p><p><b>Step 4:</b> Go to the VPC Management Console and click on "Launch VPC Wizard".<br /></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-hpEB7UIn-40/XzJiAWWGxmI/AAAAAAABKm0/CflPLaVCyWMxc4g5xRBd0XmmGj69sYT5gCLcBGAsYHQ/s1920/200.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-hpEB7UIn-40/XzJiAWWGxmI/AAAAAAABKm0/CflPLaVCyWMxc4g5xRBd0XmmGj69sYT5gCLcBGAsYHQ/w512-h248/200.jpg" width="512" /></a></div><p><b>Step 5:</b> Select "VPC with Public and Private Subnets" Option and click on Select.<br /></p><p style="text-align: center;"><a href="https://1.bp.blogspot.com/-vA9U_8ZSfXw/XzK3W3J60ZI/AAAAAAABKrg/flmyrDoG4OQhwKDp35KX2UBzoUmAkRUgACLcBGAsYHQ/s1920/201.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-vA9U_8ZSfXw/XzK3W3J60ZI/AAAAAAABKrg/flmyrDoG4OQhwKDp35KX2UBzoUmAkRUgACLcBGAsYHQ/w512-h248/201.jpg" width="512" /></a></p><p><b>Step 6:</b> Enter the name of the VPC as MyVPC and select the Elastic IP created in the previous steps. Finally, click on "Create VPC". Rest of the default options are good enough.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-p8-6-9ccUso/XzJiAmdD8gI/AAAAAAABKm8/y6-AATF3NwE2_T4AwY8E4blsvbRHUNUewCLcBGAsYHQ/s1920/202.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-p8-6-9ccUso/XzJiAmdD8gI/AAAAAAABKm8/y6-AATF3NwE2_T4AwY8E4blsvbRHUNUewCLcBGAsYHQ/w512-h248/202.jpg" width="512" /></a></div><p></p><p>The VPC creation process process takes a few minutes and the VPC screen will be updated as shown below.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-E91ltfAZ_Mk/XzJiA6_L7cI/AAAAAAABKnA/Xl6IFM-OwVY33GUPGUPH13-AR-073iAJACLcBGAsYHQ/s1920/205.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-E91ltfAZ_Mk/XzJiA6_L7cI/AAAAAAABKnA/Xl6IFM-OwVY33GUPGUPH13-AR-073iAJACLcBGAsYHQ/w512-h248/205.jpg" width="512" /></a></div><p></p><p><b>Step 7:</b> Any EC2 created in the Public Subnet, will only have the Private IP, we need to make sure it also has a Public IP address. For this to happen, from the list of Subnets select the Public Subnet, Actions -> "Modify auto-assign IP settings".<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-s9aGFVyMVQE/XzJiBAwh9HI/AAAAAAABKnE/q8D7_TejHQUdSVtfjKyMWe0ltPg41UsOgCLcBGAsYHQ/s1920/251.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-s9aGFVyMVQE/XzJiBAwh9HI/AAAAAAABKnE/q8D7_TejHQUdSVtfjKyMWe0ltPg41UsOgCLcBGAsYHQ/w512-h248/251.jpg" width="512" /></a></div><p></p><p>Make sure to check "Auto-assign IPv4" and click on Save.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-2Mnp5tr23_A/XzJiBRxl34I/AAAAAAABKnI/kYpCGth1yk0Vb31Ea9Nmg0kpSsNQfzC6ACLcBGAsYHQ/s1920/252.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-2Mnp5tr23_A/XzJiBRxl34I/AAAAAAABKnI/kYpCGth1yk0Vb31Ea9Nmg0kpSsNQfzC6ACLcBGAsYHQ/w512-h248/252.jpg" width="512" /></a></div><p></p><p><b>Step 8:</b> Click on the Security Groups and click on "Create security group". Enter the name as AllowSSH, give some description and add an inbound rule as displayed below to allow Port 22/ssh inbound. Make sure to select the VPC which has been created in the previous steps. Click on "Create security group". This one will be associated with the EC2 instances later.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-FAdH82Xc7Ls/XzJiBlUm5aI/AAAAAAABKnM/LoBbfzxVbCokLT9BSqulpGBzdaZ_BasLQCLcBGAsYHQ/s1920/301.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-FAdH82Xc7Ls/XzJiBlUm5aI/AAAAAAABKnM/LoBbfzxVbCokLT9BSqulpGBzdaZ_BasLQCLcBGAsYHQ/w512-h248/301.jpg" width="512" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Z4P9ythGtrs/XzJiBjJj4qI/AAAAAAABKnQ/d5fP0XZYkogWzkR-Ksu_oRmPgeOP1SrYgCLcBGAsYHQ/s1920/302.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-Z4P9ythGtrs/XzJiBjJj4qI/AAAAAAABKnQ/d5fP0XZYkogWzkR-Ksu_oRmPgeOP1SrYgCLcBGAsYHQ/w512-h248/302.jpg" width="512" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-yVWG_D9eHg0/XzJiBywrQEI/AAAAAAABKnU/ep24AANbyIIo0Obj3Jb15Tqfhr8CSkY6gCLcBGAsYHQ/s1920/303.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-yVWG_D9eHg0/XzJiBywrQEI/AAAAAAABKnU/ep24AANbyIIo0Obj3Jb15Tqfhr8CSkY6gCLcBGAsYHQ/w512-h248/303.jpg" width="512" /></a></div><p></p><p><b>Step 9:</b> Create Ubuntu EC2 instances with t2.micro as the instance type in the MyVPC, one each in Private and Public Subnets. Make sure to select MyVPC and the appropriate Subnet in 'Configure instance' options while creating the EC2 instance. Also, select the AllowSSH Security Group and the KeyPair which has been created in the previous steps, this allows us to connect to the EC2 via SSH using the Key pair.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-WUFo1GBWsPA/XzJiCJQoljI/AAAAAAABKnY/FJIMgiqpiDY7BZfjS-Cyz79eqQktuSc3wCLcBGAsYHQ/s1920/305.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-WUFo1GBWsPA/XzJiCJQoljI/AAAAAAABKnY/FJIMgiqpiDY7BZfjS-Cyz79eqQktuSc3wCLcBGAsYHQ/w512-h248/305.jpg" width="512" /></a></div><p><b>Step 10:</b> Name the EC2 instances as MyVPC-PublicSubnet and MyVPC-PrivateSubnet appropriately. Also notice that the EC2 in the Public Subnet has both Private and Public IP address, while the EC2 in the Private Subnet has only Private IP address as shown below.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-rydjpNo3ULg/XzJiCV82goI/AAAAAAABKnc/hO--VNayrEQe4QtfbnpC_sZwBgBmjtdTgCLcBGAsYHQ/s1920/306.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-rydjpNo3ULg/XzJiCV82goI/AAAAAAABKnc/hO--VNayrEQe4QtfbnpC_sZwBgBmjtdTgCLcBGAsYHQ/w512-h248/306.jpg" width="512" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-cNd0IeM_oRA/XzJiCk8tTFI/AAAAAAABKng/yyAoc7kiV5EiPTnOkG9DmKVFD0l32GNdACLcBGAsYHQ/s1920/307.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="929" data-original-width="1920" height="248" src="https://1.bp.blogspot.com/-cNd0IeM_oRA/XzJiCk8tTFI/AAAAAAABKng/yyAoc7kiV5EiPTnOkG9DmKVFD0l32GNdACLcBGAsYHQ/w512-h248/307.jpg" width="512" /></a></div><p></p><p><b>Step 11:</b> Download putty.exe and pagent.exe from <a href="https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html" target="_blank">here</a>. There is no need to install any of these softwares, simply download them. Start the pagent.exe and add the ppk file which has been downloaded in one of the previous steps.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-4OTehkXFoRc/XzJiC1Vz6XI/AAAAAAABKnk/2nlmtrMZ5VYNAt98FQbRJdUOuFZn0zRBACLcBGAsYHQ/s766/308.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="576" data-original-width="766" height="197" src="https://1.bp.blogspot.com/-4OTehkXFoRc/XzJiC1Vz6XI/AAAAAAABKnk/2nlmtrMZ5VYNAt98FQbRJdUOuFZn0zRBACLcBGAsYHQ/w262-h197/308.jpg" width="262" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Tk5CkJ1dxSs/XzJiDNygcXI/AAAAAAABKno/TfokN9TCVtUx3qCPzupoloDnlyRi9JTVgCLcBGAsYHQ/s774/309.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="574" data-original-width="774" height="194" src="https://1.bp.blogspot.com/-Tk5CkJ1dxSs/XzJiDNygcXI/AAAAAAABKno/TfokN9TCVtUx3qCPzupoloDnlyRi9JTVgCLcBGAsYHQ/w262-h194/309.jpg" width="262" /></a></div><p><b>Step 12:</b> Open putty.exe and in the Host Name field enter the username ubuntu and the Public IP address of the EC2 in the Public Subnet separated by the symbol @ as shown below.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-UdF02XZfY4k/XzJiDbhO2PI/AAAAAAABKns/8zo1jhxsEecYpyJLtytqdkNf8JJ8vBvpACLcBGAsYHQ/s703/310.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="700" data-original-width="703" height="510" src="https://1.bp.blogspot.com/-UdF02XZfY4k/XzJiDbhO2PI/AAAAAAABKns/8zo1jhxsEecYpyJLtytqdkNf8JJ8vBvpACLcBGAsYHQ/w512-h510/310.jpg" width="512" /></a></div><p></p><p><b>Step 13:</b> Go to Connection --> SSH --> Auth and make sure to select "Allow agent forwarding" and click on Open to connect to the EC2 instance in the Public Subnet. There is no need to specify the Private key as it has been specified in the pagent.exe.<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-1hlUbLjlBQY/XzJiDQS-sGI/AAAAAAABKnw/8q7rEUM3iwURszbcCSsN6YvOjeZSBZkBwCLcBGAsYHQ/s699/311.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="697" data-original-width="699" height="510" src="https://1.bp.blogspot.com/-1hlUbLjlBQY/XzJiDQS-sGI/AAAAAAABKnw/8q7rEUM3iwURszbcCSsN6YvOjeZSBZkBwCLcBGAsYHQ/w512-h510/311.jpg" width="512" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-L8ijThZXH4o/XzJiDjY7ccI/AAAAAAABKn0/xRsbusi4Le4E4cVwV7q61EYSQ5sxBN6hACLcBGAsYHQ/s1306/313.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="739" data-original-width="1306" height="290" src="https://1.bp.blogspot.com/-L8ijThZXH4o/XzJiDjY7ccI/AAAAAAABKn0/xRsbusi4Le4E4cVwV7q61EYSQ5sxBN6hACLcBGAsYHQ/w512-h290/313.jpg" width="512" /></a></div><p></p><p><b>Step 14:</b> From the Putty session execute the below command to connect to the EC2 in the Private Subnet. Make sure to replace 1.2.3.4 with the Private IP address of the EC2 in the Private Subnet. Note that there is no need to specify the Private key this time also as it is from the pagent.exe. Now on this EC2 instance we should be able to install any back-end applications like database, business logic and so on.<br /></p><p>ssh ubuntu@1.2.3.4<br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-MVFSTAXfjVA/XzJiD6bLH6I/AAAAAAABKn4/t-K_6J_p2gMHOcr32DETBxKgc96-cZQQwCLcBGAsYHQ/s1308/314.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="733" data-original-width="1308" height="287" src="https://1.bp.blogspot.com/-MVFSTAXfjVA/XzJiD6bLH6I/AAAAAAABKn4/t-K_6J_p2gMHOcr32DETBxKgc96-cZQQwCLcBGAsYHQ/w512-h287/314.jpg" width="512" /></a></div><div class="separator" style="clear: both; text-align: left;"><b> </b></div><div class="separator" style="clear: both; text-align: left;"><b>Step 15:</b> Finally the cleanup process</div><div class="separator" style="clear: both; text-align: left;"><br /> - Terminate the EC2 instances<br /> - Delete the NAT Gateway<br /> - Wait for a few Minutes<br /> - Delete ElasticIP<br /> - Finally delete the VPC</div><div class="separator" style="clear: both; text-align: left;"><span> </span>- Delete the Key pair <br /></div><div class="separator" style="clear: both; text-align: left;"> </div><div class="separator" style="clear: both; text-align: left;"><h1 style="text-align: left;">Conclusion</h1></div><div class="separator" style="clear: both; text-align: left;">An EC2 instances in the Private has only a Private IP and is primarily used to host back-end applications like Database, Business Logic and so on. Since it doesn't have an Public IP, we won't able to connect to out from outside the Laptop using the Private IP. We have seen how to connect to it using the EC2 instance in the Public Subnet as a Bastion or Jump box.</div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;">For this we have used pagent.exe which stores Private Key in the memory and is not a safe approach. Another approach is to copy the Private key to the Bastion or Jump box which is also not a safe approach. Both the approaches are easy to use, but are not safe and is recommended in the non-production environments. For the production environments using the <a href="https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/what-is.html" target="_blank">Client VPN</a> is the preferred approach, in this the Bastion or Jump Box is altogether avoided and all the communication between to the AWS Cloud is encrypted using IPSec protocol.<br /></div>Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-14929421658710744182020-05-19T17:24:00.000+05:302020-05-19T17:24:28.685+05:30Accessing private resources using AWS Client VPN<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="https://aws.amazon.com/vpc/" target="_blank">AWS VPC</a> supports creating public and private subnets. Any EC2 in the public subnet will have public and private IP and so front end or customer facing applications like web applications are installed on this EC2. Also, this EC2 can be reached from outside the Cloud using the public IP for any maintenance.<br />
<br />
The private subnet EC2 will have only private IP and no public IP, backend applications like databases ore installed on this EC2. This EC2 can't be reached directly from outside the Cloud as it doesn't have any public IP. So, how do we connect to it for activities like updating databases, creating tables etc? This is where Jump box or the Bastion box comes into play.<br />
<br />
To connect to the EC2 in the private subnet we need to launch a Bastion box in the public subnet, connect to it and from there connect to the EC2 in the private subnet. This corresponds to step (1) and (2) in the below diagram. Step (2) pretty much like connecting to remote server, but in this case both the EC2s are in the same <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-722JaNcLnJo/XsKJ0RS-SoI/AAAAAAABHiw/IWTbDy3lqWADGP8oivTnAejMlnGIQp1ZwCLcBGAsYHQ/s1600/002-connecting-via-bastion-host.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="515" data-original-width="980" height="210" src="https://1.bp.blogspot.com/-722JaNcLnJo/XsKJ0RS-SoI/AAAAAAABHiw/IWTbDy3lqWADGP8oivTnAejMlnGIQp1ZwCLcBGAsYHQ/s400/002-connecting-via-bastion-host.jpg" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: left;">
The problem with the above approach is that the Bastion box does have an public IP and there is a probability of someone trying to access it over the public IP. We can get rid of the Bastion Box and use <a href="https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/what-is.html" target="_blank">AWS Client VPN</a> to connect the EC2 in the private subnet although the EC2 doesn't have an public IP as shown below.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-u-uNA0evvl8/XsKJ0efK6dI/AAAAAAABHis/sOuZlcwP7EQvEdJvidQJ7i0xF8TbkQnKwCLcBGAsYHQ/s1600/003-connecting-via-vpn.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="522" data-original-width="974" height="213" src="https://1.bp.blogspot.com/-u-uNA0evvl8/XsKJ0efK6dI/AAAAAAABHis/sOuZlcwP7EQvEdJvidQJ7i0xF8TbkQnKwCLcBGAsYHQ/s400/003-connecting-via-vpn.jpg" width="400" /> </a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I have created a new VPC with public and private subnet using the <a href="https://docs.aws.amazon.com/vpc/latest/userguide/VPC_wizard.html" target="_blank">VPC wizard</a>. And then created an EC2 in the private subnet. Note that there is no public IP and just the private IP. But, still we want to connect to it for doing the maintenance work on the EC2.</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-0Vfz5lux_u4/XsKJ0bSTgMI/AAAAAAABHi0/gJ1q33SYeno4I-LIZhc9vCoX-n3bmQDiACLcBGAsYHQ/s1600/001-backend-applications-on-ec2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="667" data-original-width="1366" height="195" src="https://1.bp.blogspot.com/-0Vfz5lux_u4/XsKJ0bSTgMI/AAAAAAABHi0/gJ1q33SYeno4I-LIZhc9vCoX-n3bmQDiACLcBGAsYHQ/s400/001-backend-applications-on-ec2.jpg" width="400" /></a></div>
<br />
I followed <a href="https://prasaddomala.com/2020/04/02/aws-client-vpn-setup-private-access-across-aws-accounts-and-vpcs/" target="_blank">this</a> blog and created a VPN Endpoint and connected to the EC2 in the private subnet. Note the <a href="https://aws.amazon.com/vpn/client-vpn-download/" target="_blank">AWS Client VPN</a> on the left side and connection to the EC2 from my Laptop via the Putty on the right side. Notice that the IP address in the Putty matches with the private IP address of the EC2 in the above screen.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-n-JvatXeUCo/XsKJ1J_bHWI/AAAAAAABHi4/pTHm_cGmaTMrLoKdvB74v4ABIbRYVoAIACLcBGAsYHQ/s1600/004-vpn-client-putty.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="211" data-original-width="1312" height="63" src="https://1.bp.blogspot.com/-n-JvatXeUCo/XsKJ1J_bHWI/AAAAAAABHi4/pTHm_cGmaTMrLoKdvB74v4ABIbRYVoAIACLcBGAsYHQ/s400/004-vpn-client-putty.jpg" width="400" /></a></div>
</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com1tag:blogger.com,1999:blog-8716531089719420013.post-91392148158255472522020-05-08T15:30:00.000+05:302020-05-08T15:30:04.309+05:30Sticking to the AWS free tier and building K8S cluster using kops<div dir="ltr" style="text-align: left;" trbidi="on">
In one of the <a href="https://www.thecloudavenue.com/2020/04/microk8s-k8s-setup.html" target="_blank">previous blog</a> we have seen how to setup K8S on AWS using t2.micro EC2 instance and MicroK8S. This falls under the <a href="https://aws.amazon.com/free/" target="_blank">AWS free tier</a>. The same can be done with <a href="https://kops.sigs.k8s.io/" target="_blank">kops</a> also. This blog is all about the same. Again we will try to stick to the free tier.<br />
<br />
The first step is create an Ubuntu EC2 instance and install kops and AWS CLI on it. THen on this EC2 we would be running a bunch of commands to create and tear down the K8S cluster as shown below. The steps mentioned in this blog would be creating a K8S cluster of 1 master and 2 slaves and this would allow to try how K8S behaves for worker failures and also scheduling across multiple workers. But, this configuration requires a total of 34 GB of EBS volumes out of which 30GB is covered within the AWS free tier, rest need to be paid for. To strictly fall under the free tier only one worker node can also be created, but we won't be able to try out the different K8S features.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-p3w4S48dt7g/XrUTq2ON0wI/AAAAAAABHOY/1OCqjF-b140yATZ4eXRjrPsOj3kYIiTmgCLcBGAsYHQ/s1600/kops-k8s-cluster-diagram.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="773" data-original-width="1541" height="200" src="https://1.bp.blogspot.com/-p3w4S48dt7g/XrUTq2ON0wI/AAAAAAABHOY/1OCqjF-b140yATZ4eXRjrPsOj3kYIiTmgCLcBGAsYHQ/s400/kops-k8s-cluster-diagram.jpg" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Once the cluster has been created, we can see the nodes in the ready state as shown below.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-SfK-iZfb3MY/XrUT5fxxT1I/AAAAAAABHOc/WjRC2Q8PQxEspWRmhIV0JAOVRqAEYAzNACLcBGAsYHQ/s1600/nodes.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="147" data-original-width="724" height="80" src="https://1.bp.blogspot.com/-SfK-iZfb3MY/XrUT5fxxT1I/AAAAAAABHOc/WjRC2Q8PQxEspWRmhIV0JAOVRqAEYAzNACLcBGAsYHQ/s400/nodes.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h3 style="text-align: left;">
Steps for setting up the K8S on EC2 using kops </h3>
<br />
<b>Step 1:</b> Create an Ubuntu 18.04 EC2 instance (t2.micro) and connect to it via Putty and execute the below commands.<br />
<br />
-- Install kubectl and Python3, <a href="https://aws.amazon.com/cli/" target="_blank">AWS CLI</a> and <a href="https://kubernetes.io/docs/reference/kubectl/overview/" target="_blank">kubectl</a>.<br />
<br />
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -<br />
<br />
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list<br />
<br />
sudo apt-get update<br />
<br />
sudo apt-get install -y python3-pip apt-transport-https kubectl<br />
<br />
pip3 install awscli --upgrade<br />
<br />
export PATH="$PATH:/home/ubuntu/.local/bin/"<br />
<br />
<b>Step 2:</b> Install kops (used for installing K8S on AWS)<br />
<br />
curl -LO https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-linux-amd64<br />
<br />
chmod +x kops-linux-amd64<br />
<br />
sudo mv kops-linux-amd64 /usr/local/bin/kops<br />
<br />
<b>Step 3:</b> Check if aws, kops and kubectl commands are there in the path or not<br />
<br />
<b>Step 4:</b> Create a <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html#id_users_create_console" target="_blank">AWS IAM User</a> (with API, SDK access) with the below policy and get the <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html" target="_blank">Access Keys</a> for this user.<br />
<br />
-- AmazonEC2FullAccess<br />
-- AmazonS3FullAccess<br />
-- IAMFullAccess<br />
-- AmazonVPCFullAccess<br />
<br />
<b>Step 5:</b> Configure the Access Keys and AWS Region on the EC2 instance using the `aws configure` command. Use us-east-1 or some other region. For the Output format, leave it as empty.<br />
<br />
<b>Step 6:</b> Export the keys. Note to replace the keys after the $ symbol in the below commands.<br />
<br />
export AWS_ACCESS_KEY_ID=$(aws configure get aws_access_key_id)<br />
export AWS_SECRET_ACCESS_KEY=$(aws configure get aws_secret_access_key)<br />
<br />
<b>Step 7:</b> Generate the ssh keys<br />
<br />
ssh-keygen -f .ssh/id_rsa<br />
<br />
<b>Step 8:</b> Export the name of the cluster and the S3 bucket name. Make to sure it ends with k8s.local to use the gossip protocol or else we need to work with Route53.<br />
<br />
export NAME=myfirstcluster.k8s.local<br />
export KOPS_STATE_STORE=s3://praveen-kops-cluster<br />
<br />
<b>Step 9:</b> Create S3 bucket<br />
<br />
aws s3api create-bucket --bucket praveen-kops-cluster --region us-east-1<br />
<br />
<b>Step 10:</b> Create a cluster configuration. Note that the cluster is not built. Also, as mentioned earlier we need to pay for the additional 4GB of EBS volume. To make sure we fall under the free tier, specify the node-count as 1, to create one worker node.<br />
<br />
kops create cluster --name=$NAME --state=$KOPS_STATE_STORE --zones=us-east-1a --node-count=2 --node-size=t2.micro --master-size=t2.micro --master-volume-size=8 --node-volume-size=8<br />
<br />
<b>Step 11:</b> Change the EBS volume size for etcd main and events attached to the master. By default multiple 20GB EBS volumes are created and the total storage doesn't fall under the free tier (<a href="https://kops.sigs.k8s.io/cluster_spec/#etcdclusters" target="_blank">1</a>). Edit the cluster configuration by executing the below command.<br />
<br />
kops edit cluster myfirstcluster.k8s.local<br />
<br />
and add the below after "name: a" in two different locations.<br />
<br />
volumeType: gp2<br />
volumeSize: 1<br />
<br />
<b>Step 12:</b> Build the cluster<br />
<br />
kops update cluster $NAME --yes<br />
<br />
<b>Step 13:</b> Check the status of the cluster. The cluster and the related resources will take around a few minutes for it to be created. Also execute `kubectl get nodes` to get the status of the cluster.<br />
<br />
kops validate cluster $NAME<br />
<br />
<b>Step 14:</b> Create deployments on the cluster which we created. Create a file deployment.yaml with the contents from K8S documentation <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#creating-a-deployment" target="_blank">here</a>.<br />
<br />
-- Create a deployment using the below command.<br />
<br />
kubectl apply -f deployment.yaml<br />
<br />
-- Once the deployment has been created, check the status of the pods using the below command.<br />
<br />
kubectl get pods<br />
<br />
-- Delete the deloyment<br />
<br />
kubectl delete -f deployment.yaml<br />
<br />
<b>Step 15:</b> Delete the cluster. Again the cluster deletion will take a few minutes.<br />
<br />
kops delete cluster --name $NAME --yes<br />
<br />
<b>Step 16:</b> Delete the s3 bucket.<br />
<br />
aws s3api delete-bucket --bucket praveen-kops-cluster --region us-east-1<br />
<br />
<b>Step 17:</b> Terminate the Ubuntu EC2 instance which was created manually earlier.<br />
<br />
<b>Step 18:</b> Delete the IAM User<br />
<br />
<h3 style="text-align: left;">
Conclusion</h3>
<div style="text-align: left;">
<br />
Following the above steps a K8S cluster can be created within a few minutes on AWS and the good thing is that it falls within the AWS free tier. This is not production ready, but is good enough to get started.<br />
<br />
One more thing to note, with the kops installation we have complete control over the K8S cluster. But, the user has to manage the master and the worker nodes like upgrading, patching, securing K8S on them. And so, the Cloud vendors provide different managed K8S installations like <a href="https://aws.amazon.com/eks/" target="_blank">AWS EKS</a>, <a href="https://cloud.google.com/kubernetes-engine" target="_blank">GCP GKE</a> and <a href="https://azure.microsoft.com/en-in/services/kubernetes-service/" target="_blank">Azure AKS</a>.</div>
</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-65221557887352133022020-04-24T15:45:00.001+05:302020-04-27T10:49:47.920+05:30How K8S authentication and authorization works in kubeadm?<div dir="ltr" style="text-align: left;" trbidi="on">
To get started and practice a few things around K8S, I have setup a K8S cluster on my laptop as shown below using <a href="https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/" target="_blank">kubeadm</a>. It gives me the utmost flexibility to try different things around K8S. A few days back I have posted a blog on how a K8S cluster gets <a href="https://www.thecloudavenue.com/2020/04/k8s-cluster-bootstrap-process.html" target="_blank">bootstrapped</a>. This gives an idea on how the different K8S process start and then we would be able to change some of the K8S process startup parameters to try out new features in K8S.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-wOzwhbQtNps/XowqwXZ9O1I/AAAAAAABF_g/0eRFSx6DYJI5LL-rr3rCTFcUHrA8hQBOwCLcBGAsYHQ/s400/001-k8s-cluster-on-laptop.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="241" data-original-width="400" height="192" src="https://1.bp.blogspot.com/-wOzwhbQtNps/XowqwXZ9O1I/AAAAAAABF_g/0eRFSx6DYJI5LL-rr3rCTFcUHrA8hQBOwCLcBGAsYHQ/s320/001-k8s-cluster-on-laptop.png" width="320" /></a></div>
<br />
In the above K8S cluster, the default user (kubernetes-admin) created during the cluster setup has admin privileges to the cluster. This time I was curious on how the <a href="https://kubernetes.io/docs/reference/access-authn-authz/authentication/" target="_blank">authentication</a> and <a href="https://kubernetes.io/docs/reference/access-authn-authz/authorization/" target="_blank">authorization</a> work in K8S for this user to have full access to the cluster. This will enable me to be create additional users with different privileges, authentication and authorization mechanisms. It took me some time to get my mind/thoughts around it, but it's all interesting. This blog is all about the same.<br />
<br />
Note that the cluster has been setup using kubeadm, for kops and other the below varies a little bit. And also, kubeadm cluster setup default used X509 certificates for authentication. Authentication Providers are not built into K8S and so has to be integrated with external systems like Google Accounts, Active Directory, LDAP etc.<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<b>1)</b> By default with kubeadm installation, the user and cluster details are stored in the .kube/config file on the client side. This file helps kubectl command to connect and authenticate to the K8S cluster. The client-certificate-data and client-key-data fields in this file are used for authentication.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-HYkD0SkUbKU/XqKrFCILM1I/AAAAAAABGig/99xbUI70JxcRgyJLET_NTZ3kg2l36cLwQCLcBGAsYHQ/s1600/002-kubernetes-admin-user-details.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="414" data-original-width="480" height="343" src="https://1.bp.blogspot.com/-HYkD0SkUbKU/XqKrFCILM1I/AAAAAAABGig/99xbUI70JxcRgyJLET_NTZ3kg2l36cLwQCLcBGAsYHQ/s400/002-kubernetes-admin-user-details.png" width="400" /></a></div>
<br />
<b>2)</b> The X509 certificate from the .kube/config can be extracted and encoded. Finally the user details can be extracted with the opensll using the below commands. Note the output of the openssl command, the user name (CN) is kubernetes-admin and belongs to the system:masters organization (O).<br />
<br />
grep "client-certificate-data" config | sed 's/ client-certificate-data: //g' > encoded.txt<br />
<br />
base64 --decode encoded.txt > kubernetes-admin.crt<br />
<br />
openssl x509 -noout -subject -in kubernetes-admin.crt<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-7-Y9D6J8Sfk/XqKrFGaCN3I/AAAAAAABGik/knINnOubOfcnbfjCmkOnVMJwIk8WUyhRwCLcBGAsYHQ/s1600/003-kubernetes-admin-user-details-extract.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="149" data-original-width="1178" height="50" src="https://1.bp.blogspot.com/-7-Y9D6J8Sfk/XqKrFGaCN3I/AAAAAAABGik/knINnOubOfcnbfjCmkOnVMJwIk8WUyhRwCLcBGAsYHQ/s400/003-kubernetes-admin-user-details-extract.png" width="400" /></a></div>
<br />
<b>3)</b> Now from a K8S cluster perspective. The cluster-admin (a <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding" target="_blank">ClusterRoleBinding</a> object) binds the cluster-admin (a <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole" target="_blank">ClusterRole</a> object with admin privilages) to the system:masters group. Any user of this group will have full cluster access.<br />
<br />
kubectl describe clusterrolebindings.rbac.authorization.k8s.io cluster-admin<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-sf0VmMNxR3Y/XqKrF10_61I/AAAAAAABGis/Affv9MgtL8ESULYL7lTDj1D3Je11biD7ACLcBGAsYHQ/s1600/004-cluster-role-binding-details.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="237" data-original-width="1006" height="93" src="https://1.bp.blogspot.com/-sf0VmMNxR3Y/XqKrF10_61I/AAAAAAABGis/Affv9MgtL8ESULYL7lTDj1D3Je11biD7ACLcBGAsYHQ/s400/004-cluster-role-binding-details.png" width="400" /></a></div>
<br />
kubectl describe clusterrole cluster-admin<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-dTssGGk2Jf8/XqKrGMWfJWI/AAAAAAABGiw/CDsASgZy1twjpa9OdOoFQTeb5EZoJbyjgCLcBGAsYHQ/s1600/005-cluster-role-details.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="201" data-original-width="665" height="120" src="https://1.bp.blogspot.com/-dTssGGk2Jf8/XqKrGMWfJWI/AAAAAAABGiw/CDsASgZy1twjpa9OdOoFQTeb5EZoJbyjgCLcBGAsYHQ/s400/005-cluster-role-details.png" width="400" /></a></div>
<br />
<b>4)</b> So, kubernetes-admin which belongs to the system:masters group as seen in the output of the openssl command has full access to the cluster.<br />
<br />
<h3 style="text-align: left;">
Conclusion</h3>
<br />
K8S with the different concepts is a bit intimidating and it takes a bit of digging and research into the concepts to get a good understanding of the concepts. So, below is the mindmap for the above concept.<br />
<br />
<div style="text-align: center;">
<a href="https://1.bp.blogspot.com/-TCDgOwNUf7M/XqKrFMQLQJI/AAAAAAABGio/9fNHQFMWgo8rDXd9TAssYXJcQWvlfuUFwCLcBGAsYHQ/s1600/001-authentication-authorization-flow.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="548" data-original-width="1600" height="136" src="https://1.bp.blogspot.com/-TCDgOwNUf7M/XqKrFMQLQJI/AAAAAAABGio/9fNHQFMWgo8rDXd9TAssYXJcQWvlfuUFwCLcBGAsYHQ/s400/001-authentication-authorization-flow.png" width="400" /></a></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Note that (7 in the above diagram) maps the user details the organization name
(system-masters) in the certificate with the group name
(system-masters) in the ClusterRoleBinding. This is what gives the
kubernetes-admin user full access to the K8S cluster.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
As mentioned earlier, any user part of the system-masters group will have full access to the cluster.This Bitnami article (<a href="https://docs.bitnami.com/tutorials/configure-rbac-in-your-kubernetes-cluster/" target="_blank">1</a>) talks about the adding K8S user called 'employee' with X509 certificates. The same with some tweaks can be used to create another user besides the kubernetes-admin part of the system:master group.</div>
</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-63024093457836235942020-04-10T20:05:00.004+05:302020-04-10T20:05:58.309+05:30MicroK8S - Easiest way to get started with K8S for those familiar with AWS<div dir="ltr" style="text-align: left;" trbidi="on">
AWS provides different ways of running K8S in the Cloud via EKS, ECS and with/without Fargate. These are production ready setups for deploying K8S applications. In this blog we would look at <a href="https://microk8s.io/docs/" target="_blank">microk8s</a>, one of the most easiest way to get started with K8S. Here the assumption is that you already know how to create an EC2 instance and login to it.<br />
<br />
<h3 style="text-align: left;">
Steps for setting up the K8S on EC2 using microk8s</h3>
<br />
-- Create an Ubuntu EC2 t2.micro instance with the default Security Groups. This Security Group allows all the traffic inbound and outbound. This is OK non-production purpose. Usually depending on the core services, add-ons installed the appropriate ports have to be opened in the Security Group.<br />
<br />
<div style="text-align: center;">
<a href="https://1.bp.blogspot.com/-wfS4rNXTaCg/XpBmNNmkeeI/AAAAAAABGNM/6lrFhU3zpgMMHzDagNao0ZsJ7mxaBCCjgCLcBGAsYHQ/s1600/001-ec2-instance.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="739" data-original-width="1366" height="216" src="https://1.bp.blogspot.com/-wfS4rNXTaCg/XpBmNNmkeeI/AAAAAAABGNM/6lrFhU3zpgMMHzDagNao0ZsJ7mxaBCCjgCLcBGAsYHQ/s400/001-ec2-instance.png" width="400" /></a> </div>
<br />
-- Login to the EC2 via Putty or some other SSH client and execute the below commands.<br />
<br />
#Install the microk8s. The latest release can be got from <a href="https://microk8s.io/docs/release-notes" target="_blank">here</a>.<br />
sudo snap install microk8s --classic --channel=1.18/stable<br />
<br />
#Create an alias in the .bashrc file<br />
alias kubectl='microk8s kubectl'<br />
<br />
#Add the user part of the microk8s group and change the ownership of the ~/.kube<br />
#Exit and login back into the EC2 for these changes to take place into effect <br />
sudo usermod -a -G microk8s $USER <br />
sudo chown -f -R $USER ~/.kube<br />
<br />
#check the status of the microk8s installation<br />
microk8s status --wait-ready<br />
<br />
-- Create a file deployment.yaml with the below content. Run 'kubectl create -f deployment.yaml' to create a <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/" target="_blank">Deployment</a> with 3 <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/" target="_blank">Pods</a> with the nginx Containers. BTW, kubectl is the CLI for interacting with K8S.<br />
<br />
apiVersion: apps/v1<br />
kind: Deployment<br />
metadata:<br />
name: nginx-deployment<br />
labels:<br />
app: nginx<br />
spec:<br />
replicas: 3<br />
selector:<br />
matchLabels:<br />
app: nginx<br />
template:<br />
metadata:<br />
labels:<br />
app: nginx<br />
spec:<br />
containers:<br />
- name: nginx<br />
image: nginx:1.14.2<br />
ports:<br />
- containerPort: 80<br />
<br />
-- 'kubectl get deployments' and 'kubectl get pods' will give the list of deployments and pods created in the above step.<br />
<br />
<div style="text-align: center;">
<a href="https://1.bp.blogspot.com/-H5LzjGJonJg/XpBmNB7Ru4I/AAAAAAABGNE/onnPbYXLtjo7iHIDlAIO0HTmGtYcsCiNQCLcBGAsYHQ/s1600/002-deployments-and-pods.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="303" data-original-width="1366" height="87" src="https://1.bp.blogspot.com/-H5LzjGJonJg/XpBmNB7Ru4I/AAAAAAABGNE/onnPbYXLtjo7iHIDlAIO0HTmGtYcsCiNQCLcBGAsYHQ/s400/002-deployments-and-pods.png" width="400" /></a> </div>
<br />
--Now lets try to install the <a href="https://microk8s.io/docs/addon-dashboard" target="_blank">K8S Dashboard</a>, which is a graphical way of interfacing with K8S. Execute the below commands in the EC2.<br />
<br />
#Enable the K8S Dashboard<br />
microk8s enable dashboard<br />
<br />
#Get the token for the default user<br />
token=$(kubectl -n kube-system get secret | grep default-token | cut -d " " -f1)<br />
kubectl -n kube-system describe secret $token<br />
<br />
#Port forward the K8S Dashboard port 443 to port 10443 onto the EC2.<br />
microk8s kubectl port-forward -n kube-system service/kubernetes-dashboard 10443:443 --address 0.0.0.0<br />
<br />
-- Now the K8S Dashboard is ready to be accessed.<br />
<br />
#Open the below URL in the browser. Note to replace the ec2-ip with the Public IP of the EC2.<br />
#Provide the token got from the previous step<br />
#The Deployments and Pods created earlier can be seen.<br />
https://ec2-ip:10443/<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-iv_aF8vxJ9U/XpBmNKUS-dI/AAAAAAABGNI/xTdeGC08MBk-IgrrVTfZR3u9GAxlsbGrgCLcBGAsYHQ/s1600/003-k8s-dashboard.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="735" data-original-width="1366" height="215" src="https://1.bp.blogspot.com/-iv_aF8vxJ9U/XpBmNKUS-dI/AAAAAAABGNI/xTdeGC08MBk-IgrrVTfZR3u9GAxlsbGrgCLcBGAsYHQ/s400/003-k8s-dashboard.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-4Y2Y-dhq2Ak/XpBmNy9GFYI/AAAAAAABGNQ/GpNKCq5rE48PBY637OTykFTYQdmhBvpLgCLcBGAsYHQ/s1600/004-deplloyments-in-dashboard.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="733" data-original-width="1366" height="213" src="https://1.bp.blogspot.com/-4Y2Y-dhq2Ak/XpBmNy9GFYI/AAAAAAABGNQ/GpNKCq5rE48PBY637OTykFTYQdmhBvpLgCLcBGAsYHQ/s400/004-deplloyments-in-dashboard.png" width="400" /> </a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h3 style="text-align: left;">
Conclusion</h3>
<br />
In the above setup a <a href="https://aws.amazon.com/ec2/instance-types/t2/" target="_blank">t2.micro</a> instance was used which provides 1 vCPU and 1 GB of RAM. The only reason for using t2.micro is because it falls under the <a href="https://aws.amazon.com/free/" target="_blank">AWS Free Tier</a>. Once the Deployment and Pods are started, the CPU is reaching the peak while some memory left is as shown below. With the t2.micro I have noticed that microk8s behaves a bit slow
especially the Dashboard. Try to pickup a bigger instance for trying out
various features of K8S.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-L0221qEBufA/XpBmOqv7oLI/AAAAAAABGNU/YjM5G6ebzZU768tmKTpXQzBTt9Hbv_6GQCLcBGAsYHQ/s1600/005-cpu-utilization.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="737" data-original-width="1366" height="215" src="https://1.bp.blogspot.com/-L0221qEBufA/XpBmOqv7oLI/AAAAAAABGNU/YjM5G6ebzZU768tmKTpXQzBTt9Hbv_6GQCLcBGAsYHQ/s400/005-cpu-utilization.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-obyyYeaAF9k/XpBmO8RClXI/AAAAAAABGNY/Dr7NY-V1WBwSAXrNuQFYdGFdoTFJJkCAQCLcBGAsYHQ/s1600/006-memory-consumed.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="129" data-original-width="904" height="56" src="https://1.bp.blogspot.com/-obyyYeaAF9k/XpBmO8RClXI/AAAAAAABGNY/Dr7NY-V1WBwSAXrNuQFYdGFdoTFJJkCAQCLcBGAsYHQ/s400/006-memory-consumed.png" width="400" /></a></div>
<br />
The above steps are for creating a single node K8S cluster, multi node K8S cluster is also possible with microk8s as mentioned in the documentation <a href="https://microk8s.io/docs/clustering" target="_blank">here</a>.<br />
<br />
Setting up K8S with proper configuration especially for the Production setup is not an easy task. But, microk8s, <a href="https://k3s.io/" target="_blank">K3S</a> and <a href="https://kubernetes.io/docs/tasks/tools/install-minikube/" target="_blank">Minikube</a> have made it easy to get started with K8S.</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-51002104614230316532020-04-08T11:28:00.000+05:302020-04-08T11:28:21.463+05:30How does the K8S cluster gets bootstrapped?<div dir="ltr" style="text-align: left;" trbidi="on">
Although the different Cloud vendors provide managed service like <a href="https://aws.amazon.com/eks/" target="_blank">AWS EKS</a>, <a href="https://cloud.google.com/kubernetes-engine" target="_blank">GCP GKE</a>, <a href="https://azure.microsoft.com/en-in/services/kubernetes-service/" target="_blank">Azure AKS</a> and others, nothing beats running K8S on the local machine. Not only we can get started quickly, it's absolutely free and provides the ultimate freedom for any experimentation. Here is the setup I have using Oracle VirtualBox on Windows 10 OS.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-wOzwhbQtNps/XowqwXZ9O1I/AAAAAAABF_g/0eRFSx6DYJI5LL-rr3rCTFcUHrA8hQBOwCLcBGAsYHQ/s1600/001-k8s-cluster-on-laptop.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="574" data-original-width="951" height="241" src="https://1.bp.blogspot.com/-wOzwhbQtNps/XowqwXZ9O1I/AAAAAAABF_g/0eRFSx6DYJI5LL-rr3rCTFcUHrA8hQBOwCLcBGAsYHQ/s400/001-k8s-cluster-on-laptop.png" width="400" /></a></div>
<br />
It's a 3 node K8S cluster with one Control Plane node to do the orchestration and scheduling of the containers and two Worker nodes for the execution of the the containers. Recently, I upgraded to the latest version of K8S (1.18 as of this writing) and was able to try some of the new features.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-6rnMRQjvHTg/XowqwdJHz3I/AAAAAAABF_k/jHTcoWx5lwMPVxhTYcFf34ndGlVZHIkMwCLcBGAsYHQ/s1600/002-k8s-cluster-nodes.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="210" data-original-width="1354" height="61" src="https://1.bp.blogspot.com/-6rnMRQjvHTg/XowqwdJHz3I/AAAAAAABF_k/jHTcoWx5lwMPVxhTYcFf34ndGlVZHIkMwCLcBGAsYHQ/s400/002-k8s-cluster-nodes.png" width="400" /></a></div>
<br />
The command 'kubectl get pods --all-namespaces -o wide' gives the Pods in all the namespaces. Below is the screenshot listing the Pods on the Control Plane and the Workers node. I was curious on how the Pods get started, this will give us a chance to check out the initialization parameters, tweak them and also to enable/disable features. This blog is all about the same. Note that the instructions are specific to installation using kubeadm and differ a bit for other installation process.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-FC0pOrdnkxU/Xowqwb_WdwI/AAAAAAABF_o/WHvcMyGlEBUTxxt9e6xRWLy7K8d-etHHgCLcBGAsYHQ/s1600/003-k8s-cluster-pods.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="417" data-original-width="1600" height="103" src="https://1.bp.blogspot.com/-FC0pOrdnkxU/Xowqwb_WdwI/AAAAAAABF_o/WHvcMyGlEBUTxxt9e6xRWLy7K8d-etHHgCLcBGAsYHQ/s400/003-k8s-cluster-pods.png" width="400" /></a></div>
<br />
I was poking around in the K8S cluster and this StackOverflow Query (<a href="https://stackoverflow.com/questions/61025940/how-does-k8s-auto-start-the-deployments-and-daemonsets" target="_blank">1</a>) helped me. Below is the workflow on the how the K8S cluster gets bootstrapped. It all starts with the kubelet getting started automatically as a <a href="https://freedesktop.org/wiki/Software/systemd/" target="_blank">systemd</a> service on all the Control Plane and the Workers nodes, which starts the minimum required static Pods for K8S to start working. Once the K8S Cluster boots up, additional Pods are started to get the K8S cluster into the desired state as stored in the etcd database.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-d1ChjfsE12M/XowqxOny8YI/AAAAAAABF_s/l_fN30vzHDMKz3Y9j9bVFpVR4agew0NuwCLcBGAsYHQ/s1600/004-k8s-bootstrap-process.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="670" data-original-width="1600" height="167" src="https://1.bp.blogspot.com/-d1ChjfsE12M/XowqxOny8YI/AAAAAAABF_s/l_fN30vzHDMKz3Y9j9bVFpVR4agew0NuwCLcBGAsYHQ/s400/004-k8s-bootstrap-process.png" width="400" /></a></div>
<br />
<h3 style="text-align: left;">
Here is the workflow in a bit more detail</h3>
<br />
- Below is how the kubelet gets started as a systemd Service. /etc/systemd/system/kubelet.service.d/10-kubeadm.conf has the command to start the kubelet process and also the initialization parameters. Note that one of the parameter is /var/lib/kubelet/config.yaml location.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-OTdHEdSws2Q/XowqxWbmOcI/AAAAAAABF_w/rkHN4XmzS6w_HegmbjNOs1ahAPJSv5gzwCLcBGAsYHQ/s1600/005-k8s-kubelet-initialization.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="419" data-original-width="1590" height="105" src="https://1.bp.blogspot.com/-OTdHEdSws2Q/XowqxWbmOcI/AAAAAAABF_w/rkHN4XmzS6w_HegmbjNOs1ahAPJSv5gzwCLcBGAsYHQ/s400/005-k8s-kubelet-initialization.png" width="400" /></a></div>
<br />
- In the /var/lib/kubelet/config.yaml file the staticPodPath variable is set to /etc/kubernetes/manifests path which has the yaml files for the <a href="https://kubernetes.io/docs/tasks/configure-pod-container/static-pod/" target="_blank">static Pods</a> to be started once the kubelet starts. As of now the apiserver, scheduler and etcd haven't started yet. So, kubelet starts them and manages them. Although these Pods are visible to the apiserver later, the apiserver doesn't manage them, kubelet is the one which manages them.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-1wWh64IMPgo/Xowqxh4HjkI/AAAAAAABF_0/1TDS-fyR1rIsK5raUyvrwSvx3vXj_rZGQCLcBGAsYHQ/s1600/006-k8s-kubelet-static-pod-path.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="680" data-original-width="918" height="296" src="https://1.bp.blogspot.com/-1wWh64IMPgo/Xowqxh4HjkI/AAAAAAABF_0/1TDS-fyR1rIsK5raUyvrwSvx3vXj_rZGQCLcBGAsYHQ/s400/006-k8s-kubelet-static-pod-path.png" width="400" /></a></div>
<br />
- In the /etc/kubernetes/manifests folder we have the yaml definitions for the etcd, apiserver, contoller-manager and the scheduler. These files will help us to understand how the K8S system Pods are initialized.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-o1FaL7mql0c/Xowqx0wfMkI/AAAAAAABF_4/NzwP4eYfM1IfDYbqrkVc34sSffoc2N5-gCLcBGAsYHQ/s1600/007-k8s-static-pods.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="274" data-original-width="1254" height="86" src="https://1.bp.blogspot.com/-o1FaL7mql0c/Xowqx0wfMkI/AAAAAAABF_4/NzwP4eYfM1IfDYbqrkVc34sSffoc2N5-gCLcBGAsYHQ/s400/007-k8s-static-pods.png" width="400" /></a></div>
<br />
- OK, how about the coredns, flannel and proxy Pods getting started? The coredns and the proxy Pods are created by kubeadm during the K8S cluster initialization phase (<a href="https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init-phase/#cmd-phase-addon" target="_blank">1</a>, <a href="https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/cmd/phases/init/addons.go" target="_blank">2</a>) using kubeadm. The flannel Pods were created by me manually while setting up the K8S cluster (<a href="https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network" target="_blank">1</a>). The details of these are stored in the etcd database as with any other user created K8S objects and K8S will automatically start them once the cluster starts.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-Cmubb1fu3GI/XowqyExCliI/AAAAAAABF_8/sQLs_Xs13YUFlq441l2j7AhMc-KxU5bxQCLcBGAsYHQ/s1600/008-k8s-rest-of-the-pods.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="222" data-original-width="1142" height="77" src="https://1.bp.blogspot.com/-Cmubb1fu3GI/XowqyExCliI/AAAAAAABF_8/sQLs_Xs13YUFlq441l2j7AhMc-KxU5bxQCLcBGAsYHQ/s400/008-k8s-rest-of-the-pods.png" width="400" /></a></div>
<br />
Mystery solved, now that we know on how the K8S cluster gets bootstrapped, more can be explored. For using and administering K8S it is required to know the K8S bootstrap process in a bit more detail.</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-80480607897413044872020-04-07T08:59:00.000+05:302020-04-15T12:49:04.566+05:30How does Key Pair work behind the scenes for Linux EC2 authentication?<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: left;">
Different ways of authenticating against Linux EC2</h3>
<br />
Once a Linux EC2 instance has been created, the same can be accessed via Putty or some other SSH client. To access the EC2, first we need to authenticate the user. Either the Username/Password or the KeyPair can be used for authentication. There are pros and cons of each of them. AWS has chosen to go with the KeyPair way of authentication by default. If required this can be disabled and the Username/Password can be enabled.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-jVr6btrMt24/XovlfspjPiI/AAAAAAABF-w/fVyNw2rB3bkCBoUs7BwFTSGxKQPZMtZFACLcBGAsYHQ/s1600/001-different-ec2-authentications.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="301" data-original-width="786" height="152" src="https://1.bp.blogspot.com/-jVr6btrMt24/XovlfspjPiI/AAAAAAABF-w/fVyNw2rB3bkCBoUs7BwFTSGxKQPZMtZFACLcBGAsYHQ/s400/001-different-ec2-authentications.png" width="400" /> </a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h3 style="text-align: left;">
How does KeyPair work behind the scenes for Linux EC2 authentication?</h3>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-V-W66iy_7wg/XovlgDmlD4I/AAAAAAABF-0/9Oi2bLJTqb8AAoAMHIngUI4OD1yTF7Z6gCLcBGAsYHQ/s1600/002-keypair-authentication-mechanism.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="934" data-original-width="1600" height="232" src="https://1.bp.blogspot.com/-V-W66iy_7wg/XovlgDmlD4I/AAAAAAABF-0/9Oi2bLJTqb8AAoAMHIngUI4OD1yTF7Z6gCLcBGAsYHQ/s400/002-keypair-authentication-mechanism.png" width="400" /></a></div>
<br />
A picture conveys more than words, so is the above workflow. A KeyPair consists of a Private Key and a Public Key. The Private Key goes onto the Laptop and the Public Key automatically goes into the EC2 instance. Go through the above workflow to get to know what happens behind the scenes.<br />
<br />
Note that the Private Key never leaves the laptop, this is one of the advantages of using the Key Pairs. Also, the way we never share the passwords with anyone, we should never share the Private Key with anyone. This would allow them to access the EC2. Also, the way we never use the same password across multiple services, never we should use the same Key Pair across multiple EC2 instances for the obvious reasons.<br />
<br />
Also, it's always better to create a different set of Key Pairs for multiple users accessing the same EC2 instance, very similar to different passwords. Here is a small write-up from the AWS site on the same (<a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html" target="_blank">1</a>).<br />
<br />
<i>>> If you have several users that require access to a single instance, you can add user accounts to your instance. For more information, see Managing User Accounts on Your Linux Instance. You can create a key pair for each user, and add the public key information from each key pair to the .ssh/authorized_keys file for each user on your instance. You can then distribute the private key files to your users. That way, you do not have to distribute the same private key file that's used for the AWS account root user to multiple users.</i><br />
<br />
Here are a few articles on some of the regularly performed tasks around the combination of EC2 and Key Pairs.<br />
<br />
1) How do I enable a password login instead of a key pair when logging into my EC2 instance using SSH? (<a href="https://aws.amazon.com/premiumsupport/knowledge-center/ec2-password-login/" target="_blank">1</a>, <a href="https://stackoverflow.com/questions/6119774/ssh-to-aws-instance-without-key-pairs/7696451#7696451" target="_blank">2</a>)<br />
<br />
2) How do I add new user accounts with SSH access to my Amazon EC2 Linux instance? (<a href="https://aws.amazon.com/premiumsupport/knowledge-center/new-user-accounts-linux-instance/" target="_blank">1</a>, <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/managing-users.html" target="_blank">2</a>) <br />
<br />
3) Connecting to your Linux instance if you lose your private key (<a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#replacing-lost-key-pair" target="_blank">1</a>) <br />
<br />
4) Rotating the Key Pairs (<a href="https://stackoverflow.com/questions/7881469/change-key-pair-for-ec2-instance/36667264#36667264" target="_blank">1</a>) - Note that there are many better ways, but this is the easiest.<br />
<br />
There is lot more to Key Pairs and how the authentication works, but this article gives a basic gist on what happens BTS (behind the scenes) when we use KeyPairs to access an EC2 instance. I like to keep things clear and simple, this helps me to get my concepts clear and express myself better.</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-19653371622360481182020-04-06T10:09:00.000+05:302020-04-06T10:09:02.652+05:30AWS AMI vs Launch Templates<div dir="ltr" style="text-align: left;" trbidi="on">
Very often I get asked about the differences between <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html" target="_blank">AMI (Amazon Machine Images)</a> and <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html" target="_blank">EC2 Launch Templates</a>, although both of them have different purpose. This blog is about getting these concepts clear.<br />
<br />
<h3 style="text-align: left;">
What is AMI?</h3>
<br />
There might be a requirement where we need to install softwares and applications on hundreds of EC2s. It's not practically possible to login to each of the EC2 and perform the task as it is time consuming and also prone to errors. We can automate these tasks using AMIs. Below is the workflow to start working with the AMIs.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-AMUihYTFA5k/XomDiti4XbI/AAAAAAABF2U/u-hCoXHUj7ACs_AWdYf86t_xHD5yuInVwCLcBGAsYHQ/s1600/001-ami-creation-process.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="572" data-original-width="1600" height="142" src="https://1.bp.blogspot.com/-AMUihYTFA5k/XomDiti4XbI/AAAAAAABF2U/u-hCoXHUj7ACs_AWdYf86t_xHD5yuInVwCLcBGAsYHQ/s400/001-ami-creation-process.png" width="400" /></a></div>
<br />
Once an EC2 has been created, the appropriate software/applications
have to be installed along with the configurations and then an AMI has to be
created. The AMI has to OS and all the software/applications on top of
it. The AMI usually is a couple of GB, based on the original EC2 which
was created and is immutable (no changes can be made to it).<br />
<br />
With this
AMI, additional EC2s can be created and each one of the EC2 will have the
original software automatically installed as in the case
of Apache2. This makes getting the EC2 ready much easier/quicker.<br />
<br />
<h3 style="text-align: left;">
Different ways of launching EC2 Instance</h3>
<br />
AWS provides multiple ways of launching EC2 instances via the <a href="https://aws.amazon.com/tools/" target="_blank">SDK</a>, <a href="https://aws.amazon.com/cli/" target="_blank">CLI</a>, <a href="https://aws.amazon.com/cloudformation/" target="_blank">CloudFormation</a> and the EC2 Management Console (Web UI). From the EC2 Management Console again there are two ways, one is is using the Launch Instance wizard and other is via the EC2 Launch Templates as shown below.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-jAOVeo5RiNU/XomDih-CwVI/AAAAAAABF2Y/B5Gy8aYaRGsUDSUxVcLmeQsDhTV7JSvsACLcBGAsYHQ/s1600/002-ways-of-launching-ec2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="206" data-original-width="595" height="137" src="https://1.bp.blogspot.com/-jAOVeo5RiNU/XomDih-CwVI/AAAAAAABF2Y/B5Gy8aYaRGsUDSUxVcLmeQsDhTV7JSvsACLcBGAsYHQ/s400/002-ways-of-launching-ec2.png" width="400" /></a></div>
<br />
Both these approaches lead to launching an EC2 and takes the required parameters for the same like the AMI which was mentioned above, the EBS volume size/type, SecurityGroup, KeyPair to be used and a few other details.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-3fKqKbsbytY/XomDi0h_kgI/AAAAAAABF2c/i7NCCYQOPGYQ8w_EhZb_bsUU_KqHWXayACLcBGAsYHQ/s1600/003-ec2-creation-process.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="480" data-original-width="1248" height="153" src="https://1.bp.blogspot.com/-3fKqKbsbytY/XomDi0h_kgI/AAAAAAABF2c/i7NCCYQOPGYQ8w_EhZb_bsUU_KqHWXayACLcBGAsYHQ/s400/003-ec2-creation-process.png" width="400" /></a></div>
<br />
In the EC2 creation via the Wizard approach we need to select each and every time the AMI, Instance Type, Network Settings, Storage, Security Groups, pricing model etc from the different options. In the below screen the AMI has to be selected from the available ones. This is OK when we launch an EC2 a few times, but it's more of a routine task and is also time consuming.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-j9hQyOCUUrw/XomDjjMQ8hI/AAAAAAABF2g/QJ4r9p8vTokP8BWyyvk-YIAvG3_xaDDMwCLcBGAsYHQ/s1600/004-ec2-via-wizard.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="870" data-original-width="1600" height="216" src="https://1.bp.blogspot.com/-j9hQyOCUUrw/XomDjjMQ8hI/AAAAAAABF2g/QJ4r9p8vTokP8BWyyvk-YIAvG3_xaDDMwCLcBGAsYHQ/s400/004-ec2-via-wizard.png" width="400" /></a></div>
<br />
This is where the EC2 Launch Templates come to the rescue. We can create a template and predefine all the EC2 properties and reuse the same to launch an EC2 instance. This way we don't need to select the EC2 properties again and again. Below is the template with the AMI, instance type, KeyPair, SecurityGroup predefined. Using this template we should be able to create an EC2 instance using Option (6). <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-JBQvw2BKwlI/XomDj3liK5I/AAAAAAABF2k/22XHTYGHjY8MuU1wMT2LoDWZReIkbDKlQCLcBGAsYHQ/s1600/005-ec2-via-launch-template.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="848" data-original-width="1600" height="211" src="https://1.bp.blogspot.com/-JBQvw2BKwlI/XomDj3liK5I/AAAAAAABF2k/22XHTYGHjY8MuU1wMT2LoDWZReIkbDKlQCLcBGAsYHQ/s400/005-ec2-via-launch-template.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<h3 style="text-align: left;">
CloudFormation Templates vs EC2 Launch Templates</h3>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
While the EC2 Launch Templates can be used for the automation of the EC2 Instance creation. CloudFormation Templates are much more than that. It's possible to create many of the AWS resources via the CloudFormation Templates and connect them together, watch the drift (changes to the AWS resources) and much more. <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html" target="_blank">Here</a> is the list of AWS Resources that can be created by CloudFormation.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-D6634mgvIzE/XomDj_Zx8cI/AAAAAAABF2o/eYy3X00ATpI3NtPWE9EoziuB8XImWHGXACLcBGAsYHQ/s1600/006-cloudformation-vs-ec2-launch-templates.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="571" data-original-width="1292" height="176" src="https://1.bp.blogspot.com/-D6634mgvIzE/XomDj_Zx8cI/AAAAAAABF2o/eYy3X00ATpI3NtPWE9EoziuB8XImWHGXACLcBGAsYHQ/s400/006-cloudformation-vs-ec2-launch-templates.png" width="400" /></a></div>
</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com3tag:blogger.com,1999:blog-8716531089719420013.post-39142320367344087832020-03-31T20:43:00.000+05:302020-03-31T20:45:51.073+05:30Debugging K8S applications with Ephemeral Containers<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<span style="font-size: small;"><span style="font-family: inherit;"><a href="https://1.bp.blogspot.com/-v10dpa67jMA/XoMiJxtdfII/AAAAAAABFgI/GohKAOtew28NWy2Yno0UQoNPCd01ZENAgCLcBGAsYHQ/s1600/k8s-debugging-ephemeral-container.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="451" data-original-width="1302" height="136" src="https://1.bp.blogspot.com/-v10dpa67jMA/XoMiJxtdfII/AAAAAAABFgI/GohKAOtew28NWy2Yno0UQoNPCd01ZENAgCLcBGAsYHQ/s400/k8s-debugging-ephemeral-container.png" width="400" /></a></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">It's always CRITICAL to pack a Container image with the minimal binaries required as this makes the surface area of attack minimal, upgrading the image and testing also becomes easier as there are less variables to be addressed. <a href="https://github.com/GoogleContainerTools/distroless" target="_blank">Distroless Docker images</a> can be used for the same. In the above diagram Container (A) has only the application and the dependent binaries and nothing more. So, if there are no debugging tools in the Container (A) nor any way to check the status of the process then how do we debug any problem in the application? Once a pod is created, it's even not possible to add Containers to it for additional debugging tools.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">That's where the Ephemeral Containers come into picture as in the Container (B) in the above picture. These Containers are temporary that can be included in the Pod dynamically with additional debugging tools. Once a Ephemeral Container has been created, we can connect to it as usual using the kubectl attach, kubectl exec and kubectl logs commands.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">Here are the steps for creating an Ephemeral Container. The assumption is that <a href="lchttps://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/" target="_blank">kubeadm</a> has been used to create the K8S Cluster and the Cluster is using K8S 1.18 version or newer which is the latest as of this writing.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><b>Step 1)</b> The Ephemeral Containers feature has to be enabled before creating it. More about enabling K8S features <a href="https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/" target="_blank">here</a>.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">Edit the below files to add '- --feature-gates=EphemeralContainers=true' in the command section. there is no need to restart the Pods as the kubelet process continuously monitors these files and restarts the Pods automatically. Run the 'kubectl get pods --all-namespaces' and notice the changes to the AGE column.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">/etc/kubernetes/manifests/kube-apiserver.yaml</span></span><br />
<span style="font-size: small;"><span style="font-family: inherit;">/etc/kubernetes/manifests/kube-scheduler.yaml</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><b>Step 2)</b> Edit the '/etc/systemd/system/kubelet.service.d/10-kubeadm.conf' to pass the '--feature-gates=EphemeralContainers=true' parameter to kubelet as shown below.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">ExecStart=/usr/bin/kubelet <b>--feature-gates=EphemeralContainers=true</b> $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><b>Step 3)</b> Restart the kubelet service for the above changes to take place into effect.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="margin: 0in; text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">sudo systemctl
daemon-reload</span></span></div>
<div style="margin: 0in; text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">sudo systemctl
restart kubelet</span></span></div>
<div style="margin: 0in; text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="margin: 0in; text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">Note that Step 2 and 3 have to be run on all the Control Plane Nodes and the Worker Nodes. </span></span></div>
<div style="margin: 0in; text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="margin: 0in; text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><b>Step 4)</b> Now that the feature has been enabled, lets create a Pod with an application. For sake of simplicity, a pause Pod is created using the below command.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">kubectl run ephemeral-demo --image=k8s.gcr.io/pause --restart=Never</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">There is no shell or any other debugging tools in this container, so the below command will fail.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">kubectl exec -it ephemeral-demo -- sh </span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><b>Step 5)</b> Let's start an Ephemeral Container with the below command. Note that this particular feature is in alpha release of K8S, so the parameter alpha in the kubectl command. Once the feature graduates to stable, we don't need this any more. The below command opens a terminal to the Ephemeral Container and from where we can start debugging the main application.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">kubectl <b>alpha</b> debug -it ephemeral-demo --image=busybox --target=ephemeral-demo </span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><b>Step 6)</b> Open a new terminal and run the below command to get the details of the Ephemeral Container.</span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">kubectl describe pod ephemeral-demo</span></span><br />
<span style="font-size: small;"><span style="font-family: inherit;"><br /></span></span>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-size: small;"><span style="font-family: inherit;"><a href="https://1.bp.blogspot.com/-VdqKOfBPi-k/XoM5cDmAuxI/AAAAAAABFgU/3gXttj4juUwnqYt1VVBAVi010vf_t5T3wCLcBGAsYHQ/s1600/k8s-epheemeral-container.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="218" data-original-width="895" height="96" src="https://1.bp.blogspot.com/-VdqKOfBPi-k/XoM5cDmAuxI/AAAAAAABFgU/3gXttj4juUwnqYt1VVBAVi010vf_t5T3wCLcBGAsYHQ/s400/k8s-epheemeral-container.png" width="400" /></a></span></span></div>
</div>
<div style="text-align: left;">
<span style="font-size: small;"><span style="font-family: inherit;">Hope you had fun learning about Ephemeral Containers, more about <a href="https://kubernetes.io/docs/tasks/debug-application-cluster/debug-running-pod/#ephemeral-container" target="_blank">here</a> and <a href="https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/" target="_blank">here</a>.</span></span></div>
</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-56291155627876116512020-01-13T19:14:00.000+05:302020-01-13T19:23:38.013+05:30Prajval in '32nd South Zone Aquatic Championship – 2019'<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal">
My son SS Prajval got Silver Medal in the finals of ‘4x50 mts
Medley Relay – Group III – Boys’ <a href="https://swimindia.in/complete-updates-32nd-south-zone-aquatic-championship-2019-hyderabad" target="_blank">32nd South Zone Aquatic Championship– 2019</a>. This was the first time Prajval represented in the finals of
the South Zone Competitions and he was able to grab a Silver Medal in the same.<br />
<br />
The District and the State level competitions were held at ‘B V Gurumurthy Memorial MCH Swimming
Pool, Secunderabad, while the finals were held at ‘GMC Balayogi Athletic
Stadium, Gachibowli, Hyderabad’. These events were conducted by the Telangana Swimming Association. The event was well covered by all the major media outlets. </div>
<div class="MsoNormal">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-J9gSIiJiPbA/XhxI2_QcwbI/AAAAAAABBDI/9Qj6z5_S-JwFi9P-dqfZpuCDqkP3iX-WgCLcBGAsYHQ/s1600/swimming-pool-2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="451" data-original-width="1600" height="111" src="https://1.bp.blogspot.com/-J9gSIiJiPbA/XhxI2_QcwbI/AAAAAAABBDI/9Qj6z5_S-JwFi9P-dqfZpuCDqkP3iX-WgCLcBGAsYHQ/s400/swimming-pool-2.jpg" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-FWs2_iif8Gg/XhxI7Wq36nI/AAAAAAABBDM/8ixqIkaM-x0THhkmBpfBUbvhulprn1WDgCLcBGAsYHQ/s1600/swimming-pool-1.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://1.bp.blogspot.com/-FWs2_iif8Gg/XhxI7Wq36nI/AAAAAAABBDM/8ixqIkaM-x0THhkmBpfBUbvhulprn1WDgCLcBGAsYHQ/s400/swimming-pool-1.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">GMC Balayogi Athletic
Stadium, Gachibowli, Hyderabad</td></tr>
</tbody></table>
<br />
<div class="MsoNormal">
In the finals Swimmers from Andhra Pradesh, Telangana, Kerala,
Karnataka, Tamil Nadu and Puducherry (Southern States of India) represented. It was nice to see kids really motivated with the Swimming. The
final events were held on 3<sup>rd</sup>/4<sup>th</sup>/5<sup>th</sup> of
January 2020. We had been to the events on
the all three days and it was an enriching and motivating experience for Prajval
and us.</div>
<div class="MsoNormal">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-DCv-OKHoWgc/XhxJFrRzqjI/AAAAAAABBDQ/Ua8_FO4quYM14aTvE85XU-dsv3QQk1NPgCLcBGAsYHQ/s1600/receiving-silver-medal.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1280" data-original-width="960" height="400" src="https://1.bp.blogspot.com/-DCv-OKHoWgc/XhxJFrRzqjI/AAAAAAABBDQ/Ua8_FO4quYM14aTvE85XU-dsv3QQk1NPgCLcBGAsYHQ/s400/receiving-silver-medal.jpg" width="300" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Receiving the Silver Medal</td></tr>
</tbody></table>
<div class="MsoNormal">
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-1ABRGbpmZWs/XhxJLzTSyuI/AAAAAAABBDY/J8FgBVP3FTAm6pMERQE1TOexj4W6wKWOgCLcBGAsYHQ/s1600/with-mr-satish.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1600" data-original-width="1200" height="400" src="https://1.bp.blogspot.com/-1ABRGbpmZWs/XhxJLzTSyuI/AAAAAAABBDY/J8FgBVP3FTAm6pMERQE1TOexj4W6wKWOgCLcBGAsYHQ/s400/with-mr-satish.jpg" width="300" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">With his Swimming Coach Mr Satish Balasubramanian (NIS Coach)</td></tr>
</tbody></table>
<br />
<div class="MsoNormal">
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-l6xyUbkaA5A/XhxJSaeL1nI/AAAAAAABBDc/4p1KXmFSHykmT_147HBHZ758xfr740LcQCLcBGAsYHQ/s1600/with-pool-friends-001.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://1.bp.blogspot.com/-l6xyUbkaA5A/XhxJSaeL1nI/AAAAAAABBDc/4p1KXmFSHykmT_147HBHZ758xfr740LcQCLcBGAsYHQ/s400/with-pool-friends-001.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">With his buddies - BEFORE the competition</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-fiIvbIw_PR8/XhxJSnEaUJI/AAAAAAABBDg/-z9k6UKz5Rsm87v9aei9eNKqUdw4LRMOACLcBGAsYHQ/s1600/with-pool-friends-002.jpg" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="300" src="https://1.bp.blogspot.com/-fiIvbIw_PR8/XhxJSnEaUJI/AAAAAAABBDg/-z9k6UKz5Rsm87v9aei9eNKqUdw4LRMOACLcBGAsYHQ/s400/with-pool-friends-002.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">With his buddies - AFTER the competetion</td></tr>
</tbody></table>
</div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-VCYg5a09uXE/XhxJ_K0GJLI/AAAAAAABBDw/O9_Y8dGdpRshqgEznD_JRghkCb6ruAUJQCLcBGAsYHQ/s1600/news-article-1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="384" data-original-width="813" height="188" src="https://1.bp.blogspot.com/-VCYg5a09uXE/XhxJ_K0GJLI/AAAAAAABBDw/O9_Y8dGdpRshqgEznD_JRghkCb6ruAUJQCLcBGAsYHQ/s400/news-article-1.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sakshi </td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-3QYCwjntEpQ/XhxJ_BqoakI/AAAAAAABBD0/7nNEc87ZA0gJdYNUQaWADZN6QfBzxBQwwCLcBGAsYHQ/s1600/news-article-2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="464" data-original-width="439" height="400" src="https://1.bp.blogspot.com/-3QYCwjntEpQ/XhxJ_BqoakI/AAAAAAABBD0/7nNEc87ZA0gJdYNUQaWADZN6QfBzxBQwwCLcBGAsYHQ/s400/news-article-2.png" width="377" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Deccan Chronicle</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-OklG5wSij64/XhxJ_o6lukI/AAAAAAABBD4/ADX34CYjvowUD7gpVLLv_mtR6ddPLp3vACLcBGAsYHQ/s1600/news-article-3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="480" data-original-width="563" height="340" src="https://1.bp.blogspot.com/-OklG5wSij64/XhxJ_o6lukI/AAAAAAABBD4/ADX34CYjvowUD7gpVLLv_mtR6ddPLp3vACLcBGAsYHQ/s400/news-article-3.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sakshi</td></tr>
</tbody></table>
<div class="MsoNormal">
It's a tough balancing act between studies, swimming and peer pressures. But it's good to see that the kids are able to balance and prioritize them. Wish him luck for all the future competitions. </div>
<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-IN</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="false"
DefSemiHidden="false" DefQFormat="false" DefPriority="99"
LatentStyleCount="376">
<w:LsdException Locked="false" Priority="0" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 9"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="header"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footer"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index heading"/>
<w:LsdException Locked="false" Priority="35" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of figures"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope return"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="line number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="page number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of authorities"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="macro"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="toa heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 5"/>
<w:LsdException Locked="false" Priority="10" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Closing"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Signature"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="true"
UnhideWhenUsed="true" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Message Header"/>
<w:LsdException Locked="false" Priority="11" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Salutation"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Date"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Block Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Hyperlink"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="FollowedHyperlink"/>
<w:LsdException Locked="false" Priority="22" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Document Map"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Plain Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="E-mail Signature"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Top of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Bottom of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal (Web)"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Acronym"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Cite"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Code"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Definition"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Keyboard"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Preformatted"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Sample"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Typewriter"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Variable"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Table"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation subject"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="No List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Contemporary"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Elegant"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Professional"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Balloon Text"/>
<w:LsdException Locked="false" Priority="39" Name="Table Grid"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Theme"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" QFormat="true"
Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" QFormat="true"
Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" QFormat="true"
Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" QFormat="true"
Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" QFormat="true"
Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" QFormat="true"
Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" SemiHidden="true"
UnhideWhenUsed="true" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="TOC Heading"/>
<w:LsdException Locked="false" Priority="41" Name="Plain Table 1"/>
<w:LsdException Locked="false" Priority="42" Name="Plain Table 2"/>
<w:LsdException Locked="false" Priority="43" Name="Plain Table 3"/>
<w:LsdException Locked="false" Priority="44" Name="Plain Table 4"/>
<w:LsdException Locked="false" Priority="45" Name="Plain Table 5"/>
<w:LsdException Locked="false" Priority="40" Name="Grid Table Light"/>
<w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="46" Name="List Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Mention"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Smart Hyperlink"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Hashtag"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Unresolved Mention"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Smart Link"/>
</w:LatentStyles>
</xml><![endif]--><!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin-top:0cm;
mso-para-margin-right:0cm;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}
</style>
<![endif]--></div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-74423173281968675132019-12-09T15:47:00.001+05:302019-12-09T15:50:26.116+05:30Running Containers on K8S using AWS Fargate<div dir="ltr" style="text-align: left;" trbidi="on">
Container orchestration is all in hype. And there are different ways of running containers on AWS, using either <a href="https://aws.amazon.com/eks/" target="_blank">EKS</a> or <a href="https://aws.amazon.com/ecs/" target="_blank">ECS</a>. EKS uses K8S behind the scenes and ECS uses AWS proprietary technologies.With EKS it's easy to migrate containers from one Cloud to another, but not with ECS. In fact <a href="https://cloud.google.com/anthos/" target="_blank">Google Cloud Anthos</a> makes it easy to manage K8S across Clouds and on premise.<br />
<div style="text-align: center;">
<a href="https://1.bp.blogspot.com/-if4uPfWCxyI/Xe3faIAYiSI/AAAAAAAA-9U/glxo8laQt70BP8mof8JQoL8edlwtbFWtgCLcBGAsYHQ/s1600/containers-on-aws.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="348" data-original-width="504" height="220" src="https://1.bp.blogspot.com/-if4uPfWCxyI/Xe3faIAYiSI/AAAAAAAA-9U/glxo8laQt70BP8mof8JQoL8edlwtbFWtgCLcBGAsYHQ/s320/containers-on-aws.png" width="320" /></a><br />
<div style="text-align: left;">
With the EKS, AWS has announced <a href="https://aws.amazon.com/blogs/containers/eks-managed-node-groups/" target="_blank">Managed Node Groups</a> which takes away the burden of maintaining the K8S worker nodes. In the recent re:Invent 2019 AWS <a href="https://aws.amazon.com/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/" target="_blank">announced</a> another exiting feature around EKS. Now, it's possible to run EKS the <a href="https://aws.amazon.com/fargate/" target="_blank">Fargate</a> way as mentioned in the above diagram as Option 3. The rest of the options had been there for some time. AWS Fargate follows the serverless pattern and there is no need to think in terms of number of EC2 and size of them. All we need is to create an EKS Cluster and run the Pods on them. We exactly <a href="https://aws.amazon.com/fargate/pricing/" target="_blank">pay</a> for the vCPU and Memory resources consumed by the Pods.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Here are the steps for creating an AWS EKS Cluster using the eksctl via Fargate approach.</div>
<div style="text-align: left;">
<br />
<b>Step 1</b>: Create an Ubuntu EC2 Instance (t2.micro) and connect to
it. On this Instance we would be running the eksctl and other commands
for creating the AWS EKS Cluster.<br />
<br />
<b>Step 2:</b> Execute the below commands on Ubuntu to create key pairs
and install AWS CLI, aws-iam-authenticator, kubectl and eksctl
softwares.<br />
<br />
#installation of the required software<br />
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -<br />
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list<br />
<br />
sudo apt-get update<br />
sudo apt-get install -y python3-pip kubectl<br />
<br />
pip3 install awscli --upgrade<br />
export PATH="$PATH:/home/ubuntu/.local/bin/"<br />
<br />
curl --silent --location
"https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname
-s)_amd64.tar.gz" | tar xz -C /tmp<br />
sudo mv /tmp/eksctl /usr/local/bin<br />
<br />
<b>Step 3:</b> Get the access keys for the root and provide them using the `aws configure` command.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b>Step 4:</b> From here on use the steps mentioned in AWS Blog '<a href="http://Amazon EKS on AWS Fargate Now Generally Available" target="_blank">Amazon EKS on AWS Fargate Now Generally Available</a>'.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Note that there is a cost associated for running the EKS K8S Cluster and also for the NAT Gateway which is part of the VPC created in the above steps mentioned in the AWS Blog (Step 4). Also, make sure to <a href="https://docs.aws.amazon.com/cli/latest/reference/eks/delete-cluster.html#examples" target="_blank">delete</a> the EKS Cluster and any other AWS resources created as part of the sequence of steps.</div>
</div>
</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-41149356782841339182019-11-29T16:11:00.002+05:302020-10-02T11:19:01.722+05:30Changes to the AWS EC2 Instance Metadata Service (IMDS) around the recent Capital One hack<div dir="ltr" style="text-align: left;" trbidi="on">
Captial One Bank (<a href="https://www.capitalone.com/facts2019/" target="_blank">1</a>) and 30 different organizations were hacked around end of July, I have written a blog (<a href="http://www.thecloudavenue.com/2019/08/how-capital-one-hack-was-achieved-in-aws.html" target="_blank">1</a>) around the same time on how to recreate the hack in your own AWS account and also a few mitigations around the same. Now, AWS has made a few changes to the AWS EC2 Instance Metadata Service (IMDS) around the same (<a href="https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/" target="_blank">1</a>, <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html" target="_blank">2</a>). <a href="https://www.youtube.com/watch?v=2B5bhZzayjI" target="_blank">Here</a> (Security best practices for the Amazon EC2 instance metadata service) is the AWS re:Invent 2019 session around the same.<br />
<br />
The old/existing approach is called IMDSv1 and the new one IMDSv2. Although IMDSv1 solves a few problems like not storing the access keys on the EC2, it bought its own headaches which lead to the hacks. Earlier the access keys for a IAM Role can be got using the below command. But, with the new IMDSv2 the same command would lead to `401 - Unauthorized` error after enabling IMDSv2. This would block <a href="https://www.hackerone.com/blog-How-To-Server-Side-Request-Forgery-SSRF" target="_blank">SSRF</a> and other such attacks.<br />
<br />
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/Role4EC2-MetaDataMod<br />
<br />
Lets try to see it in action with the sequence of steps.<br />
<br />
<b>Step 1:</b> Create an Ubuntu EC2 (t2.micro) and login to it via Putty.<br />
<br />
<b>Step 2:</b> Create a role (Role4EC2-MetaDataMod) with the below JSON Policy (MetaDataModPolicy). Attach the role to the EC2.<br />
<br />
{<br />
"Version": "2012-10-17",<br />
"Statement": [<br />
{<br />
"Effect": "Allow",<br />
"Action": "ec2:ModifyInstanceMetadataOptions",<br />
"Resource": "*"<br />
}<br />
]<br />
}<br />
<br />
<b>Step 3:</b> Execute the below commands on the EC2 instance to configure the AWS CLI.<br />
<br />
sudo apt-get update; sudo apt-get install -y python3-pip<br />
pip3 install awscli --upgrade<br />
export PATH="$PATH:/home/ubuntu/.local/bin/"<br />
<br />
Configure the AWS CLI, make sure to provide only the region (us-east-1 or some other) and rest of them as blank.<br />
<br />
aws configure<br />
<br />
<b>Step 4:</b> Execute the below command on the EC2 instance to get the access keys associated with the IAM Role. The access keys would be displayed in the console.<br />
<br />
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/Role4EC2-MetaDataMod<br />
<br />
<b>Step 5:</b> Turn off IMDSv1 executing the below command on the EC2 instance, by default both (IMDSv1 and IMDSv2) of them are turned on. Make sure to replace the EC2 instance-id in the below command. With this IMDSv1 is disabled and IMDSv2 is enabled. If IMDSv1 and IMDSv2 both should be enabled for the sake of compatibility with the existing applications the http-token can be set to optional. More on the command syntax and the documentation <a href="https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-metadata-options.html" target="_blank">here</a>.<br />
<br />
aws ec2 modify-instance-metadata-options <b>--instance-id i-053cb17f19ca95067</b> --profile default --http-endpoint enabled --http-token required<br />
<br />
<b>Step 6:</b> Now the same command from Step 4 leads to `401 - Unauthorized` error message. as shown in the below screen.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-fKY8UwhZtcg/XeDuVNCx44I/AAAAAAAA-aU/mW8Uq7tUETcbF7j0VuCFS9CbRuFPqoE6wCLcBGAsYHQ/s1600/001-access-denied-after-enabling-IMDSv2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="301" data-original-width="972" height="123" src="https://1.bp.blogspot.com/-fKY8UwhZtcg/XeDuVNCx44I/AAAAAAAA-aU/mW8Uq7tUETcbF7j0VuCFS9CbRuFPqoE6wCLcBGAsYHQ/s400/001-access-denied-after-enabling-IMDSv2.png" width="400" /></a></div>
<br />
<b>Step 7:</b> With the IMDSv2 a session token has to be got via the HTTP PUT method and then the token used to retrieve the access keys or in-fact with any of the EC2 Instance Metadata Service.<br />
<br />
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`<br />
echo $TOKEN<br />
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/Role4EC2-MetaDataMod -H "X-aws-ec2-metadata-token: $TOKEN"<br />
<br />
Now the access keys can be got as shown in the below screen.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-ai5GFRfG2cY/XeDu7dJXF_I/AAAAAAAA-ac/kqP1XaeB3XUVayNsQrMA8oyf3jWAcip6gCLcBGAsYHQ/s1600/002-access-keys-after-passing-secret-token.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="508" data-original-width="1287" height="157" src="https://1.bp.blogspot.com/-ai5GFRfG2cY/XeDu7dJXF_I/AAAAAAAA-ac/kqP1XaeB3XUVayNsQrMA8oyf3jWAcip6gCLcBGAsYHQ/s400/002-access-keys-after-passing-secret-token.png" width="400" /> </a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h3 style="text-align: left;">
How IMDSv2 would prevent the hack from happening?</h3>
<br />
IMDSv2 would not prevent hacks like the Capital One hack from happening, but it addresses many misconfigurations and application bugs to some extent. There might be still a probability that a WAF is not configured properly to block the HTTP PUT requests with a bug in the application code to get the IMDS token and access the EC2 Metadata Service form outside the EC2.<br />
<br />
For those who are hosting applications on AWS, would recommend enabling IMDSv2 and disabling IMDSv1 as mentioned here (<a href="https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/" target="_blank">1</a>, <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html" target="_blank">2</a>, <a href="https://docs.aws.amazon.com/cli/latest/reference/ec2/modify-instance-metadata-options.html" target="_blank">3</a>). But, before making the changes make sure the applications on AWS EC2 are compatible with IMDSv2. There won't be any changes required for any applications using the AWS SDK as this internally gets the token and then accesses the EC2 Instance Metadata Service using the token. But, if your application accesses the EC2 Instance Metadata Service using the HTTP endpoint as in the case of Step 4, it would require changes to the code to get the token.<br />
<br />
This is a nice step from AWS, but it took more than 100 days for them to come up with a solution to block more such attacks. GCP has also the instance metadata service (<a href="https://cloud.google.com/compute/docs/storing-retrieving-metadata" target="_blank">1</a>), not really sure if the same vulnerability is in GCP also and how GCP handles it. If you are familiar with how GCP tackles it, please let me know in the comments section and I will update this blog.<br />
<br />
Along, the same lines AWS also introduced managed WAF rules (<a href="https://aws.amazon.com/blogs/aws/announcing-aws-managed-rules-for-aws-waf/" target="_blank">1</a>) to avoid similar attacks.</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com1tag:blogger.com,1999:blog-8716531089719420013.post-62138572150892354982019-11-28T12:41:00.000+05:302019-11-28T13:04:42.312+05:30Creating a K8S Cluster on AWS using eksctl<div dir="ltr" style="text-align: left;" trbidi="on">
As mentioned in the official K8S documentation <a href="https://kubernetes.io/docs/setup/" target="_blank">(1)</a>, there are different ways of setting up K8S, some used for learning purpose and some for production setup. Same is the case, there are different ways of setting up K8S on AWS (<a href="https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html" target="_blank">1</a>). Today we will explore setting up using the eksctl way (<a href="https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html" target="_blank">1</a>) which is kinda easy. There are lot of tools which make the K8S installation easier, but if you are looking on how to build from scratch then K8S the hard way (<a href="https://github.com/kelseyhightower/kubernetes-the-hard-way" target="_blank">1</a>) is the way to go. This will also help with the CKA certification (<a href="https://www.cncf.io/certification/cka/" target="_blank">1</a>).<br />
<br />
As of this writing AWS EKS charges $0.20 per hour for each Cluster created, this is independent of the worker nodes in the Cluster. And there is a separate charge for EC2 and EBS for the worker nodes. As the Cluster charges are flat, there is no way to optimize it. So, I used a t2.micro for the worker nodes to optimize the cost. But, it didn't work out as t2.micro supports a maximum of 2 network interfaces and 2 IPv4 addresses per network interface (<a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI" target="_blank">1</a>). This boils to the fact that t2.micro can have maximum of 4 IP addresses.<br />
<br />
Whenever a new Pod is created on the worker node, AWS allocates a new IP address from the VPC subnet. As mentioned above, t2.micro can have a maximum of 4 IP address, one of which is attached to the EC2 itself. There are 3 more IP address left, which can be allocated to the Pods on the worker nodes. This makes is difficult to use the t2.micro EC2 Instance which falls under the AWS free tier, as some of the pods are used by K8S installation itself. The below steps would be using a single EC2 Spot Instance (t3.small or t2.medium) for the worker nodes.<br />
<br />
Also, AWS has recently introduced managed Node Groups (<a href="https://aws.amazon.com/blogs/containers/eks-managed-node-groups/" target="_blank">1</a>). With this most of the grunt work like upgrading K8S on the worker nodes is managed by AWS with no additional cost. So, we have node-groups which have to be managed by the customer and the new managed-node-groups which are managed by AWS.<br />
<br />
Node-groups had been there for some time and support EC2 Spot Instances, but managed-node-groups is relatively new and looks like it doesn't support the EC2 Spot Instances as of now.<br />
<br />
Here are the steps for creating an AWS EKS Cluster using the eksctl. References (<a href="https://eksctl.io/usage/managing-nodegroups/" target="_blank">1</a>, <a href="https://eksctl.io/usage/eks-managed-nodegroups/" target="_blank">2</a>, <a href="https://eksctl.io/usage/spot-instances/" target="_blank">3</a>)<br />
<br />
<b>Step 1</b>: Create an Ubuntu EC2 Instance (t2.micro) and connect to it. On this Instance we would be running the eksctl and other commands for creating the AWS EKS Cluster.<br />
<br />
<b>Step 2:</b> Execute the below commands on Ubuntu to create key pairs and install AWS CLI, aws-iam-authenticator, kubectl and eksctl softwares.<br />
<br />
#generation of ssh keypairs to be used by the worker K8S Instances<br />
ssh-keygen -f .ssh/id_rsa<br />
<br />
#installation of the required software<br />
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -<br />
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list<br />
<br />
sudo apt-get update<br />
sudo apt-get install -y python3-pip kubectl<br />
<br />
curl -o aws-iam-authenticator https://amazon-eks.s3-us-west-2.amazonaws.com/1.14.6/2019-08-22/bin/linux/amd64/aws-iam-authenticator<br />
chmod +x ./aws-iam-authenticator<br />
sudo mv ./aws-iam-authenticator /usr/local/bin<br />
<br />
pip3 install awscli --upgrade<br />
export PATH="$PATH:/home/ubuntu/.local/bin/"<br />
<br />
curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp<br />
sudo mv /tmp/eksctl /usr/local/bin<br />
<br />
<b>Step 3:</b> Get the access keys for the root and provide them using the `aws configure` command.<br />
<b><br /></b>
<b>Step 4:</b> Create a cluster.yaml file for a nodegroup or managed nodegroup and start creating the Cluster. The yaml configuration file for both of them has been mentioned below. It would take 10-15 minutes time. This configuration will use existing VPC, note to change the availability zone and subnet-id in the yaml to where the worker nodes have to be deployed.<br />
<br />
eksctl create cluster -f cluster.yaml<br />
<br />
Check the number of nodes in the cluster.<br />
<br />
kubectl get nodes<br />
<br />
Once the cluster has been created the below command can be used to login to each of the worker nodes. Make sure to replace the EC2 IP of the worker nodes.<br />
<br />
ssh -i ./.ssh/id_rsa ec2-user@3.81.92.65<br />
<br />
<b>Step 5:</b> Create a deployment with 2 ngnix pods and get the pod details.<br />
<br />
kubectl run nginx --image=nginx -r=2<br />
kubectl getpods -o wide<br />
<br />
<b>Step 6:</b> Delete the Cluster. Again, the deletion of the Cluster would take 10-15 minutes of time. The progress would be displayed in the console.<br />
<br />
eksctl delete cluster --wait --region=us-east-1 --name=praveen-k8s-cluster<br />
<br />
<b>Step 7:</b> Make sure to terminate the EC2 Instance created in Step 1. <br />
<br />
<b># An example of ClusterConfig showing nodegroups with spot instances </b><br />
<pre class="brush:xml">---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: praveen-k8s-cluster
region: us-east-1
vpc:
subnets:
public:
us-east-1a: { id: subnet-32740f6e }
us-east-1b: { id: subnet-78146a1f }
us-east-1c: { id: subnet-16561338 }
nodeGroups:
- name: ng-1
ssh:
allow: true
minSize: 1
maxSize: 2
instancesDistribution:
instanceTypes: ["t3.small", "t3.medium"]
onDemandBaseCapacity: 0
onDemandPercentageAboveBaseCapacity: 0
spotInstancePools: 2
</pre>
<br />
<b># An example of ClusterConfig showing managed nodegroups with spot instances</b><br />
<pre class="brush:xml">---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
vpc:
subnets:
public:
us-east-1a: { id: subnet-32740f6e }
us-east-1b: { id: subnet-78146a1f }
us-east-1c: { id: subnet-16561338 }
metadata:
name: praveen-k8s-cluster
region: us-east-1
managedNodeGroups:
- name: managed-ng-1
instanceType: t3.small
minSize: 1
maxSize: 1
desiredCapacity: 1
volumeSize: 20
ssh:
allow: true
</pre>
<h3 style="text-align: left;">
Screen shots from the above sequence of steps</h3>
<br />
1. eksctl uses CloudFormation templates to create the EKS Cluster and the NodeGroup. The status of the Stack creation can be monitored from the CloudFomation Management Console.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-R7x5pL7Q9rY/Xd9qhIHME5I/AAAAAAAA-Vg/-i6JiHMl-1cy92leM-tZaRKJxGLnKRh3wCLcBGAsYHQ/s1600/003-cloudformation-stacks.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="347" data-original-width="1076" height="128" src="https://1.bp.blogspot.com/-R7x5pL7Q9rY/Xd9qhIHME5I/AAAAAAAA-Vg/-i6JiHMl-1cy92leM-tZaRKJxGLnKRh3wCLcBGAsYHQ/s400/003-cloudformation-stacks.png" width="400" /></a></div>
<br />
2. The above mentioned CloudFormation templates create a EKS Cluster and the NodeGroup as shown in the below EKS Management Console.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-vZ6G53z11dc/Xd9qlzD_YRI/AAAAAAAA-Vk/4Mqf54DmQEoBw0yJjO5E58wJRBlfHizhgCLcBGAsYHQ/s1600/001-eks-cluster.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="570" data-original-width="1010" height="225" src="https://1.bp.blogspot.com/-vZ6G53z11dc/Xd9qlzD_YRI/AAAAAAAA-Vk/4Mqf54DmQEoBw0yJjO5E58wJRBlfHizhgCLcBGAsYHQ/s400/001-eks-cluster.png" width="400" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-XXZQFBDTzhc/Xd9qpddC3SI/AAAAAAAA-Vo/Ar3EDqf06A8nyFx8ZGbIv8vzpXyPZzTYQCLcBGAsYHQ/s1600/002-eks-managed-node-group.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="178" data-original-width="1004" height="70" src="https://1.bp.blogspot.com/-XXZQFBDTzhc/Xd9qpddC3SI/AAAAAAAA-Vo/Ar3EDqf06A8nyFx8ZGbIv8vzpXyPZzTYQCLcBGAsYHQ/s400/002-eks-managed-node-group.png" width="400" /></a></div>
<br />
3. One of the EC2 Instance was created in Step 1. Other one was created by eksctl for the NodeGroup worker nodes.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-pg2pYV2r9vw/Xd9qth1sjDI/AAAAAAAA-Vs/1PT-ODFe_icoLamIm8GxF9CtizOjPBZXgCLcBGAsYHQ/s1600/004-ec2-instances.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="256" data-original-width="1354" height="75" src="https://1.bp.blogspot.com/-pg2pYV2r9vw/Xd9qth1sjDI/AAAAAAAA-Vs/1PT-ODFe_icoLamIm8GxF9CtizOjPBZXgCLcBGAsYHQ/s400/004-ec2-instances.png" width="400" /></a></div>
<br />
4. Interaction with the Cluster using kubctl to<br />
- get the nodes<br />
- create a deployment and get the list of pods<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-szfHGmDNqtM/Xd9qws9cQYI/AAAAAAAA-Vw/9fhBD23nXzMY0vRa3YTL3FPBq1TfHD7WQCLcBGAsYHQ/s1600/005-kubectl-creating-pods.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="393" data-original-width="1093" height="143" src="https://1.bp.blogspot.com/-szfHGmDNqtM/Xd9qws9cQYI/AAAAAAAA-Vw/9fhBD23nXzMY0vRa3YTL3FPBq1TfHD7WQCLcBGAsYHQ/s400/005-kubectl-creating-pods.png" width="400" /></a></div>
<br />
5. Finally, deletion of the EKS Cluster and the NodeGroup.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-qSiG9IoFhYg/Xd9qwhgZVGI/AAAAAAAA-V0/UuT7AmDjJ_gPkoLwx6v9l_BCwmuzhk71QCLcBGAsYHQ/s1600/006-eks-cluster-deletion.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="263" data-original-width="1107" height="95" src="https://1.bp.blogspot.com/-qSiG9IoFhYg/Xd9qwhgZVGI/AAAAAAAA-V0/UuT7AmDjJ_gPkoLwx6v9l_BCwmuzhk71QCLcBGAsYHQ/s400/006-eks-cluster-deletion.png" width="400" /> </a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h3 style="text-align: left;">
Conclusion</h3>
<br />
As shown above eksctl provides an easy way to create a K8S Cluster in the AWS easily. The same thing can be done with the AWS Management Console also (<a href="https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html" target="_blank">1</a>). Doing with the AWS Management Console is more of a manual way, which gives us clarity on the different resources getting created and how they interact with each other.</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com0tag:blogger.com,1999:blog-8716531089719420013.post-2522001459953084482019-11-25T15:21:00.000+05:302019-11-25T15:21:25.281+05:30Interacting with AWS S3 using Java on EC2<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: left;">
Many web applications are being built on top of the Cloud Infrastructure. Let's take the case of a photo sharing website like Instagram. The website can be deployed on EC2 and let it interact with S3 to store the pictures. Building a full fledged photo sharing website is beyond the scope of this blog, but we will explore how to execute a Java program on top of EC2 to interact with S3.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-kuVeDeE49w0/XdqX0axxZ2I/AAAAAAAA-O4/2APX-SpN5wUjpO6SLazeMWBNn3_Y0aikQCLcBGAsYHQ/s1600/photo-sharing-website.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="561" data-original-width="1206" height="185" src="https://1.bp.blogspot.com/-kuVeDeE49w0/XdqX0axxZ2I/AAAAAAAA-O4/2APX-SpN5wUjpO6SLazeMWBNn3_Y0aikQCLcBGAsYHQ/s400/photo-sharing-website.png" width="400" /></a></div>
<br />
Here are the sequence of steps with the assumption that the reader is familiar with the basics of AWS. Also, all the steps in this blog will fall under the free tier.<br />
<br />
<b>Step 1:</b> Create an Ubuntu
EC2 instance (t2.micro) and connect to it via Putty or any other means. For the EC2 SecurityGroup the port 22/SSH has to be opened in the inbound.<br />
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<b>Step 2:</b> Get the list of softwares and install maven, java-common by executing the below commands on Ubuntu EC2 instance.<br />
</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
sudo apt-get update</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
sudo apt install
maven java-common</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<b>Step 3:</b> By default EC2 has no permission to interact with S3 or in fact with any other service. Create a role with
AmazonS3FullAccess policy attached. This policy gives full permissions to all the folders and files in S3, which is not recommended. It's always best to give limited privileges by creating a custom policy and attaching it to the EC2.</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<b>Step 4:</b> Attach the policy to
the EC2. Now the EC2 has permissions to interact with S3.</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<b>Step 5:</b> Get the link to the latest Amazon Corretto Java from <a href="https://docs.aws.amazon.com/corretto/latest/corretto-11-ug/downloads-list.html" target="_blank">this link</a> and replace the link in the below wget command. Execute the wget commands in the Ubuntu EC2 to download Amazon Corretto Java. OpenJDK or Oracle JDK can also be used.</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
wget https://d3pxv6yz143wms.cloudfront.net/11.0.5.10.1/java-11-amazon-corretto-jdk_11.0.5.10-1_amd64.deb</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
Install Amazon Corretto Java on Ubuntu using the dpkg command as an administrator.</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
sudo dpkg --install
java-11-amazon-corretto-jdk_11.0.5.10-1_amd64.deb</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<b>Step 6:</b> Create basic maven
package using the below command. This will create a myapp folder with pom.xml, App.java and other artifacts.</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
mvn -B
archetype:generate \</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<span style="mso-spacerun: yes;">
</span>-DarchetypeGroupId=org.apache.maven.archetypes \</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<span style="mso-spacerun: yes;"> </span>-DgroupId=org.example.basicapp \</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<span style="mso-spacerun: yes;"> </span>-DartifactId=myapp</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<span style="mso-spacerun: yes;"> </span></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<b>Step 6:</b> In the myapp folder, remove the pom.xml and replace with pom.xml mentioned <a href="https://www.dropbox.com/s/8fsb9kedztlvw5p/pom.xml?dl=0" target="_blank">here</a>. Replace the
exec-maven-plugin (<a href="https://mvnrepository.com/artifact/org.codehaus.mojo/exec-maven-plugin" target="_blank">1</a>) and aws-java-sdk (<a href="https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk" target="_blank">1</a>) versions with the latest versions got from the maven repository in the pom.xml file.</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<b>Step 7:</b> Remove the App.java file created by maven.</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
cd
/home/ubuntu/myapp/src/main/java/org/example/basicapp</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
rm App.java</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
Create a file by the S3Sample.java with the java program mentioned <a href="https://www.dropbox.com/s/p160md3y3tlot67/S3Sample.java?dl=0" target="_blank">here</a> in the above folder.</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<b>Step 8:</b> Execute the below commands to compile the S3Sample.java program and execute it.</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
cd ~/myapp</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
mvn clean compile
exec:java</div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<b>Step 9:</b> Note that
interaction with the S3 happening in the output of the above command towards the end as shown below. A bucket is created in S3, a file uploaded/downloaded. The S3 cleanup is also done in the Java program, so when seen in the S3 Management Console there won't be any changes.<br />
<br />
Modify the Java program not to do the S3 cleanup and the changes done by the Java program can be seen in the S3 Management Console. </div>
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-kz1jxP62LeQ/XduDBWwzACI/AAAAAAAA-Tk/i-vOImWTEZwVpYg-nTDDCtx0wM-yuJY-ACLcBGAsYHQ/s1600/putty-s3-interaction.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="620" data-original-width="1026" height="241" src="https://1.bp.blogspot.com/-kz1jxP62LeQ/XduDBWwzACI/AAAAAAAA-Tk/i-vOImWTEZwVpYg-nTDDCtx0wM-yuJY-ACLcBGAsYHQ/s400/putty-s3-interaction.png" width="400" /></a></div>
</div>
Praveen Sripatihttp://www.blogger.com/profile/11782284194201977787noreply@blogger.com1