Thursday, 14 January 2021

Ansible Open Source

 

What Is Ansible?

Ansible is an open source IT Configuration Management, Deployment & Orchestration tool. It aims to provide large productivity gains to a wide variety of automation challenges. This tool is very simple to use yet powerful enough to automate complex multi-tier IT application environments.
All we do is to open a file and start adding tasks.

A task could be Installing NGINX webserver, for example.

In Ansible, we name a task and write down the command we want it to execute.

A task can be part of bigger thing like bringing up our e-commerce website.

Other tasks like applying updates, adding our custom config file can also be added.

The bigger thing or a group of tasks is grouped in what we call a Playbook.

A Playbook is just a file where we tell Ansible the tasks we want it to execute in an orderly fashion.

Ansible doesn't depend on additional daemons, client or servers. 

The mechanics of Ansible

Control node (that has Ansible installed) reads a Playbook file and executes the tasks listed in the playbook.

We also mention in the playbook the host or group of hosts where such tasks should be executed.

The inventory file is where we have a list of individual hosts.

We can group individual hosts into groups within the Inventory file.

In the example below, we execute ansible-playbook <playbook_name> command on Ansible control node (10.10.10.100).

It then reads a Playbook file that has 2 tasks.

Task1 is executed on DBServers group of hosts and Task2 on WebServers group:

Ansible Terms:

  • Controller Machine: The machine where Ansible is installed, responsible for running the provisioning on the servers you are managing.
  • Inventory: An initialization file that contains information about the servers you are managing.
  • Playbook: The entry point for Ansible provisioning, where the automation is defined through tasks using YAML format.
  • Task: A block that defines a single procedure to be executed, e.g. Install a package.
  • ModuleAnsible modules are discrete units of code which can be used from the command line or in a playbook task. 
  • Role: A pre-defined way for organizing playbooks and other files in order to facilitate sharing and reusing portions of a provisioning.
  • Play: A provisioning executed from start to finish is called a playIn simple words, execution of a playbook is called a play.
  • Facts: Global variables containing information about the system, like network interfaces or operating system.
  • Handlers: Used to trigger service status changes, like restarting or stopping a service.

ANSIBLE ARCHITECTURE      

 

       


Ansible installation on linux AWS

Step1:

Launch Two (Amazon Linux 2) Aws instances(one will be the controller, the other will be the Target host)




Step 2:

On The Target host machines Set password Authentication:

Switch to root user

sudo su -

Then edit the sshd_config file to enable password authentication

vi /etc/ssh/sshd_config

look for the below line and change the entry from no to yes

PasswordAuthentication yes

#PermitEmptyPasswords no

#PasswordAuthentication no


Next Create a password for ec2-user

passwd ec2-user

#then enter the password twice and press enter(you can use admin123)

Note: The password will not show on the screen as u type it. Just type and press enter when u are done

Next Edit the sudoers file to enable ec2-user have full previledges

vi /etc/sudoers

Insert the below line in the editor and save

ec2-user ALL=NOPASSWD: ALL

Save ---> :wq!

Next restart the ssh service with below command

systemctl restart sshd

Step 3:On Ansible Controller machine Install Ansible

Switch to root

sudo su -

Install Ansible

sudo yum update -y

sudo yum install ansible -y

ansible --version 


Next edit the hosts file which will contain inventory of all ur target hosts and add ur target host ip

vi /etc/ansible/hosts

Uncomment [webservers] delete the entries under it and Add ip of Target host under it


Save then switch to ec2-user

su - ec2-user

Generate a keypair

ssh-keygen -t ed25519

#Press enter four times to generate ssh key to connect the hosts machine



Next send the public key of the Ansible Controller to the target machine by executing this command

ssh-copy-id ec2-user@ipofansiblehost

eg ssh-copy-id ec2-user@192.168.25.1

You will be prompted for password. Enter ur password: admin123



Now try and connect to the target host

ssh ec2-user@ipofansiblehost

eg ssh ec2-user@192.168.25.1


Then exit

exit




#check for remote connection to your hosts machine with below command

ansible -m ping webservers


1. The Anatomy of the Command

  • ansible: This invokes the Ansible command-line tool for "ad-hoc" commands (one-off tasks that don't require a full playbook).

  • -m ping: This tells Ansible to use the ping module.

    • Note: This is not an ICMP ping (like the one you use in a terminal to check if an IP is alive). It is a Python-based check that logs into the server via SSH and verifies that Python is installed and usable.

  • webservers: This is the pattern or group name. Ansible looks into your inventory file (usually located at /etc/ansible/hosts or a local hosts.ini) and runs the command against every server listed under the [webservers] header


#Ansible Module: A module is a command or set of similar Ansible commands meant to be executed on the client-side

#

Understanding Ansible Modules

Modules perform tasks remotely.

Example:

Create user:

ansible webservers -m user -a "name=devops" --become

Install package:

ansible webservers -m yum -a "name=httpd state=present" --become

Start service:

ansible webservers -m service -a "name=httpd state=started" --become

Think of --become as the Ansible equivalent of typing sudo before a command in a Linux terminal.

When you add --become to your command:

  1. Ansible connects as your normal user (e.g., ec2-user).

  2. It then "becomes" another user (by default, root) to execute the specific task.

  3. Once the task is finished, it drops those privileges.

2. Example Comparison

If you want to install Apache on your webservers, a normal user doesn't have the "keys" to the system's package manager.

This will fail: ansible webservers -m yum -a "name=httpd state=present" (Error: You need to be root to perform this command.)

This will succeed: ansible webservers -m yum -a "name=httpd state=present" --become (Success: Ansible uses sudo to install the package.)


Let's use some playbook
sudo vi playbook.yml

Insert the below lines into the playbook
---
- name: Install Web Server
  hosts: webservers
  become: true

  tasks:

  - name: Install HTTPD
    yum:
      name: httpd
      state: present

  - name: Start HTTPD
    service:
      name: httpd
      state: started
      enabled: yes
Save with :wq!
#check for syntax errors with below command
ansible-playbook playbook.yml --syntax-check

#do a dry run with below command

ansible-playbook playbook.yml --check

#Run the playbook with the below command
ansible-playbook playbook.yml 
Now go to the target server and check if httpd is installed
systemctl status httpd

Lets try another playbook to install tomcat


sudo vi playbook02.yml

Paste the below lines into the editor and save
---
- hosts: webservers
  become: true

  tasks:

  - name: Install Apache
    yum:
      name: httpd
      state: present

  - name: Deploy index file
    copy:
      content: "Hello from Ansible Automation"
      dest: /var/www/html/index.html
    notify: restart apache

  handlers:

  - name: restart apache
    service:
      name: httpd
      state: restarted

#Now run the playbook
ansible-playbook playbook02.yml



open port 80


🧠 Important Ansible Concepts


✅ Inventory

Defines servers.

webservers
dbservers
k8snodes

✅ Playbook

Automation workflow written in YAML.


✅ Tasks

Individual automation steps.


✅ Modules

Examples:

ModulePurpose
yumInstall packages
serviceManage services
copyTransfer files
userCreate users
gitClone repos

✅ Handlers

Triggered only when changes occur.


✅ Become

Privilege escalation (sudo).

become: true

🏗️ REAL DEVOPS USE CASES

Students can automate:

✅ Jenkins installation
✅ Docker setup
✅ Kubernetes nodes
✅ Tomcat deployment
✅ NGINX configuration
✅ Application rollout


SECURITY BEST PRACTICES (IMPORTANT)

✅ Use SSH keys only
✅ Restrict Security Groups
✅ Avoid password authentication
✅ Avoid root login
✅ Use private subnets for automation






Friday, 25 December 2020

Deploy Jenkins Server with Terraform

TERRAFORM LAB 3

Before proceeding with this lab, reference the following link below:

In this lab, we will be deploying a Jenkins standalone server with terraform. 

Prerequisites:

AWS access and secret keys are required to provision resources on AWS cloud.

  • Open Visual Code Studio then click on File Preferences > Extensions then search and install Terraform extension


























  • Login to AWS console, click on Username on top right corner and go to My Security Credentials



  • Click on Access Keys and Create New Key













Step I: Open File Explorer, navigate to Desktop and create a folder jenkins_workspace.









Step II: Once folder has been created, open Visual Code Studio and add folder to workspace












Step III: Create a new folder files in workspace and follow the below steps:

  • In files folder, create a file environment and copy the below code and save it
         export JAVA_HOME=/home/ubuntu/jdk1.8.0_251/bin
  • In files folder, download JAVA dependencies on your machine and copy it in the directory.

Step IV: Create a new file main.tf and copy the below code in yellow color


















provider "aws" {
    region = var.region
    access_key = var.accesskey
    secret_key  = var.secretkey
}

resource "aws_instance" "ec2" {

## The provisioner file allows to copy files from your local machine to remote server via ssh and winrm

    provisioner "file" {
    source      = "./files/environment"
    destination = "/home/ubuntu/environment"
  }


  provisioner "file" {
    source      = "./files/jdk-8u251-linux-x64.tar.gz"
    destination = "/home/ubuntu/jdk-8u251-linux-x64.tar.gz"
  }

    ################################################################################################
    #### Input code here to configure your jenkins server (yum install, sudo this, sudo that, pip those)
    ################################################################################################
    provisioner "remote-exec" {
        inline = [
            "sudo pip install awscli",
            "echo This is installing 1",
            "sudo apt-get install -y unzip",
            "echo This is installing 2",
            "sudo apt-get install wget",
            "echo This is installing 3",
            "sudo yum install java -y",
            "sudo tar xvf /home/ubuntu/jdk-8u251-linux-x64.tar.gz",
            "java -version",
            "echo Completed installing java",
            "sudo mv -f /home/ubuntu/environment /etc/",
            "source /etc/environment",
            "echo Set JAVA HOME",
            "wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -",
            "echo This is installing 8",
            "sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'",
            "sudo apt-get update",
            "sudo apt-get install jenkins -y",
            "sudo sed -i 's|/bin:/usr/bin:/sbin:/usr/sbin|/bin:/usr/bin:/sbin:/usr/sbin:/home/ubuntu/jdk1.8.0_251/bin|g' /etc/init.d/jenkins",
            "sudo systemctl daemon-reload",
            "sudo systemctl start jenkins",
            "echo Installation Complete",
            "cd /home/ubuntu/",
            "wget https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip",
            "unzip terraform_0.12.24_linux_amd64.zip",
            "sudo mv terraform /usr/bin/",
            "sudo pwd"
        ]
    }

    ami = "ami-0782e9ee97725263d"
    root_block_device {
    volume_type           = "gp2"
    volume_size           = 200
    encrypted             = true
  }

    tags = {
        Name = var.stackname
        CreatedBy = var.launched_by
        Application = var.application
        OS = var.os
    }

    instance_type = "t2.micro"
    key_name = "terraform"
    vpc_security_group_ids = [aws_security_group.ec2_sg.id]

    #This connection string is to establish a connection via ssh to configure the instance
    
connection {
        user = "ubuntu"
        type = "ssh"
        host = self.public_ip
        private_key = file("
KEYNAME.pem")
        timeout = "2m"
    }
}


Add the block below in main.tf to output the Private IP, Public IP and EC2 Name after creation. (Note: This is not required)

output "ec2_ip" {

    value = [aws_instance.ec2.*.private_ip]

}


output "ec2_ip_public" {

    value = [aws_instance.ec2.*.public_ip]

}


output "ec2_name" {

    value = [aws_instance.ec2.*.tags.Name]

}



Step V: Create a new file security.tf and copy the below code in yellow color

resource "aws_security_group" "ec2_sg" {
name = "jenkins-dev-sg"
description = "EC2 SG"

ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["10.0.0.0/8"]
}

   ingress {
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["10.0.0.0/8"]
}

#Allow all outbound
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
    Name = "jenkins-dev-sg"
  }
}


Step VI: Create a new file variable.tf and copy the below code in yellow color. 


variable region {

  type        = string

  default = "us-east-2"

}


############## tags

variable accesskey {

  type        = string

  default = "ENTER ACCESS KEY HERE"

}


variable secretkey {

  type        = string

  default = "ENTER SECRET KEY HERE"

}


variable stackname {

  type        = string

  default = "u2-dev-jenas"

}


variable application {

  type        = string

  default = "Jenkins"

}


variable os {

  type        = string

  default = "Ubuntu"

}


variable launched_by {

  type        = string

  default = "ENTER YOUR NAME HERE"

}



Step X: Open Terminal in VSCode
















Step XI: Execute command below

terraform init
the above command will download the necessary plugins for AWS.

terraform plan
the above command will show how many resources will be added.
Plan: 2 to add, 0 to change, 0 to destroy.

Execute the below command
terraform apply
Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Yay! 
We have successfully deployed your first jenkins server with terraform............................

Now login to AWS console, to verify jenkins is up and running

Thursday, 17 December 2020

How to Create a Bitbucket Pull Request


Using a pull request, you're requesting that another developer or the project owner pull a branch from your repository and add it to the project. Here's how to create a pull request from the Bitbucket website.

How to Create a Bitbucket Pull Request

To create a pull request, you'll have to make sure that your branch is updated with new code that you want to be reviewed. Before going any further, check and save your work on the local machine and update the branch.

Then, open the Bitbucket website on your browser and navigate to the repository in question.




Click on the menu button in the top-left, and select Pull Requests.



Here, click on the Create pull request button.


The pull request creation form will open.


There are a few things to do on this form:

  • From the left dropdown menu, select the sub-branch (source repository) that you want to merge into the master branch.

  • On the right, make sure that the project and the master branch (destination repository) are selected.

  • You can create a separate title for the pull request and write a short description of the code that you've written or the task that you've completed.

  • At the bottom, you'll indicate the Reviewers. This is the most important step: Type in the name of one or more developers who need to review your code.

  • If you automatically want to close the branch once the pull request is merged, click the Close branch checkbox.

Before submitting the request, you can scroll below the form to take a look at the code differences between the test branch and the master branch and go over the recent commits.

Once you're confident about it, click Create pull request.




The process from your side is now complete. The reviewers will get an email notification about your pull request. They'll be able to view all the contents of your branch, compare it with the master branch, and leave comments.

Once they're satisfied with your work, they can click Merge to merge your branch's code with the master branch.







Bash Script To Install Ansible Automation Platform ( AWX)

#!/bin/bash # --- Configuration --- AWX_OPERATOR_VERSION="2.19.1" NAMESPACE="awx" KUBECONFIG_PATH="/etc/rancher/k3s...