Nagios – Continuous Monitoring

 

What is Nagios?

Nagios is used for Continuous monitoring of systems, applications, services, and business processes etc in a DevOps culture. In the event of a failure, Nagios can alert technical staff of the problem, allowing them to begin remediation processes before outages affect business processes, end-users, or customers. With Nagios, you don’t have to explain why an unseen infrastructure outage affect your organization’s bottom line.

Nagios runs on a server, usually as a daemon or a service.

It periodically runs plugins residing on the same server, they contact hosts or servers on your network or on the internet. One can view the status information using the web interface. You can also receive email or SMS notifications if something happens.
The Nagios daemon behaves like a scheduler that runs certain scripts at certain moments. It stores the results of those scripts and will run other scripts if these results change.

Plugins: These are compiled executables or scripts (Perl scripts, shell scripts, etc.) that can be run from a command line to check the status or a host or service. Nagios uses the results from the plugins to determine the current status of the hosts and services on your network.

Let’s now discuss it’s architecture.

Nagios Architecture:

• Nagios is built on a server/agents architecture.
• Usually, on a network, a Nagios server is running on a host, and Plugins interact with local and all the remote hosts that need to be monitored.
• These plugins will send information to the Scheduler, which displays that in a GUI

Features of Nagios

Following are the important features of Nagios monitoring tool:

• Relatively scalable, Manageable, and Secure
• Good log and database system
• Informative and attractive web interfaces
• Automatically send alerts if condition changes
• If the services are running fine, then there is no need to do check that host is an alive
• Helps you to detect network errors or server crashes
• You can troubleshoot the performance issues of the server.
• The issues, if any, can be fixed automatically as they are identified during the monitoring process
• You can monitor the entire business process and IT infrastructure with a single pass
• The product's architecture is easy writing new plugins in the language of your choice
• Nagios allows you to read its configuration from an entire directory which helps you to decide how to define individual files
• Utilizes topology to determine dependencies
• Monitor network services like HTTP, SMTP, HTTP, SNMP, FTP, SSH, POP, etc.
• Helps you to define network host hierarchy using parent hosts
• Ability to define event handlers which runs during service or host events for proactive problem resolution
• Support for implementing redundant monitoring hosts

Installation of Nagios on EC2

Spin two Servers ( Amazon Linux 2 AMI) ready for this demo 

Do the following command on Nagios Master 

yum install httpd php gcc glibc glibc-comman gd gd-devel make net-snmp unzip -y

wget https://sourceforge.net/projects/nagios/files/nagios-4.x/nagios-4.3.2/nagios-4.3.2.tar.gz

wget https://nagios-plugins.org/download/nagios-plugins-2.2.1.tar.gz

tar -zxvf nagios-4.3.2.tar.gz

tar -zxpvf nagios-plugins-2.2.1.tar.gz

useradd nagios

passwd nagios

groupadd nagcmd

usermod -G nagcmd nagios

usermod -G nagcmd apache

cd nagios-4.3.2

 ./configure --with-command-group=nagcmd

 make all

 make install

 make install-init

 make install-commandmode

 make install-config

 make install-webconf

 htpasswd -s -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

 systemctl restart httpd

 cd ..

 cd nagios-plugins-2.2.1/

 ./configure --with-nagios-user=nagios --with-nagios-group=nagios

 make

 make install

 /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

 systemctl enable  nagios

 systemctl start  nagios

 systemctl restart httpd

systemctl restart  nagios

open the browser http://ip address/nagios

uid:nagiosadmin(set in the previous steps)

pwd: nagios

cd /usr/local/nagios/ ls cd etc/

vi nagios.cfg

# then uncomment this line /usr/local/nagios/etc/servers

mkdir servers cd servers/ ls hostname vi ip-172-31-21-155.ec2.internal.cfg

cat /usr/local/nagios/etc/objects/localhost.cfg

#copy the above file into ip-172-31-21-155.ec2.internal.cfg and replace servers address with master private ip address and localhost with private dns of the master server

systemctl restart nagios

On Nagios Host (linux):

rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

yum install nrpe nagios-plugins-all -y

vim /etc/nagios/nrpe.cfg

# enter edit model by pressing i and replace the server ip with with the node private ip and Host ip with Nagios Master with the private ip

and master ip with private ip

systemctl enable nrpe

systemctl start nrpe

let do some exercise by installing LAMP on the Host 

sudo yum update -y

sudo amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2

cat /etc/system-release

sudo yum install -y httpd mariadb-server

yum info package_name

sudo systemctl start httpd

sudo systemctl enable httpd

sudo systemctl is-enabled httpd

#log into the master server with the following

cd /

cd var 

cd log

 cat yum.log

Ansible Open Source

 

What Is Ansible?

Ansible is an open source IT Configuration Management, Deployment & Orchestration tool. It aims to provide large productivity gains to a wide variety of automation challenges. This tool is very simple to use yet powerful enough to automate complex multi-tier IT application environments.

All we do is to open a file and start adding tasks.

A task could be Installing NGINX webserver, for example.

In Ansible, we name a task and write down the command we want it to execute.

A task can be part of bigger thing like bringing up our e-commerce website.

Other tasks like applying updates, adding our custom config file can also be added.

The bigger thing or a group of tasks is grouped in what we call a Playbook.

A Playbook is just a file where we tell Ansible the tasks we want it to execute in an orderly fashion.

Ansible doesn't depend on additional daemons, client or servers. 

The mechanics of Ansible

A Control node (that has Ansible installed) reads a Playbook file and executes the tasks listed in the playbook.

We also mention in the playbook the host or group of hosts where such tasks should be executed.

The inventory file is where we have a list of individual hosts.

We can group individual hosts into groups within the Inventory file.

In the example below, we execute ansible-playbook <playbook_name> command on Ansible control node (10.10.10.100).

It then reads a Playbook file that has 2 tasks.

Task1 is executed on DBServers group of hosts and Task2 on WebServers group:

Ansible Terms:

• Controller Machine: The machine where Ansible is installed, responsible for running the provisioning on the servers you are managing.
• Inventory: An initialization file that contains information about the servers you are managing.
• Playbook: The entry point for Ansible provisioning, where the automation is defined through tasks using YAML format.
• Task: A block that defines a single procedure to be executed, e.g. Install a package.
• Module: Ansible modules are discrete units of code which can be used from the command line or in a playbook task. 
• Role: A pre-defined way for organizing playbooks and other files in order to facilitate sharing and reusing portions of a provisioning.
• Play: A provisioning executed from start to finish is called a play. In simple words, execution of a playbook is called a play.
• Facts: Global variables containing information about the system, like network interfaces or operating system.
• Handlers: Used to trigger service status changes, like restarting or stopping a service.

ANSIBLE ARCHITECTURE      

 

       

Ansible installation on linux AWS

Step1:

Launch Two (Amazon Linux 2) Aws instances(one will be the controller, the other will be the Target host)

Step 2:

On The Target host machines Set password Authentication:

Switch to root user

sudo su -

Then edit the sshd_config file to enable password authentication

vi /etc/ssh/sshd_config

look for the below line and change the entry from no to yes

PasswordAuthentication yes

#PermitEmptyPasswords no

#PasswordAuthentication no

Next Create a password for ec2-user

passwd ec2-user

#then enter the password twice and press enter(you can use admin123)

Note: The password will not show on the screen as u type it. Just type and press enter when u are done

Next Edit the sudoers file to enable ec2-user have full previledges

vi /etc/sudoers

Insert the below line in the editor and save

ec2-user ALL=NOPASSWD: ALL

Save ---> :wq!

Next restart the ssh service with below command

systemctl restart sshd

Step 3:On Ansible Controller machine Install Ansible

Switch to root

sudo su -

Install Ansible

sudo yum update -y

amazon-linux-extras install ansible2 -y

ansible --version 

Next edit the hosts file which will contain inventory of all ur target hosts and add ur target host ip

vi /etc/ansible/hosts

Uncomment [webservers] delete the entries under it and Add ip of Target host under it

Save then switch to ec2-user

su - ec2-user

Generate a keypair

ssh-keygen

#Press enter four times to generate ssh key to connect the hosts machine

Next send the public key of the Ansible Controller to the target machine by executing this command

ssh-copy-id -i ec2-user@ipofansiblehost

eg ssh-copy-id -i ec2-user@192.168.25.1

You will be prompted for password. Enter ur password: admin123

Now try and connect to the target host

ssh ec2-user@ipofansiblehost

eg ssh ec2-user@192.168.25.1

Then exit

exit

#check for remote connection to your hosts machine with below command

ansible -m ping webservers

#Ansible Module: A module is a command or set of similar Ansible commands meant to be executed on the client-side

#example of module command

ansible -m user -a "name=paul password=paul" webservers --become

#yum module

ansible -m yum -a "name=httpd state=present" webservers --become

#check with httpd is started with below command in the hosts machine

systemctl status httpd

#use service module to start the hosts machine
ansible -m service -a "name=httpd state=started" webservers --become

Let's use some playbook

sudo vi playbook.yml

Insert the below lines into the playbook

---

- hosts: webservers

  become: true

  become_user: root

  tasks:

  - name: Install httpd

    yum: name=httpd state=present

  - name: start httpd

    service: name=httpd state=started

Save with :wq!

#check for syntax errors with below command

ansible-playbook playbook.yml --syntax-check

#do a dry run with below command

ansible-playbook playbook.yml --check

#Run the playbook with the below command

ansible-playbook playbook.yml 

Now go to the target server and check if httpd is installed

systemctl status httpd

Lets try another playbook to install tomcat

sudo vi playbook02.yml

Paste the below lines into the editor and save

---

- hosts: webservers

  become: true

  become_user: root

  tasks:

  - name: Install tomcat

    yum: name=tomcat state=present

  - name: start tomcat

    service: name=tomcat state=started

  - name: Deploy war file

    get_url: url=https://tomcat.apache.org/tomcat-7.0-doc/appdev/sample/sample.war

             dest=/usr/share/tomcat/websapps

    notify: restart tomcat

  handlers:

  - name: restart tomcat

    service: name=tomcat state=restarted

#Now run the playbook

ansible-playbook playbook02.yml

Now open port 8080 and go to the ip address of the target host:8080 you will see tomcat running

How Devops Came to Be

 

What is DevOps? | DevOps Introduction | DevOps Tools | DevOps Tutorial For Beginners | Simplilearn from Simplilearn

Deploy Jenkins Server with Terraform

TERRAFORM LAB 3

Before proceeding with this lab, reference the following link below:

• INSTALLING TERRAFORM ON YOUR LOCAL MACHINE 

In this lab, we will be deploying a Jenkins standalone server with terraform. 

Prerequisites:

AWS access and secret keys are required to provision resources on AWS cloud.

• Open Visual Code Studio then click on File > Preferences > Extensions then search and install Terraform extension

• 
  

• Login to AWS console, click on Username on top right corner and go to My Security Credentials

• Click on Access Keys and Create New Key

Step I: Open File Explorer, navigate to Desktop and create a folder jenkins_workspace.

Step II: Once folder has been created, open Visual Code Studio and add folder to workspace

Step III: Create a new folder files in workspace and follow the below steps:

• In files folder, create a file environment and copy the below code and save it

         export JAVA_HOME=/home/ubuntu/jdk1.8.0_251/bin

• In files folder, download JAVA dependencies on your machine and copy it in the directory.

Step IV: Create a new file main.tf and copy the below code in yellow color

provider "aws" {
    region = var.region
    access_key = var.accesskey
    secret_key  = var.secretkey
}

resource "aws_instance" "ec2" {

## The provisioner file allows to copy files from your local machine to remote server via ssh and winrm

    provisioner "file" {
    source      = "./files/environment"
    destination = "/home/ubuntu/environment"
  }

  provisioner "file" {
    source      = "./files/jdk-8u251-linux-x64.tar.gz"
    destination = "/home/ubuntu/jdk-8u251-linux-x64.tar.gz"
  }

    ################################################################################################
    #### Input code here to configure your jenkins server (yum install, sudo this, sudo that, pip those)
    ################################################################################################
    provisioner "remote-exec" {
        inline = [
            "sudo pip install awscli",
            "echo This is installing 1",
            "sudo apt-get install -y unzip",
            "echo This is installing 2",
            "sudo apt-get install wget",
            "echo This is installing 3",
            "sudo yum install java -y",
            "sudo tar xvf /home/ubuntu/jdk-8u251-linux-x64.tar.gz",
            "java -version",
            "echo Completed installing java",
            "sudo mv -f /home/ubuntu/environment /etc/",
            "source /etc/environment",
            "echo Set JAVA HOME",
            "wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -",
            "echo This is installing 8",
            "sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'",
            "sudo apt-get update",
            "sudo apt-get install jenkins -y",
            "sudo sed -i 's|/bin:/usr/bin:/sbin:/usr/sbin|/bin:/usr/bin:/sbin:/usr/sbin:/home/ubuntu/jdk1.8.0_251/bin|g' /etc/init.d/jenkins",
            "sudo systemctl daemon-reload",
            "sudo systemctl start jenkins",
            "echo Installation Complete",
            "cd /home/ubuntu/",
            "wget https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip",
            "unzip terraform_0.12.24_linux_amd64.zip",
            "sudo mv terraform /usr/bin/",
            "sudo pwd"
        ]
    }

    ami = "ami-0782e9ee97725263d"
    root_block_device {
    volume_type           = "gp2"
    volume_size           = 200
    encrypted             = true
  }

    tags = {
        Name = var.stackname
        CreatedBy = var.launched_by
        Application = var.application
        OS = var.os
    }

    instance_type = "t2.micro"
    key_name = "terraform"
    vpc_security_group_ids = [aws_security_group.ec2_sg.id]

    #This connection string is to establish a connection via ssh to configure the instance
    

connection {
        user = "ubuntu"
        type = "ssh"
        host = self.public_ip
        private_key = file("KEYNAME.pem")
        timeout = "2m"
    }
}

Add the block below in main.tf to output the Private IP, Public IP and EC2 Name after creation. (Note: This is not required)

output "ec2_ip" {

    value = [aws_instance.ec2.*.private_ip]

}

output "ec2_ip_public" {

    value = [aws_instance.ec2.*.public_ip]

}

output "ec2_name" {

    value = [aws_instance.ec2.*.tags.Name]

}

Step V: Create a new file security.tf and copy the below code in yellow color

resource "aws_security_group" "ec2_sg" {

name = "jenkins-dev-sg"

description = "EC2 SG"

ingress {

from_port = 22

to_port = 22

protocol = "tcp"

cidr_blocks = ["10.0.0.0/8"]

}

   ingress {

from_port = 8080

to_port = 8080

protocol = "tcp"

cidr_blocks = ["10.0.0.0/8"]

}

#Allow all outbound

egress {

from_port = 0

to_port = 0

protocol = "-1"

cidr_blocks = ["0.0.0.0/0"]

}

tags = {
    Name = "jenkins-dev-sg"
  }

}

Step VI: Create a new file variable.tf and copy the below code in yellow color. 

variable region {

  type        = string

  default = "us-east-2"

}

############## tags

variable accesskey {

  type        = string

  default = "ENTER ACCESS KEY HERE"

}

variable secretkey {

  type        = string

  default = "ENTER SECRET KEY HERE"

}

variable stackname {

  type        = string

  default = "u2-dev-jenas"

}

variable application {

  type        = string

  default = "Jenkins"

}

variable os {

  type        = string

  default = "Ubuntu"

}

variable launched_by {

  type        = string

  default = "ENTER YOUR NAME HERE"

}

Step X: Open Terminal in VSCode

Step XI: Execute command below

terraform init

the above command will download the necessary plugins for AWS.

terraform plan

the above command will show how many resources will be added.

Plan: 2 to add, 0 to change, 0 to destroy.

Execute the below command

terraform apply

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Yay! 

We have successfully deployed your first jenkins server with terraform............................

Now login to AWS console, to verify jenkins is up and running