Overview
The goal is to implement Devops best practices to run Terraform in Jenkins Pipelines to deploy a tomcat. We will go over the main concepts that need to be considered and a Jenkinsfile that runs Terraform. The Jenkinsfile will consists of parameters that allows us to pass data as variables in our pipeline job.
- Install Terraform on Jenkins Server
- Install Terraform Plugin on Jenkins
- Configure Terraform
- Store and Encrypt Credentials in Jenkins
- Setting up CD Pipeline with Terraform
- Bash Script to Deploy Tomcat
- Run Pipeline Job
Install Terraform on Jenkins Server
Use the following commands to install Terraform on Jenkins server and move the binaries to the correct path as shown below.
- wget https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip
- unzip terraform_0.12.24_linux_amd64.zip
- sudo mv terraform /usr/bin/
Install Terraform plugin on Jenkins
Go to Manage Jenkins > Manage Plugins >Available > search Terraform as shown below:
As you can see, Terraform Plugin is already installed on my Jenkins hence why it's displayed in the Installed section.
Store and Encrypt Credentials in Jenkins (Access and Secret Key)
In this step, we will be storing and encrypting the access and secret key in Jenkins to maximize security and minimize the chances of exposing our credentials.
- Go to Manage Jenkins > Manage Credentials > Click on Jenkins the highlighted link as shown below
- Select Add Credentials
- Choose Secret text in the Kind field
- Enter the following below:
Note: Modify the yellow highlighted text with the right value.
- Secret = EnterYourSecretKeyHere
- ID = AWS_SECRET_ACCESS_KEY
- Description = AWS_SECRET_ACCESS_KEY
Click OK
Add another credential and enter the following:
- Secret = EnterYourAccessIDHere
- ID = AWS_ACCESS_KEY_ID
- Description = AWS_ACCESS_KEY_ID
Click OK
Configure Terraform
Go to Manage Jenkins > Global Tool Configuration > It will display Terraform on the list.
- Enter terraform in the Name field
- Provide the path /usr/bin/ as shown below
Setting up CD Pipeline for Terraform
- Go to Jenkins > New Items. Enter tomcat-pipeline in name field > Choose Pipeline > Click OK
- Select Configure after creation.
- Go to Build Triggers and enable Trigger builds remotely.
- Enter tf_token as Authentication Token
Bitbucket Changes
- Create a new Bitbucket Repo and call it tomcat-pipeline
- Go to Repository Settings after creation and select Webhooks
- Click Add Webhooks
- Enter tf_token as the Title
- Copy and paste the url as shown below
http://JENKINS_URL:8080/job/terraform-pipeline//buildWithParameters?token=tf_token
- Status should be active
- Click on skip certificate verification
- triggers --> repository push
- Go back to Jenkins, select Pipeline Script From SCM
- Enter credentials for Bitbucket, Leave the Branch master as the default, Make sure script path is Jenkinsfile
- Right click on Pipeline Syntax and open in a new tab.
- Choose Checkout from Version Control in the Sample Step field
- Enter Bitbucket Repository URL and Credentials, leave the branches blank
- Click GENERATE PIPELINE SCRIPT, copy credentialsId and url (This is required for Jenkinsfile script)
Open VSCode
Open Visual Code Studio and click on
Terminal
- Navigate to tomcat-pipeline repo in Bitbucket
- Run the command before cloning repo: git init
- Clone the repo with SSH or HTTPS
Create a new file main.tf and copy the below code in yellow color
provider "aws" {
region = var.region
version = "~> 2.0"
}
resource "aws_instance" "ec2" {
user_data = base64encode(templatefile("deploy.sh", {JAVA_HOME = var.java_home}))
ami = "ami-0782e9ee97725263d" ##Change AMI to meet OS requirement as needed.
root_block_device {
volume_type = "gp2"
volume_size = 200
delete_on_termination = true
encrypted = true
}
tags = {
Name = "u2-${var.environment}-${var.application}"
CreatedBy = var.launched_by
Application = var.application
OS = var.os
Environment = var.environment
}
instance_type = var.instance_type
key_name = "Enter_KEYPAIR_Name_Here"
vpc_security_group_ids = [aws_security_group.ec2_SecurityGroups.id]
}
output "ec2_ip" {
value = [aws_instance.ec2.*.private_ip]
}
output "ec2_ip_public" {
value = [aws_instance.ec2.*.public_ip]
}
output "ec2_name" {
value = [aws_instance.ec2.*.tags.Name]
}
output "ec2_instance_id" {
value = aws_instance.ec2.*.id
}
region = var.region
version = "~> 2.0"
}
resource "aws_instance" "ec2" {
user_data = base64encode(templatefile("deploy.sh", {JAVA_HOME = var.java_home}))
ami = "ami-0782e9ee97725263d" ##Change AMI to meet OS requirement as needed.
root_block_device {
volume_type = "gp2"
volume_size = 200
delete_on_termination = true
encrypted = true
}
tags = {
Name = "u2-${var.environment}-${var.application}"
CreatedBy = var.launched_by
Application = var.application
OS = var.os
Environment = var.environment
}
instance_type = var.instance_type
key_name = "Enter_KEYPAIR_Name_Here"
vpc_security_group_ids = [aws_security_group.ec2_SecurityGroups.id]
}
output "ec2_ip" {
value = [aws_instance.ec2.*.private_ip]
}
output "ec2_ip_public" {
value = [aws_instance.ec2.*.public_ip]
}
output "ec2_name" {
value = [aws_instance.ec2.*.tags.Name]
}
output "ec2_instance_id" {
value = aws_instance.ec2.*.id
}
- Create a new file security.tf and copy the below code in yellow color.
(Note: Change the tomcat port highlighted in red as needed if default is not 3000)
resource "aws_security_group" "ec2_SecurityGroups" {
name = "u2-${var.environment}-sg-${var.application}"
description = "EC2 SG"
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
name = "u2-${var.environment}-sg-${var.application}"
description = "EC2 SG"
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
## TOMCAT PORT
ingress {
ingress {
from_port = 4000
to_port = 4000
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
#Allow all outbound
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
to_port = 4000
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
#Allow all outbound
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
- Create a new file variable.tf and copy the below code in yellow color.
variable region {
type = string
default = "us-east-2"
}
variable "instance_type" {}
variable "application" {}
variable "environment" {}
type = string
default = "us-east-2"
}
variable "instance_type" {}
variable "application" {}
variable "environment" {}
variable "java_home" {
type = string
default = "/opt/java/jdk1.8.0_251"
}
variable os {
type = string
default = "Ubuntu"
}
variable launched_by {
type = string
default = "USER"
}
############## end tags
default = "USER"
}
############## end tags
Bash Script to Deploy Tomcat and Java application
- Create a new file deploy.sh and copy the below code in yellow color.
#!/bin/bash
set -x
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
echo ""
echo "........................................"
echo "Installation of application"
echo "........................................"
echo "Today's date: `date`"
echo "........................................"
echo ""
sudo apt-get install -y unzip
sudo apt update
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws --version
sudo apt-get install wget
echo 'Installing Java...'
wget https://deploy-app-servers.s3.us-east-2.amazonaws.com/java/jdk-8u251-linux-x64.tar.gz -P /opt/java/
echo untar JAVA
sudo tar -xf /opt/java/jdk-8u251-linux-x64.tar.gz -C /opt/java/
sudo touch environment
echo "export JAVA_HOME=/opt/java/jdk1.8.0_251" >> environment
sudo mv -f environment /etc/
source /etc/environment
echo Set JAVA HOME
echo Completed installing java
echo Set JAVA HOME
function check_java_home {
if [ -z ${JAVA_HOME} ]
then
echo 'Could not find JAVA_HOME. Please install Java and set JAVA_HOME'
exit
else
echo 'JAVA_HOME found: '$JAVA_HOME
if [ ! -e ${JAVA_HOME} ]
then
echo 'Invalid JAVA_HOME. Make sure your JAVA_HOME path exists'
exit
fi
fi
}
echo 'Installing tomcat server...'
echo 'Checking for JAVA_HOME...'
check_java_home
echo 'Downloading tomcat-9.0...'
if [ ! -f /etc/apache-tomcat-9*tar.gz ]
then
curl -O https://downloads.apache.org/tomcat/tomcat-9/v9.0.54/bin/apache-tomcat-9.0.54.tar.gz
fi
echo 'Finished downloading...'
echo 'Creating install directories...'
sudo mkdir -p '/opt/tomcat/9_0'
if [ -d "/opt/tomcat/9_0" ]
then
echo 'Extracting binaries to install directory...'
sudo tar -xzf apache-tomcat-9*.tar.gz -C "/opt/tomcat/9_0" --strip-components=1
echo 'Creating tomcat user group...'
sudo groupadd tomcat
sudo useradd -s /bin/false -g tomcat -d /opt/tomcat tomcat
echo 'Setting file permissions...'
cd "/opt/tomcat/9_0"
sudo chgrp -R tomcat "/opt/tomcat/9_0"
sudo chmod -R g+r conf
sudo chmod -R g+x conf
# This should be commented out on a production server
sudo chmod -R g+w conf
sudo chown -R tomcat webapps/ work/ temp/ logs/
echo 'Setting up tomcat service...'
sudo touch tomcat.service
sudo chmod 777 tomcat.service
echo "[Unit]" > tomcat.service
echo "Description=Apache Tomcat Web Application Container" >> tomcat.service
echo "After=network.target" >> tomcat.service
echo "[Service]" >> tomcat.service
echo "Type=forking" >> tomcat.service
echo "Environment=JAVA_HOME=$JAVA_HOME" >> tomcat.service
echo "Environment=CATALINA_PID=/opt/tomcat/9_0/temp/tomcat.pid" >> tomcat.service
echo "Environment=CATALINA_HOME=/opt/tomcat/9_0" >> tomcat.service
echo "Environment=CATALINA_BASE=/opt/tomcat/9_0" >> tomcat.service
echo "Environment='CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC'" >> tomcat.service
echo "Environment=JAVA_OPTS=-Djava.awt.headless=true-Djava.security.egd=file:/dev/./urandom" >> tomcat.service
echo "ExecStart=/opt/tomcat/9_0/bin/startup.sh" >> tomcat.service
echo "ExecStop=/opt/tomcat/9_0/bin/shutdown.sh" >> tomcat.service
echo "User=tomcat" >> tomcat.service
echo "Group=tomcat" >> tomcat.service
echo "UMask=0007" >> tomcat.service
echo "RestartSec=10" >> tomcat.service
echo "Restart=always" >> tomcat.service
echo "[Install]" >> tomcat.service
echo "WantedBy=multi-user.target" >> tomcat.service
sudo mv tomcat.service /etc/systemd/system/tomcat.service
sudo chmod 755 /etc/systemd/system/tomcat.service
sudo systemctl daemon-reload
sleep 3
sudo sed -i 's/8080/4000/g' /opt/tomcat/9_0/conf/server.xml
echo 'Starting tomcat server....'
sudo systemctl start tomcat
exit
else
echo 'Could not locate installation directory..exciting..'
exit
fi
sudo systemctl start tomcat
echo ""
echo "........................................"
echo "Installation of application"
echo "........................................"
echo "Today's date: `date`"
echo "........................................"
echo ""
- Create a new file Jenkinsfile and copy the below code in yellow color.
pipeline {
agent {
node {
label "master"
}
}
parameters {
string(name: 'AppName', defaultValue: 'Enter App Name', description: 'Name of application', )
choice(choices: ['master', 'dev', 'qa', 'uat', 'prod'], description: 'Select lifecycle to Deploy', name: 'Branch')
choice(choices: ['t2.micro', 't2.small', 't2.large', 't2.xlarge'], description: 'Select Instance Size', name: 'InstanceSize')
booleanParam(name: 'autoApprove', defaultValue: false, description: 'Automatically run apply after generating plan?')
}
environment {
AWS_ACCESS_KEY_ID = credentials('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = credentials('AWS_SECRET_ACCESS_KEY')
TF_VAR_instance_type = "${params.InstanceSize}"
TF_VAR_environment = "${params.Branch}"
TF_VAR_application = "${params.AppName}"
}
//
stages {
stage('checkout') {
steps {
echo "Pulling changes from the branch ${params.Branch}"
git credentialsId: 'paste-credentialsId-here', url: 'paste-url-here' , branch: "${params.Branch}"
}
}
stage('terraform plan') {
steps {
sh "pwd ; terraform init -input=true"
sh "terraform plan -input=true -out tfplan"
sh 'terraform show -no-color tfplan > tfplan.txt'
}
}
stage('terraform apply approval') {
when {
not {
equals expected: true, actual: params.autoApprove
}
}
steps {
script {
def plan = readFile 'tfplan.txt'
input message: "Do you want to apply the plan?",
parameters: [text(name: 'Plan', description: 'Please review the plan', defaultValue: plan)]
}
}
}
stage('terraform apply') {
steps {
sh "terraform apply -input=true tfplan"
}
}
stage('terraform destroy approval') {
steps {
input 'Run terraform destroy?'
}
}
stage('terraform destroy') {
steps {
sh 'terraform destroy -force'
}
}
}
}
- Commit and push code changes to Repo with the following:
- In Vscode, navigate to Source Code Icon on the right tabs on the side
- Enter commit message
- Click the + icon to stage changes
- Push changes by clicking on the 🔄0 ⬇️ 1 ⬆️ as shown below
Run Pipeline Job
- Go to tomcat-pipeline on Jenkins and run build
Note: The pipeline job will fail the first time to capture the parameters in Jenkinsfile
- The next time you run a build you should see as shown below
- Enter AppName
- Select a Branch/Lifecycle to deploy server
- Choose instance size for the server.
- Go to Console Output to track progress
Note: You can abort the destroy step and rerun the step by installing Blue Ocean Plugin on Jenkins to delete the resources created.
