How To Create A Scalable Jenkins CI/CD Environment Using Containers?

Jenkins is a platform for creating a Continuous Integration/Continuous Delivery (CI/CD) environment. It is written in Java and provides support for various version control tools such as Git, running Bash scripts and Windows batch files, building Ant and Apache Maven-based projects, and more. Jenkins helps build, test, and continually deploy software. It is an open-source system and one of the leading automation servers. Jenkins requires scripting some automation steps, but it provides a fast and robust way to systematize the software development lifecycle. The program runs web containers and plugins, such as Apache Tomcat, and helps manage lifecycle and access rights requests. Over 1700 plugins for Jenkins enrich the software integration, automation, and delivery processes and provide a customizable environment. Jenkins is used for the whole software delivery pipeline, including deployment.

Don’t forget to join my Discord: https://discord.gg/YbSYGsQYES

Prerequisites

  • Installed and working Podman
  • Network access
  • Console access

Installing Jenkins As A Podman Container

Create a separate network for the containers. In this example I create the n_jenkins network.

podman network create n_jenkins

Now create a new pod.

podman pod create itcafe

Create the Jenkins container. All data will be persistent under the jenkins_home volume.

podman volume create jenkins_home

podman create --pod itcafe --restart=on-failure \
  -v jenkins_home:/var/jenkins_home \
  --name jenkins --publish 8080:8080 \
  --network n_jenkins \
  --publish 50000:50000 \
  docker.io/jenkins/jenkins:lts-jdk17

You can start the pod now.

podman pod start itcafe

Setting Up The Jenkins Controller Node

Navigate to the Web UI: http://localhost:8080

Check the initial Admin password.

podman exec jenkins \
  cat /var/jenkins_home/secrets/initialAdminPassword

Use the acquired password to sign in the first time and install the suggested plugins.

The process is going to run for several minutes. We can modify, install and uninstall other plugins later.

After installing the first set of plugins now create the first Admin user of the system.

In the next step set up the Jenkins URL and you are ready to use the system!

Adding Build Agents To The Setup

In Jenkins, build nodes are the machines on which build agents run. Jenkins supports two types of nodes: agents and built-in nodes. Agents are small Java client processes that connect to a Jenkins controller and are assumed to be unreliable. They manage the task execution on behalf of the Jenkins controller by using executors. Built-in nodes, on the other hand, are nodes that exist within the Jenkins controller process. It is possible to use agents and the built-in node to run tasks, but running tasks on the built-in node is discouraged for security, performance, and scalability reasons.

Building on the built-in node can be a security issue. You should set up distributed builds. See the documentation.

To add build nodes to the setup navigate to the Manage Jenkins menu and choose the Nodes in the System Configuration section.

Use the New Node button to create a new node. Give it a name, set it to Permanent Agent and use the Create button.

Set the “Remote root directory” to /home/jenkins/agent.

Save the configuration on the bottom of the page. An offline agent will be created.

Click on the agent name to check the secret code for connecting the agent to the Jenkins instance!

Run the inbound-agent container with the secret code, the Jenkins URL and the agent name.

podman run --pod itcafe \
  -d --rm --name agent1 \
  --init docker.io/jenkins/inbound-agent \
  -url http://localhost:8080 \
  -secret <secret_code> \
  -name Agent1

If you’ve done everything right the agent will be available.

You can add as many agents as you need.

The executors can be removed now in the Built-In Node to enhance security.

Backup

Backup the pod with podman kube.

podman kube generate itcafe --filename jenkins.yml

Using this file we can restore our setup from scratch anytime we need it.

podman kube play jenkins.yml

Don’t forget to join my Discord: https://discord.gg/YbSYGsQYES

Leave a comment