What is Jenkins and how can Docker be used as a Jenkins Agent?

It’s hard to picture a business without a robust continuous integration and delivery (CI/CD) tool in 2024. If you ask why, then the answer is simple – to be able to manage complex codebases, ensure consistent deployments, and maintain collaboration across development teams while reducing errors, delays or inefficiencies. One such open-source automation tool is Jenkins which can help you accomplish your software project requirements as planned.

In this blog, I’ll help you understand Jenkins. More importantly, I’ll help you with easy steps so that you can use Docker as a Jenkins Agent.

Note: Docker is a platform that uses containers to package and run applications consistently across different environments. It ensures portability, efficiency, and easy management of app dependencies.

So, sit back and enjoy this insightful read!

About Jenkins and its Architecture

Jenkins is an open-source CI/CD automation tool known for its extensive plugin ecosystem. It is widely used for its ease of installation and configuration, along with its high level of extensibility, allowing teams to customize and enhance their workflows with custom plugins. It works as an open-source automation server that can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software products.

When talking about the architecture, Jenkins features a distributed structure that can be easily scaled by distributing the workloads across nodes and agents. Its master agent (formerly known as master-slave) allows the Jenkins master to pull code from the remote SCM repository whenever code is committed to the SCM branch. The master then delegates tasks like building, testing, and generating reports to the agents.

This design is ideal for distributed build environments. That is, it allows to use different environments for each build project. Thus, balancing the workload among multiple agents and running jobs in parallel.

Now, let’s get to know about the agent.

What is a Jenkins Agent?

Refer to the following diagram for a better understanding:

Fig: Jenkins Agents

Note: What types of machines can be configured as Jenkins agents?

Well, the basic requirements for a Jenkins agent include Java installation and a network connection to the Jenkins controller. Jenkins agents can be configured on physical machines, virtual machines, Kubernetes clusters, and using Docker images.

Each of these platforms has specific methods for defining and configuring agents. From both a security and performance perspective, it’s generally more advantageous to run builds on agents rather than directly on the controller. As mentioned earlier, an agent can be a physical machine, a virtual machine, a Kubernetes cluster, or a Docker image.

Learn how we can use Docker as a Jenkins agent next!

Defining Docker as a Jenkins Agent

A CI/CD automation framework encompasses a diverse array of machines, platforms, toolchains, and operating systems. For DevOps professionals, maximizing flexibility in managing these components is crucial for seamless automation and control. To achieve interchangeable build environments and avoid dependency on specific machines, leveraging Docker containers as build agents proves to be highly effective. This approach enhances flexibility, accelerates application development, and improves overall efficiency.

Here are the prerequisites for configuring docker as a Jenkins agent:

• A machine with java and docker installed on it
• A SCM repository

Clear with the basics? Great!

To effectively use Docker as a build agent in your CI/CD pipeline, follow these steps:

Step 1. Install one plugin “Docker pipeline” for using docker as an agent. To do so, you need to do these:

• Go to Manage Jenkins and then select Manage Plugins.
• In the available section, search for “Docker Pipeline” as a plugin and install it.

Fig: Docker Pipeline Plugin

Step 2. Next, configure the machine where docker is installed as a node. For that, follow these steps:-

• Navigate to Manage Jenkins and select Manage Nodes and Clouds.
• Select “New Node” option.
• Provide the Node name and make this agent as a “Permanent agent”.
• Finally, create it.

Step 3. Next, configure the node by providing the necessary information and specifying how you want to use it. Follow these steps:

• Give a Name and Description for the node.
• In the Number of executers field, provide how many concurrent builds you want to run on that node.
• In the Remote Root Directory, specify the path on the node where tasks will be executed.
• Note: Labels are used to identify and categorize the node for specific builds.
• In the Usage section, specify how frequently you want to use the node. For optimal utilization, select “Use this node as much as possible.”

Fig: Configure Agent part 1

Step 4. The launch method describes how Jenkins starts this agent. Here, if you choose SSH, you must provide the host and private key to connect to the agent. You also have to specify the method by which you want to verify the host key.

Note: Availability specifies when the agent should start and stop.

Step 5. In the Node Properties section, specify the environment variables and tool locations required for your build, then click Save.

Fig: Configure Agent part 2

Step 6. After you have configured the agent, you need to launch the agent so that you can start using it for build. So, click on Launch agent and Jenkins will try to connect to the node.

Note: Once the connection is established, initiating the build automatically triggers the pipeline job on the master or controller. Also, you’ll be able to see a security symbol with one message beside the user’s name on Jenkins’ home page which says it’s good to use agent rather than controller.

Fig: Security Notification

Step 7: Navigate to the security message and select the “Manage” option. Then, assign a random label and ensure it matches the node. Finally, save the settings.

You can now use the configured node for your tasks.

Note: To use Docker as the agent on the node you configured, make sure your Jenkinsfile specifies Docker as the agent.

Step 8. So, create a Jenkinsfile which should look like this:

pipeline {
agent {
docker { image 'image-name' }
}
stages {
stage('stage1'){
steps {
sh 'commands'
}
}
}
}

You need to save this and keep it in the SCM (Source Control Management) repository.

Step 9. Now to create a pipeline job for your specified agent, you need to follow these small steps:-

• Select New item from Jenkins home console.
• Give a name to the project and select the pipeline.
• In the project, go to the “Pipeline” section.
• Select the Pipeline Script from the SCM option.
• Provide the following details – Repository URL, Credentials, branch and path to Jenkinsfile that we created earlier.
• Click Apply and Save.
• Finally, start building the project by clicking on Build Now

In the Console output, you’ll see that the project is running on the node you configured.

The Jenkinsfile will then designate Docker as the agent, pull the specified image from Docker Hub, spin up the container, and execute the tasks outlined in the steps.

Done!

As seen above, Docker can be used as a global agent in the project. Similarly, you can also configure Docker as a stage-specific agent as shown below:

pipeline {
agent none
stages {
stage('StageName'){
agent {
docker { image 'image-name' }
}
stpes {
sh 'command'
}
}
stage('StageName2'){
agent {
docker { image 'image-name2' }
}
stpes {
sh 'command'
}
}
}
}

If you’ve defined the global agent as ‘none’ but specified a Docker agent for each stage, then at every stage, the Docker agent will pull the specified image, spin up the container, and execute the designated tasks.

Note: If you use this Jenkinsfile after the initial one and specify the same image in any stage, it will not pull the image from Docker Hub again. Instead, it will use the existing image to spin up the container.

Bonus: Here’s another interesting way how you can define the way to use Docker as an agent.

To build an image from a Dockerfile and use that image to spin up a container for a specific task, you can define it as shown below.

pipeline {
  agent { dockerfile true }
stages {
stage('stagename'){
steps {
sh 'commands'
}
}
}
}

In this setup, you need to designate the agent that will build the image using the Dockerfile stored in your SCM repository. This agent will then create the image based on that Dockerfile.

So, I hope you’ve enjoyed exploring the various ways in which you can use Docker as an agent. A pro tip from my end would be – utilizing tools directly at runtime can streamline your workflow by eliminating the need for Jenkins-wide installations and dependency management. This approach can save businesses both time and cost.

To know more about cutting-edge software engineering solutions, feel free to reach out to Nitor Infotech.