Overview
AdventureTube utilizes a backend system comprised of microservices. Each microservice is independently built as a Docker container and managed using Docker Compose. To automate and optimize these processes, Jenkins has been integrated to establish a Continuous Integration and Continuous Deployment (CI/CD) pipeline.
The pipeline is designed to work with two Raspberry Pi devices:
- Pi1: Runs Jenkins Master, PostgreSQL, MongoDB, and Kafka.
- Pi2: Runs Jenkins Agent, microservices and Spring Cloud components (Eureka, Config Server, Gateway).
Jenkins Master/Agent Architecture
Jenkins master/agent architecture is the best choice for testing conditions in a completely isolated environment. The master handles only orchestration, while the agent executes the tasks. This guarantees job performance and clear separation.
Most importantly, it provides complete toolset configuration freedom — you can configure and modify environments without affecting other builds.
Jenkins Master/Agent SSH Connection Setup
SSH is used for secure connections between the Jenkins master and agents. Pre-made SSH keys (jenkins_agent_key) are set up for authentication.
In this setup, “Launch agents via SSH” is used, allowing the controller to initiate, control, and remove agents dynamically.
Key Differences in SSH Setup
- Jenkins Master: Holds the private key (SSH Server)
- Jenkins Agent: Holds the public key (SSH Client)
This is the opposite of the standard Jenkins setup where the agent connects to the master.
Step 1: SSL Configuration
SSL Certification Configuration
- Using an Nginx proxy server as a reverse proxy
- Self-signed certificates to the Java Keystore
Port Configuration
To enable HTTPS on Jenkins Master:
ENV JENKINS_OPTS --httpPort=-1 --httpsPort=8443 --httpsKeyStore="/var/jenkins_home/.ssl/keystore.jks" --httpsKeyStorePassword="****"Step 2: SSH Connections in Jenkins
1. Controller-Agent Connection
Generate SSH key pair and register for both master and agent:
ssh-keygen -t ed25519 -C "jenkins-agent"- Name the private key
jenkins_agent_key - Register the private key in Jenkins Master as a credential
- Set the public key as an environment variable in Docker Compose for the agent
2. Git Repository SSH Connections
The agent needs a second SSH connection to access the Git repository. When code is pushed, Jenkins receives a webhook notification, and the agent pulls the code for testing, building, and deployment.
Docker in Docker Issue on Jenkins Agent
Jenkins Agent may encounter permission issues when accessing /var/run/docker.sock. This happens because the Docker group in the jenkins-agent base image may have a different group ID than the host machine.
To resolve:
sudo nano /usr/lib/systemd/system/docker.service
# Append to Service section:
SupplementaryGroups=docker
ExecStartPost=/bin/chmod 666 /var/run/docker.sock
sudo service docker restart