Jenkins - CI/CD Pipelines

Jenkins is the industry standard for on-premise and self-hosted automation. Unlike GitHub Actions (cloud-native), Jenkins is deployed on your own infrastructure, giving you complete control and flexibility. It uses a Jenkinsfile to define the pipeline logic using a Groovy-based syntax.

1. Terminology & Architecture

Core Concepts

Term	Description
Pipeline	The entire CI/CD workflow defined in a Jenkinsfile.
Stage	A major section of the pipeline (Build, Test, Deploy).
Step	An individual task within a stage.
Agent	A worker machine (controller or agent node) that executes steps.
Executor	A thread on an agent that can run a build.
Credentials	Securely stored passwords, tokens, SSH keys.
Plugin	Extension that adds functionality (Docker, Kubernetes, AWS, etc.).

Jenkins Architecture

┌─────────────────────────────────────┐
│    Jenkins Controller (Master)       │
│  - Web UI (port 8080)                │
│  - Job Configuration                 │
│  - Build Orchestration               │
│  - Credentials Storage               │
│  - Plugin Management                 │
└──────────┬──────────────────────────┘
           │
    ┌──────┴──────┬──────────┬──────────┐
    │             │          │          │
┌───▼───┐   ┌────▼───┐  ┌──▼──────┐ ┌─▼────────┐
│Agent 1│   │Agent 2 │  │Agent 3  │ │Agent N   │
│Linux  │   │macOS   │  │Windows  │ │(On-Prem) │
│2 exec │   │4 exec  │  │2 exec   │ │Variable  │
└───────┘   └────────┘  └─────────┘ └──────────┘

Each Agent runs builds in parallel across multiple executors

2. Installation & Setup

Install Jenkins

Docker:

docker run -d -p 8080:8080 -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  -e JAVA_OPTS="-Xmx512m" \
  jenkins/jenkins:lts

Linux (Ubuntu):

curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo tee /usr/share/keyrings/jenkins.asc
echo "deb [signed-by=/usr/share/keyrings/jenkins.asc] https://pkg.jenkins.io/debian-stable binary/" | \
  sudo tee /etc/apt/sources.list.d/jenkins.list
sudo apt-get update
sudo apt-get install jenkins openjdk-11-jre
sudo systemctl start jenkins
sudo systemctl enable jenkins

Initial Setup

Navigate to http://localhost:8080
Retrieve initial admin password: cat /var/jenkins_home/secrets/initialAdminPassword
Install recommended plugins
Create first admin user
Configure Jenkins URL (System → Configure System)

3. Declarative Pipeline: Complete Reference

Basic Structure

pipeline {
    agent any                           // Run on any available agent
 
    options {
        timestamps()                    // Add timestamps to console output
        timeout(time: 30, unit: 'MINUTES')  // Kill job if > 30 min
        buildDiscarder(logRotator(numToKeepStr: '10'))  // Keep 10 latest builds
    }
 
    parameters {
        string(name: 'ENVIRONMENT', defaultValue: 'staging', description: 'Deploy to which environment?')
        choice(name: 'LOG_LEVEL', choices: ['INFO', 'DEBUG', 'TRACE'], description: 'Logging level')
        booleanParam(name: 'RUN_TESTS', defaultValue: true, description: 'Run tests?')
    }
 
    environment {
        APP_VERSION = '1.0.0'
        REGISTRY = credentials('docker-registry-url')
        AWS_CREDENTIALS = credentials('aws-credentials-id')
    }
 
    stages {
        stage('Checkout') {
            steps {
                checkout scm
                echo "Checking out from ${env.GIT_BRANCH}"
            }
        }
 
        stage('Build') {
            steps {
                script {
                    echo "Building version ${APP_VERSION}"
                    sh 'npm install && npm run build'
                }
            }
        }
 
        stage('Test') {
            when {
                expression { params.RUN_TESTS == true }
            }
            steps {
                sh 'npm test'
            }
        }
 
        stage('Deploy') {
            when {
                branch 'main'                           // Only on main branch
            }
            environment {
                DEPLOY_ENV = "${params.ENVIRONMENT}"
            }
            steps {
                sh './scripts/deploy.sh'
            }
        }
    }
 
    post {
        always {
            junit 'test-results.xml'                    // Parse test results
            archiveArtifacts artifacts: 'dist/**'       // Save build artifacts
        }
        success {
            echo 'Pipeline succeeded!'
        }
        failure {
            echo 'Pipeline failed!'
        }
        unstable {
            echo 'Pipeline unstable (tests failed but continued)'
        }
        cleanup {
            deleteDir()                                 // Clean workspace
        }
    }
}

4. Scripted Pipeline: Advanced Flexibility

For complex logic, use Scripted Pipeline (Groovy):

node('linux-agent') {
    try {
        stage('Checkout') {
            checkout scm
        }
 
        stage('Build') {
            sh 'npm install'
            def version = sh(script: "npm run get-version", returnStdout: true).trim()
            echo "Built version: ${version}"
        }
 
        stage('Parallel Tests') {
            parallel(
                'Unit Tests': {
                    sh 'npm run test:unit'
                },
                'Integration Tests': {
                    sh 'npm run test:integration'
                },
                'E2E Tests': {
                    sh 'npm run test:e2e'
                }
            )
        }
 
        stage('Deploy') {
            if (env.BRANCH_NAME == 'main') {
                sh './deploy.sh production'
            } else if (env.BRANCH_NAME == 'develop') {
                sh './deploy.sh staging'
            }
        }
    } catch (Exception e) {
        echo "Build failed: ${e.message}"
        throw e
    } finally {
        junit 'test-results/**/*.xml'
    }
}

5. Agents & Distributed Builds

Setup an Agent Node

Go to Manage Jenkins → Manage Nodes and Clouds → New Node
Enter node name, select "Permanent Agent"
Configure:

Node Name: linux-builder-1
Executors: 4                    # Run 4 builds in parallel
Remote root directory: /var/jenkins
Labels: linux, docker, builder
Launch method: SSH (or Java Web Start)
SSH credentials: Select SSH key
Host: 192.168.1.100

Use Agent in Pipeline

pipeline {
    agent {
        label 'linux && docker'         // Use agents with both labels
    }
    // or specific agent:
    // agent { node { label 'linux-builder-1' } }
    
    stages {
        stage('Build') {
            steps {
                sh 'docker build -t myapp:latest .'
            }
        }
    }
}

Agent Pools for Scaling

pipeline {
    agent none
 
    stages {
        stage('Build on Linux') {
            agent { label 'linux' }
            steps {
                sh 'make build'
            }
        }
 
        stage('Test on macOS') {
            agent { label 'macos' }
            steps {
                sh 'make test'
            }
        }
 
        stage('Package on Windows') {
            agent { label 'windows' }
            steps {
                bat 'make.bat package'
            }
        }
    }
}

6. Stages, Steps & Post Actions

Conditional Stages

stage('Deploy to Production') {
    when {
        allOf {
            branch 'main'
            not { changeRequest() }     // Not a pull request
        }
    }
    steps {
        sh './deploy.sh production'
    }
}
 
stage('Run Security Scan') {
    when {
        expression { 
            currentBuild.result == null  // No failures yet
        }
    }
    steps {
        sh 'npm audit'
    }
}

Post Actions

post {
    always {
        // Always run
        sh 'echo "Cleaning up..."'
        deleteDir()
    }
    success {
        // Only if successful
        slackSend(channel: '#deployments', message: 'Build succeeded!')
    }
    failure {
        // Only if failed
        emailext(
            subject: "Build Failed: ${env.JOB_NAME} #${env.BUILD_NUMBER}",
            body: "Build logs: ${env.BUILD_URL}console",
            to: '[email protected]'
        )
    }
    unstable {
        // Tests failed but build continued
        echo 'Tests failed but continuing...'
    }
    cleanup {
        // Last action (after post{} block)
        sh 'rm -rf temp/*'
    }
}

7. Parameters & User Input

pipeline {
    parameters {
        string(
            name: 'VERSION',
            defaultValue: '1.0.0',
            description: 'Version to release'
        )
        choice(
            name: 'ENVIRONMENT',
            choices: ['staging', 'production'],
            description: 'Deployment target'
        )
        booleanParam(
            name: 'SKIP_TESTS',
            defaultValue: false,
            description: 'Skip tests? (for emergency releases)'
        )
        text(
            name: 'RELEASE_NOTES',
            defaultValue: 'Enter release notes here',
            description: 'Notes for this release'
        )
    }
 
    stages {
        stage('Deploy') {
            steps {
                echo "Deploying v${params.VERSION} to ${params.ENVIRONMENT}"
                sh "echo '${params.RELEASE_NOTES}' > RELEASE.txt"
            }
        }
    }
}

Access parameters with params.PARAMETER_NAME or ${params.PARAMETER_NAME}

8. Credentials & Secrets

Store Credentials

Go to Manage Jenkins → Manage Credentials
Click "Add Credentials" → Select type (Username/Password, SSH Key, AWS, etc.)
Fill in credentials and give them an ID

Use in Pipeline

pipeline {
    environment {
        // Method 1: Access directly
        DB_PASSWORD = credentials('prod-db-password')
        
        // Method 2: SSH credentials
        SSH_KEY = credentials('prod-ssh-key')
        
        // Method 3: AWS credentials
        AWS_CREDS = credentials('aws-prod-creds')
    }
 
    stages {
        stage('Deploy') {
            steps {
                script {
                    // withCredentials wraps sensitive access
                    withCredentials([
                        usernamePassword(credentialsId: 'dockerhub-creds', usernameVariable: 'DOCKER_USER', passwordVariable: 'DOCKER_PASS')
                    ]) {
                        sh 'echo $DOCKER_PASS | docker login -u $DOCKER_USER --password-stdin'
                        sh 'docker push myapp:latest'
                    }
                }
            }
        }
    }
}

Mask Sensitive Output

steps {
    script {
        wrap([$class: 'BuildNameUpdater', 
              newName: 'Deployment-${BUILD_NUMBER}'
        ]) {
            sh '''
                set +x  # Disable debug output
                echo "API_KEY=${API_KEY}" > .env
                docker run --env-file .env myapp:latest
                rm .env
                set -x  # Re-enable debug
            '''
        }
    }
}

9. Essential Plugins

Plugin	Purpose	Example
Pipeline	Core declarative pipeline support	Built-in
GitLab/GitHub API	GitHub/GitLab integration	Webhooks, PR triggers
Docker Pipeline	Build and push Docker images	`docker.build()`, `docker.image().push()`
Kubernetes	Deploy to Kubernetes clusters	Pod templates, kubectl
AWS Steps	AWS operations (S3, EC2, ECS)	Upload to S3, trigger Lambda
Slack	Send Slack notifications	Post build status to Slack
Email	Email notifications	Send build reports
Blue Ocean	Modern UI for pipelines	Visualization, better logs
JUnit	Parse test results	`junit` step
Cobertura	Code coverage reports	Coverage graphs
Groovy Postbuild	Post-build Groovy scripts	Conditional logic
Matrix Authorization	Role-based access control	User permissions
HashiCorp Vault	Secret management	Centralized secrets

10. Notifications

Slack Integration

post {
    always {
        slackSend(
            channel: '#builds',
            color: currentBuild.result == 'SUCCESS' ? 'good' : 'danger',
            message: """
                Build: ${env.JOB_NAME} #${env.BUILD_NUMBER}
                Status: ${currentBuild.result}
                Duration: ${currentBuild.durationString}
                Details: ${env.BUILD_URL}
            """
        )
    }
}

Email Notifications

post {
    failure {
        emailext(
            subject: "FAILED: ${env.JOB_NAME} #${env.BUILD_NUMBER}",
            body: """
                Build failed: ${env.BUILD_URL}
                
                Changes:
                ${currentBuild.changeSets}
                
                Check console output: ${env.BUILD_URL}console
            """,
            to: '${DEFAULT_RECIPIENTS}',
            attachLog: true
        )
    }
}

Custom Webhook

post {
    always {
        httpRequest(
            url: 'https://hooks.example.com/builds',
            httpMode: 'POST',
            requestBody: """
                {
                    "job": "${env.JOB_NAME}",
                    "build": ${env.BUILD_NUMBER},
                    "status": "${currentBuild.result}",
                    "url": "${env.BUILD_URL}"
                }
            """
        )
    }
}

11. Real-World Examples

Full CI/CD Pipeline with Parallel Testing

pipeline {
    agent any
 
    options {
        timestamps()
        timeout(time: 60, unit: 'MINUTES')
        buildDiscarder(logRotator(numToKeepStr: '20'))
    }
 
    environment {
        APP_NAME = 'myapp'
        REGISTRY = 'docker.io/mycompany'
        VERSION = "${env.BUILD_NUMBER}"
    }
 
    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
 
        stage('Build') {
            steps {
                sh '''
                    npm ci
                    npm run build
                    npm run build:docker -- ${REGISTRY}/${APP_NAME}:${VERSION}
                '''
            }
        }
 
        stage('Parallel Testing') {
            parallel {
                stage('Unit Tests') {
                    steps {
                        sh 'npm run test:unit'
                    }
                }
                stage('Integration Tests') {
                    steps {
                        sh 'npm run test:integration'
                    }
                }
                stage('Lint & Format') {
                    steps {
                        sh 'npm run lint && npm run format:check'
                    }
                }
                stage('Security Scan') {
                    steps {
                        sh 'npm audit'
                    }
                }
            }
        }
 
        stage('Docker Push') {
            when {
                branch 'main'
            }
            steps {
                script {
                    withCredentials([usernamePassword(credentialsId: 'dockerhub-creds', 
                        usernameVariable: 'DOCKER_USER', 
                        passwordVariable: 'DOCKER_PASS')]) {
                        sh '''
                            echo $DOCKER_PASS | docker login -u $DOCKER_USER --password-stdin
                            docker push ${REGISTRY}/${APP_NAME}:${VERSION}
                            docker tag ${REGISTRY}/${APP_NAME}:${VERSION} ${REGISTRY}/${APP_NAME}:latest
                            docker push ${REGISTRY}/${APP_NAME}:latest
                        '''
                    }
                }
            }
        }
 
        stage('Deploy Staging') {
            when {
                branch 'develop'
            }
            steps {
                sh '''
                    kubectl set image deployment/myapp-staging \
                        myapp=${REGISTRY}/${APP_NAME}:${VERSION} \
                        -n staging
                    kubectl rollout status deployment/myapp-staging -n staging
                '''
            }
        }
 
        stage('Deploy Production') {
            when {
                branch 'main'
            }
            input {
                message "Deploy to production?"
                ok "Deploy"
            }
            steps {
                sh '''
                    kubectl set image deployment/myapp \
                        myapp=${REGISTRY}/${APP_NAME}:${VERSION} \
                        -n production
                    kubectl rollout status deployment/myapp -n production
                '''
            }
        }
    }
 
    post {
        always {
            junit 'test-results/**/*.xml'
            publishHTML([
                reportDir: 'coverage',
                reportFiles: 'index.html',
                reportName: 'Code Coverage'
            ])
            slackSend(
                channel: '#builds',
                color: currentBuild.result == 'SUCCESS' ? 'good' : 'danger',
                message: "${env.JOB_NAME} #${env.BUILD_NUMBER} - ${currentBuild.result}"
            )
        }
        failure {
            emailext(
                subject: "FAILED: ${env.JOB_NAME} #${env.BUILD_NUMBER}",
                body: "Build failed. Check logs: ${env.BUILD_URL}console",
                to: '[email protected]',
                attachLog: true
            )
        }
        cleanup {
            deleteDir()
        }
    }
}

Multi-Branch Pipeline (GitHub/GitLab)

pipeline {
    agent any
 
    stages {
        stage('Build') {
            steps {
                sh 'npm run build'
            }
        }
 
        stage('Test') {
            steps {
                sh 'npm test'
            }
        }
 
        stage('Deploy') {
            when {
                anyOf {
                    branch 'main'
                    branch 'develop'
                }
            }
            steps {
                script {
                    def target = env.BRANCH_NAME == 'main' ? 'production' : 'staging'
                    sh "aws ecs update-service --cluster ${target} --service myapp --force-new-deployment"
                }
            }
        }
    }
}

12. Scaling & High Availability

Controller Backup & Recovery

# Backup Jenkins home directory
tar -czf jenkins-backup-$(date +%Y%m%d).tar.gz /var/jenkins_home
 
# Restore from backup
tar -xzf jenkins-backup-20240101.tar.gz -C /var
sudo systemctl restart jenkins

Load Balancing Multiple Controllers

# Use HAProxy in front of Jenkins
listen jenkins
    bind 0.0.0.0:8080
    balance roundrobin
    server jenkins1 192.168.1.10:8080
    server jenkins2 192.168.1.11:8080
    server jenkins3 192.168.1.12:8080

High Availability with Shared Storage

# All controllers access same job data
jenkins1:/var/jenkins_home -> NFS mount /mnt/jenkins_shared
jenkins2:/var/jenkins_home -> NFS mount /mnt/jenkins_shared
jenkins3:/var/jenkins_home -> NFS mount /mnt/jenkins_shared

13. Debugging & Troubleshooting

Enable Debug Logging

steps {
    sh 'set -x'  # Enable debug output
    sh 'npm run build'
    sh 'set +x'  # Disable debug output
}

View Build Logs

post {
    failure {
        script {
            sh 'cat /var/jenkins_home/logs/jenkins.log | tail -100'
        }
    }
}

Check Agent Connectivity

# From Jenkins controller
ssh jenkins-agent-1 "java -version"
 
# Or use agent CLI
java -jar agent.jar -jnlpUrl http://jenkins:8080/computer/agent1/slave-agent.jnlp -secret TOKEN

Log Scripted Pipeline Steps

timestamps {
    ansiColor('xterm') {
        sh '''
            set -e  # Exit on error
            set -u  # Exit on undefined variable
            set -o pipefail  # Exit if pipe command fails
            
            echo "==== Building ===="
            npm run build
            echo "Build completed successfully"
        '''
    }
}

14. Best Practices

✅ DO:

Use Declarative Pipeline for most pipelines
Store Jenkinsfiles in Git repository
Use credentials() for secrets (never hardcode)
Run builds on agents, not controller
Use meaningful stage names
Archive test results and artifacts
Set reasonable timeouts
Use labels to route jobs to appropriate agents
Implement proper access controls (security realms, authorization)
Monitor agent load and scale accordingly
Backup controller configuration regularly
Use pipeline libraries for reusable code

❌ DON'T:

Use Scripted Pipeline unless you need advanced logic
Hardcode API keys or passwords
Run heavy builds on the controller
Ignore disk space on controller
Use broad agent labels (be specific)
Disable authentication
Run untrusted pipelines without review
Leave old builds cluttering the system
Use always: true without careful consideration
Make multiple sequential API calls instead of batch operations

15. Jenkins vs GitHub Actions

Feature	Jenkins	GitHub Actions
Hosting	Self-hosted	Cloud (GitHub.com)
Setup	Complex (install, configure)	Built-in to GitHub
Scaling	Manual (add agents)	Automatic (managed by GitHub)
Cost	Infrastructure costs	Free for public repos
Control	Full (firewall, network, etc.)	Limited (GitHub-hosted)
Plugins	1,800+ third-party	GitHub Marketplace + custom
Learning Curve	Steeper	Gentler
Suitable For	Enterprise, on-prem	SaaS, small teams, startups

Summary

Jenkins remains the dominant self-hosted CI/CD platform:

Flexible: Declarative and Scripted pipelines for any use case
Scalable: Distribute across dozens of agents
Extensible: 1,800+ plugins for any integration
Reliable: Proven in production for 18+ years
Open Source: Community-driven, no vendor lock-in
Full Control: Run on your infrastructure, your rules

Start simple (build → test → deploy), then layer in parallel tests, security scans, and multi-environment deployments as complexity grows.