aiagent.org aiagent.org Jenkins
Automation & Self-Hosting

Jenkins Automation

Establish self-hosted pipeline routines for Python AI agents. Learn Jenkinsfile scripting, Docker executors on Kubernetes, shared libraries, JUnit reporting, and credential routing.

Core Concepts Setup & Installation Jenkinsfile Syntax FAQ

Core Concepts

Foundational principles of self-hosted pipeline systems.

Declarative Pipelines

Define pipeline execution logic as code inside a `Jenkinsfile` stored directly inside your agent repository for version-tracked control.

K8s Executor Nodes

Spawn executor build pods dynamically inside Kubernetes clusters, scaling out testing capacity automatically under load spikes.

Shared Library logic

Extract common pipeline codes (like authentication or Slack webhooks) into reusable Groovy script libraries shared across code bases.

JUnit Report Parsing

Integrate JUnit XML reporters to display test result history metrics, error failures, and latency fluctuations on dashboards.

Post-Build Triggers

Configure downstream steps (like webhooks, Slack messages, or notification emails) that execute based on pipeline success or failure triggers.

Workspace Isolation

Isolate build workspace folders between concurrent pipeline runs on the same node to prevent database or caching states overlapping.

Setup & Installation

Configure Jenkins server and build agent nodes.

Launch Jenkins Server

# Run self-hosted Jenkins server via Docker
docker run -d \
  --name jenkins-blueocean \
  -p 8080:8080 -p 50000:50000 \
  -v jenkins-data:/var/jenkins_home \
  jenkins/jenkins:lts-jdk17

Jenkinsfile Configuration

A declarative pipeline syntax implementing testing, metric evaluation, JUnit report parsing, and Docker container push steps.

pipeline {
    agent {
        docker { image 'python:3.11-slim' }
    }
    stages {
        stage('Install') {
            steps {
                sh 'pip install -r requirements-dev.txt'
            }
        }
        stage('Test') {
            steps {
                # Output standard JUnit XML reports for parsing
                sh 'pytest --junitxml=reports/junit.xml'
            }
        }
        stage('Evaluate') {
            environment {
                OPENAI_API_KEY = credentials('openai-api-key')
            }
            steps {
                sh 'deepeval test run'
            }
        }
    }
    post {
        always {
            # Parse JUnit reports to render visualization charts
            junit testResults: '**/reports/junit.xml'
            archiveArtifacts artifacts: '**/reports/*.json', onlyIfSuccessful: false
        }
    }
}

Frequently Asked Questions

How do I isolate model credentials inside self-hosted Jenkins agents?▶
Navigate to your Jenkins Global Dashboard -> Credentials. Create a 'Secret Text' credential containing your API key. Inside your `Jenkinsfile`, use the `credentials('cred-id')` wrapper inside an environment block. Jenkins will automatically inject the key and mask its value in console log traces.
Is Jenkins better than GitHub Actions for testing agents?▶
GitHub Actions is simpler for cloud-native configurations and offers free runners. However, Jenkins is preferred by enterprise teams who must run testing workflows fully self-hosted within private VPCs due to compliance regulations or local database access requirements.
How do I scale Jenkins build nodes for heavy agent tests?▶
Install the Kubernetes plugin in Jenkins. Configure your Jenkins master to communicate with your K8s cluster. Within the `Jenkinsfile`, you can declare agent pod specs, prompting Jenkins to dynamically spin up container pods to run tests and terminate them upon completion.
What are Jenkins shared libraries?▶
Shared libraries are external git repositories of reusable Groovy code definitions. They allow multiple pipelines to import common functions (e.g. `slackNotify()`, `runEvaluations()`) without duplicate coding across projects.
How do we visualize test reports in Jenkins?▶
Under the `post { always { ... } }` step in your `Jenkinsfile`, invoke the `junit` DSL statement referencing your testing target XML report (e.g., `junit 'reports/junit.xml'`). Jenkins will parse this report and render interactive failure trends on the dashboard.
How do I configure automatic webhooks for Jenkins?▶
Install the GitHub integration plugin on Jenkins. On your GitHub repository settings page, add a Webhook pointing to `https://your-jenkins-domain/github-webhook/` triggering on push events, which prompts Jenkins to auto-trigger build pipelines.