🔄 CI/CD Pipeline Overview¶
What is CI/CD?¶
CI/CD (Continuous Integration/Continuous Deployment) is a software development practice that enables teams to deliver code changes more frequently and reliably. It combines continuous integration, continuous delivery, and continuous deployment to automate the software delivery process.
CI/CD Pipeline Architecture¶
graph LR
subgraph "Development"
DEV[Developer]
GIT[Git Repository]
end
subgraph "CI/CD Pipeline"
BUILD[Build & Test]
SEC[Security Scan]
DEPLOY[Deploy]
MON[Monitor]
end
subgraph "Environments"
DEV_ENV[Development]
STAGE[Staging]
PROD[Production]
end
DEV --> GIT
GIT --> BUILD
BUILD --> SEC
SEC --> DEPLOY
DEPLOY --> DEV_ENV
DEPLOY --> STAGE
DEPLOY --> PROD
PROD --> MON
Pipeline Stages¶
1. Continuous Integration (CI)¶
Code Commit¶
# Trigger pipeline on code changes
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
Build Process¶
# Build and test application
build:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Run tests
run: |
python -m pytest tests/
python -m pytest integration_tests/
- name: Build Docker image
run: |
docker build -t my-app:${{ github.sha }} .
docker push my-app:${{ github.sha }}
Quality Gates¶
# Quality checks
quality:
runs-on: ubuntu-latest
needs: build
steps:
- name: Code quality check
run: |
flake8 src/
black --check src/
isort --check-only src/
- name: Security scan
run: |
bandit -r src/
safety check
- name: Coverage report
run: |
coverage run -m pytest
coverage report --fail-under=80
2. Continuous Delivery (CD)¶
Environment Promotion¶
# Deploy to different environments
deploy-dev:
runs-on: ubuntu-latest
needs: [build, quality]
environment: development
steps:
- name: Deploy to development
run: |
kubectl set image deployment/my-app my-app=my-app:${{ github.sha }} -n dev
kubectl rollout status deployment/my-app -n dev
deploy-staging:
runs-on: ubuntu-latest
needs: deploy-dev
environment: staging
steps:
- name: Deploy to staging
run: |
kubectl set image deployment/my-app my-app=my-app:${{ github.sha }} -n staging
kubectl rollout status deployment/my-app -n staging
deploy-production:
runs-on: ubuntu-latest
needs: deploy-staging
environment: production
steps:
- name: Deploy to production
run: |
kubectl set image deployment/my-app my-app=my-app:${{ github.sha }} -n prod
kubectl rollout status deployment/my-app -n prod
Infrastructure as Code¶
# Deploy infrastructure
deploy-infrastructure:
runs-on: ubuntu-latest
steps:
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
- name: Terraform init
run: terraform init
- name: Terraform plan
run: terraform plan -out=tfplan
- name: Terraform apply
run: terraform apply tfplan
3. Continuous Deployment¶
Automated Deployment¶
# Automated deployment with approval gates
deploy:
runs-on: ubuntu-latest
needs: [build, test, security]
environment:
name: production
url: https://my-app.example.com
steps:
- name: Deploy application
run: |
# Deploy to Kubernetes
kubectl apply -f k8s/
# Wait for deployment
kubectl rollout status deployment/my-app
# Run health checks
./scripts/health-check.sh
- name: Run smoke tests
run: |
./scripts/smoke-tests.sh
- name: Notify team
run: |
curl -X POST $SLACK_WEBHOOK \
-H 'Content-type: application/json' \
-d '{"text":"Deployment completed successfully!"}'
GitHub Actions Workflows¶
1. Main CI/CD Pipeline¶
# .github/workflows/ci-cd.yml
name: CI/CD Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Run tests
run: |
pytest tests/ --cov=src --cov-report=xml
- name: Upload coverage
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml
security:
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
- name: Run security scan
uses: snyk/actions/python@master
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
args: --severity-threshold=high
build:
runs-on: ubuntu-latest
needs: [test, security]
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Log in to Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy-dev:
runs-on: ubuntu-latest
needs: build
environment: development
steps:
- uses: actions/checkout@v4
- name: Deploy to development
run: |
kubectl config use-context dev-cluster
kubectl set image deployment/my-app my-app=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} -n dev
kubectl rollout status deployment/my-app -n dev
deploy-staging:
runs-on: ubuntu-latest
needs: deploy-dev
environment: staging
steps:
- uses: actions/checkout@v4
- name: Deploy to staging
run: |
kubectl config use-context staging-cluster
kubectl set image deployment/my-app my-app=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} -n staging
kubectl rollout status deployment/my-app -n staging
deploy-production:
runs-on: ubuntu-latest
needs: deploy-staging
environment: production
steps:
- uses: actions/checkout@v4
- name: Deploy to production
run: |
kubectl config use-context prod-cluster
kubectl set image deployment/my-app my-app=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} -n prod
kubectl rollout status deployment/my-app -n prod
- name: Run post-deployment tests
run: |
./scripts/post-deployment-tests.sh
2. Infrastructure Pipeline¶
# .github/workflows/infrastructure.yml
name: Infrastructure Pipeline
on:
push:
branches: [main]
paths: ['terraform/**', 'kubernetes/**']
workflow_dispatch:
jobs:
terraform-plan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.5.0
- name: Terraform Init
run: terraform init
working-directory: ./terraform
- name: Terraform Plan
run: terraform plan -out=tfplan
working-directory: ./terraform
- name: Upload Terraform Plan
uses: actions/upload-artifact@v4
with:
name: terraform-plan
path: ./terraform/tfplan
terraform-apply:
runs-on: ubuntu-latest
needs: terraform-plan
environment: production
steps:
- uses: actions/checkout@v4
- name: Download Terraform Plan
uses: actions/download-artifact@v4
with:
name: terraform-plan
path: ./terraform
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.5.0
- name: Terraform Apply
run: terraform apply tfplan
working-directory: ./terraform
3. Monitoring Pipeline¶
# .github/workflows/monitoring.yml
name: Monitoring Pipeline
on:
schedule:
- cron: '0 */6 * * *' # Every 6 hours
workflow_dispatch:
jobs:
deploy-monitoring:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy Prometheus
run: |
kubectl apply -f kubernetes/monitoring/prometheus/
kubectl rollout status deployment/prometheus -n monitoring
- name: Deploy Grafana
run: |
kubectl apply -f kubernetes/monitoring/grafana/
kubectl rollout status deployment/grafana -n monitoring
- name: Deploy AlertManager
run: |
kubectl apply -f kubernetes/monitoring/alertmanager/
kubectl rollout status deployment/alertmanager -n monitoring
chaos-testing:
runs-on: ubuntu-latest
needs: deploy-monitoring
steps:
- uses: actions/checkout@v4
- name: Run chaos experiments
run: |
kubectl apply -f kubernetes/chaos-engineering/chaos-experiments.yaml
sleep 300 # Wait for experiments to complete
kubectl get chaos-experiments
Environment Management¶
1. Environment Configuration¶
Development Environment¶
# environments/dev.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
namespace: dev
data:
DATABASE_URL: "postgresql://dev:password@dev-db:5432/dev"
REDIS_URL: "redis://dev-redis:6379"
LOG_LEVEL: "DEBUG"
FEATURE_FLAGS: "experimental_features=true"
Staging Environment¶
# environments/staging.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
namespace: staging
data:
DATABASE_URL: "postgresql://staging:password@staging-db:5432/staging"
REDIS_URL: "redis://staging-redis:6379"
LOG_LEVEL: "INFO"
FEATURE_FLAGS: "experimental_features=false"
Production Environment¶
# environments/prod.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
namespace: prod
data:
DATABASE_URL: "postgresql://prod:password@prod-db:5432/prod"
REDIS_URL: "redis://prod-redis:6379"
LOG_LEVEL: "WARNING"
FEATURE_FLAGS: "experimental_features=false"
2. Environment Promotion¶
# Promote configuration between environments
promote-config:
runs-on: ubuntu-latest
steps:
- name: Promote to staging
run: |
kubectl get configmap app-config -n dev -o yaml | \
sed 's/namespace: dev/namespace: staging/' | \
kubectl apply -f -
- name: Promote to production
run: |
kubectl get configmap app-config -n staging -o yaml | \
sed 's/namespace: staging/namespace: prod/' | \
kubectl apply -f -
Security in CI/CD¶
1. Secret Management¶
# Use GitHub Secrets for sensitive data
env:
DATABASE_PASSWORD: ${{ secrets.DATABASE_PASSWORD }}
API_KEY: ${{ secrets.API_KEY }}
SSL_CERT: ${{ secrets.SSL_CERT }}
2. Security Scanning¶
# Security scanning in pipeline
security-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: 'my-app:${{ github.sha }}'
format: 'sarif'
output: 'trivy-results.sarif'
- name: Upload Trivy scan results
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'
3. Compliance Checks¶
# Compliance checking
compliance:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run compliance checks
run: |
# Check for sensitive data in code
trufflehog --regex --entropy=False .
# Check for license compliance
license-checker --summary
# Check for dependency vulnerabilities
npm audit --audit-level=high
Monitoring and Observability¶
1. Pipeline Metrics¶
# Track pipeline metrics
metrics:
runs-on: ubuntu-latest
steps:
- name: Collect pipeline metrics
run: |
# Build time
echo "BUILD_TIME=$(date +%s)" >> $GITHUB_ENV
# Test results
echo "TEST_PASSED=${{ steps.test.outcome == 'success' }}" >> $GITHUB_ENV
# Deployment time
echo "DEPLOY_TIME=$(date +%s)" >> $GITHUB_ENV
2. Application Monitoring¶
# Deploy monitoring stack
monitoring:
runs-on: ubuntu-latest
steps:
- name: Deploy Prometheus
run: |
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus prometheus-community/kube-prometheus-stack
- name: Deploy Grafana
run: |
helm repo add grafana https://grafana.github.io/helm-charts
helm install grafana grafana/grafana
Best Practices¶
1. Pipeline Design¶
- Keep pipelines fast: Minimize build and test time
- Use caching: Cache dependencies and build artifacts
- Parallel execution: Run independent jobs in parallel
- Fail fast: Stop pipeline early if critical tests fail
2. Security¶
- Scan for vulnerabilities: Regular security scanning
- Use secrets management: Never hardcode secrets
- Implement least privilege: Minimal required permissions
- Audit pipeline changes: Track all pipeline modifications
3. Reliability¶
- Implement rollback: Quick rollback procedures
- Use blue-green deployment: Zero-downtime deployments
- Monitor deployments: Real-time deployment monitoring
- Test in production: Use feature flags and canary deployments
4. Observability¶
- Comprehensive logging: Log all pipeline activities
- Metrics collection: Track pipeline performance
- Alerting: Proactive alerting for issues
- Dashboards: Visual pipeline monitoring
Troubleshooting¶
1. Common Issues¶
Build Failures¶
Test Failures¶
Deployment Failures¶
# Debug deployment issues
kubectl describe pod <pod-name>
kubectl logs <pod-name>
kubectl get events --sort-by=.metadata.creationTimestamp
2. Pipeline Debugging¶
3. Rollback Procedures¶
# Automated rollback
rollback:
runs-on: ubuntu-latest
if: failure()
steps:
- name: Rollback deployment
run: |
kubectl rollout undo deployment/my-app
kubectl rollout status deployment/my-app
Future Trends¶
1. GitOps¶
- Declarative deployments: Infrastructure as code
- Automated synchronization: Git as source of truth
- Pull-based deployments: Security and audit benefits
2. Progressive Delivery¶
- Canary deployments: Gradual traffic shifting
- Feature flags: Runtime feature management
- A/B testing: Data-driven deployment decisions
3. AI/ML Integration¶
- Intelligent testing: AI-powered test generation
- Predictive deployment: ML-based deployment decisions
- Automated optimization: Self-optimizing pipelines
4. Platform Engineering¶
- Internal developer platforms: Self-service infrastructure
- Developer experience: Improved developer productivity
- Standardization: Consistent development practices