Cloud & DevOps6 min read

CI/CD Pipeline Design

Design effective CI/CD pipelines: GitHub Actions workflow patterns, parallel jobs, artifact caching, environment protection rules, and progressive delivery.

ci-cdgithub-actionspipelinesdeployment

CI/CD pipelines automate the path from code commit to production. Well-designed pipelines are fast (parallel jobs, caching), safe (environment gates, automated tests), and observable (clear failure messages, deploy tracking). GitHub Actions is the standard for most teams.

Design for speed: run jobs in parallel, cache aggressively, fail fast. Design for safety: require approvals for production, run tests before deploy. Design for observability: every step should emit structured output that reveals what happened.

Workflow

Structure workflows with parallel jobsSequential jobs waste time - independent checks (lint, test, type-check) run faster in parallel.
Cache dependencies and build outputsInstalling node_modules from scratch on every run wastes 1-3 minutes. Caching reclaims it.
Use reusable workflows and composite actionsDRY principle applies to CI - shared setup steps should be defined once.
Gate deployments with environmentsEnvironment protection rules require human approval before deploying to production.
Implement progressive deliveryDeploying to 100% of users instantly is risky - progressive delivery limits blast radius.

Output Format

Parallel lint/typecheck/test jobs with `needs` for build, `actions/cache` for node_modules and build outputs, composite action for setup, environment gates with required reviewers, canary via feature flags.

Examples

Full CI: `name: CI on: [push, pull_request] jobs: quality: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: ./.github/actions/setup - run: npm run lint & npm run type-check & npm test -- --coverage - name: Upload coverage uses: codecov/codecov-action@v3`

Deploy with environment: `deploy: needs: [quality, build] runs-on: ubuntu-latest environment: name: production url: https://myapp.com if: github.ref == "refs/heads/main" steps: - run: npm run deploy:prod`

Matrix test: `test: strategy: { matrix: { os: [ubuntu-latest, macos-latest], node: ["20","22"] }, fail-fast: false } runs-on: ${{ matrix.os }} steps: - uses: actions/setup-node@v4 with: { node-version: "${{ matrix.node }}" } - run: npm test`

Edge Cases & Gotchas

Secrets in forks: GitHub secrets are not available to workflows triggered by fork PRs. For open-source projects, use `pull_request_target` carefully (security risk) or skip secret-dependent steps for forks
Job timeouts: GitHub Actions has a default 6-hour job timeout. Set `timeout-minutes: 30` per job to fail fast on hung jobs instead of consuming runner minutes
OIDC for cloud auth: use `permissions: id-token: write` + AWS/GCP OIDC federation instead of storing static cloud credentials as secrets - tokens are short-lived and cannot be leaked

When to Break These Rules

For very small teams (1-2 engineers) or prototype projects, a simple `npm test && git push` manual workflow is faster to set up. Invest in CI/CD when you have multiple engineers committing simultaneously.

Reference Files

GitHub Actions docs: https://docs.github.com/en/actions
Actions marketplace: https://github.com/marketplace?type=actions
Reusable workflows: https://docs.github.com/en/actions/sharing-automations/reusing-workflows
OIDC with AWS: https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services