What is Pipeline? | DevOps Dictionary

Understanding CI/CD Pipelines

What Is a Pipeline in Simple Terms

A pipeline is an assembly line for software. Raw material enters at one end (a Git commit), passes through a series of automated checks and transformations (build, test, scan, deploy), and comes out the other end as running software in production. Every car that comes off an assembly line went through the same process. Every deployment that reaches production went through the same pipeline.

Before CI/CD pipelines, deploying software at Swiggy meant someone running scripts manually, hoping the steps were in the right order, discovering broken dependencies in production. Pipelines made this process repeatable, auditable, and automatic.

How It Works

Bash

+------------------------------------------+
| Trigger: git push to main branch         |
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| Stage 1: Build                           |
| Compile code, build Docker image,        |
| push to container registry               |
+------------------------------------------+
                    |
              pass or fail
                    |
                    v
+------------------------------------------+
| Stage 2: Test                            |
| Unit tests, integration tests,           |
| coverage gate enforcement                |
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| Stage 3: Scan                            |
| Image vulnerability scan (Trivy),        |
| SAST (Semgrep), dependency audit         |
+------------------------------------------+
                    |
                    v
+------------------------------------------+
| Stage 4: Deploy                          |
| Dev: automatic                           |
| Staging: automatic                       |
| Production: manual approval gate         |
+------------------------------------------+

Pipeline as code:

YAML

## .github/workflows/deploy.yaml
name: CI/CD Pipeline
 
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
 
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build Docker image
        run: docker build -t payment-api:${{ github.sha }} .
 
  test:
    needs: build    ## runs after build completes
    runs-on: ubuntu-latest
    steps:
      - name: Run tests
        run: npm test
 
  deploy:
    needs: test     ## runs after test completes
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - name: Deploy to staging
        run: ./scripts/deploy.sh staging

Practical Commands

Bash

## GitHub CLI -- trigger a pipeline manually
gh workflow run deploy.yaml
 
## GitHub CLI -- view pipeline run status
gh run list --workflow=deploy.yaml
gh run view RUN_ID
 
## GitLab -- trigger pipeline via API
curl -X POST \
  -F token=PIPELINE_TRIGGER_TOKEN \
  -F ref=main \
  https://gitlab.com/api/v4/projects/PROJECT_ID/trigger/pipeline
 
## Jenkins -- trigger build via CLI
java -jar jenkins-cli.jar -s http://jenkins.internal build payment-api-pipeline
 
## View pipeline logs
gh run view RUN_ID --log

Troubleshooting

Symptom	First Check	What to Look For
Pipeline not triggering	Check branch protection and trigger config	Branch name matches trigger filter
Stage fails silently	Check job logs	Exit code and last command output
Pipeline too slow	Profile each stage duration	Parallelise independent jobs
Flaky tests causing failures	Track failure patterns	Tests depending on external state

PLACEMENT PRO TIP
**Tip:** Store your pipeline configuration as code in the same repository as the application. When a pipeline change breaks the build, you can trace it through the same Git history as the application code — and revert both together.

REMEMBER THIS
**Remember:** Artifact promotion is the most important pipeline discipline: the Docker image built and tested in Stage 1 must be the exact same image deployed in Stage 4. Never rebuild the image for each environment — rebuilding can introduce differences that invalidate everything the tests verified.

COMMON MISTAKE / WARNING
**Security:** A CI/CD pipeline has access to production deployment credentials, container registries, and cloud accounts. Treat pipeline configuration with the same security discipline as application code. Never print secrets to logs, always use secret masking, and audit who can approve production deployments.

COMMON MISTAKE / WARNING
**Common Mistake:** Adding more stages and checks to a pipeline that is already slow instead of making existing stages faster. A pipeline that takes 45 minutes trains engineers to stop watching it and context-switch away. Aim for under 10 minutes to staging and under 20 minutes to production.