In today's software development world, efficiency is key, and one way to achieve that is through Continuous Integration (CI) and Continuous Deployment (CD). These practices automate the process of building, testing, and deploying applications, ultimately leading to faster, more reliable software delivery. In this blog, we will walk you through how the CI CD pipeline works, the stages of CI/CD, what tools can help you set them up, and provide a real-life example of how a project is automatically deployed using CI/CD.

Table of contents

What is CI/CD Pipeline?

CI CD Pipeline

Image Credit: Freepik

A CI/CD pipeline is used to automate software or infrastructure-as-code delivery, from source code to production. It can be thought of as a series of steps that needs to be taken for code to be released. It is an automated process that allows teams to quickly develop, test, and deploy applications in the server.

Continuous Integration(CI) refers to the practice of automatically integrating code changes from multiple contributors into a shared repository multiple times a day. Developers push their changes to the repository, and CI tools like GitHub Actions or Jenkins automatically build and test the application to ensure the new changes don't break anything.

Continuous Deployment (CD) automates the delivery of the built code to production or staging environments. The goal is to deploy new updates automatically, ensuring a smooth, rapid release process with minimal human intervention. CD takes place after the code successfully passes the testing stage of the pipeline. It refers to the automatic release to a repository after the CI stage.

Stages of CI/CD Pipeline

There are several stages of CI/CD before deploying it to production. It outlines the steps a development team takes to ensure code changes are automatically tested, built, and deployed efficiently. Here’s a breakdown of each stage in a typical CI/CD pipeline:

1. Source Stage

  • Description: The source stage involves the code commit made by a developer to the version control system (VCS), like Git. It triggers the CI/CD pipeline.

  • Actions:

    • Developers push their changes (code, configuration, etc.) to the repository.
    • Version control tools (e.g., GitHub, GitLab, Bitbucket) detect the changes and trigger the next stage in the pipeline.

  • Goal: To initiate the pipeline by committing changes to the codebase.

2. Build Stage

  • Description: The build stage is where the code gets prepared and turned into a ready-to-use package (artifact).
  • Actions:
    • The CI server triggers the build process.
    • It checks all the code dependencies, makes sure they’re available, and then compiles the code into the correct format.
    • The build artifacts are stored in an artifact repository.
  • Goal: To convert the source code into a deployable artifact and ensure that the code integrates properly.

3. Test Stage

  • Description: The test stage is where automated tests are run to check if the code is working correctly and to catch any issues before the software is deployed.
  • Actions:
    • Unit tests: Test individual components of the code.
    • Integration tests: Ensure that different components work well together.
    • If any tests fail, the pipeline halts, and the developer is notified to address the issues.
  • Goal: To ensure that new code changes do not introduce defects or regressions.

4. Package Stage

  • Description: This stage involves creating a deployable package (artifact) if the build and tests pass.
  • Actions:
    • Packaging the code into deployable artifact.
    • Versioning and tagging the package with the correct release version.
  • Goal: To package the code into a final artifact ready for deployment.

5. Deploying Stage

  • Description: In deploying stage, the code is deployed to non-production and production environments. It means your application or product was released.
  • Actions:
    • In staging, the team does additional testing, like user acceptance testing (UAT), performance testing, or load testing, to make sure everything works as expected under conditions that are similar to real-world use.
    • This environment is usually a replica of the production environment, but not live for users.
    • Once approved, the artifact is deployed to the production environment, making it available to end users.
  • Goals: To release the new code to the users with minimal downtime and high availability.

Tools for CI/CD

To set up a CI/CD pipeline, you will need a few tools. Here are some of the popular ones:

  • GitHub Actions: A powerful CI/CD tool integrated directly into GitHub repositories.
  • Jenkins: An open-source automation server that provides hundreds of plugins to support building, deploying, and automating projects.
  • Bamboo: A CI/CD tool from Atlassian that integrates seamlessly with Jira and Bitbucket.
  • CircleCI: A cloud-native continuous integration and continuous delivery platform.
  • Docker: A tool to automate the deployment of applications in lightweight, portable containers.

Why CI/CD?

Continuous Integration and Continuous Deployment (CI/CD) offer several benefits that enhance software development efficiency, quality, and reliability. Here are the key benefits:

1. Faster Development

  • Automated Builds: Automates testing and deployment, reducing the time needed to release new features. This means developers don’t have to spend time manually building and deploying applications.
  • Faster Time-to-Market: With automated pipelines, new features, bug fixes, and updates can be delivered much faster, giving businesses a competitive edge.
  • Real-World Example: Companies like Netflix and Amazon deploy code thousands of times per day using CI/CD. This allows them to continuously deliver new features without disrupting users.

2. Improve Code Quality

  • Early Bug Detection: Continuous testing catches bugs early, preventing issues from reaching production.
  • Consistent Code Reviews: Since every change goes through automated testing and validation, developers get immediate feedback, leading to better code quality.
  • Real-World Example: Google’s CI/CD process includes thousands of automated tests per minute, ensuring only high-quality code reaches production.

3. Better Collaboration Between Teams

  • Faster Code Merging: Developers work on different branches, and CI/CD automatically integrates changes into the main codebase. This makes collaboration seamless.
  • Improved Communication: With automated testing and deployment, teams (developers, testers, DevOps) can focus on innovation instead of fixing integration issues.
  • Real-World Example: Facebook uses CI/CD to ensure thousands of engineers can work on the same codebase without conflicts.

Relationship Between DevOps and CI/CD

DevOps and CI/CD are closely related concepts in modern software development. CI/CD is a crucial component of DevOps, helping teams achieve faster and more reliable software delivery. While DevOps focuses on culture and collaboration, CI/CD automates the technical processes that enable rapid and reliable software releases.

DevOps is a philosophy and culture that promotes collaboration between development (Dev) and operations (Ops) teams. It focuses on automation, continuous monitoring, and integration to improve software quality and delivery speed.

CI/CD helps realize the full potential of DevOps by enabling speed, efficiency, and reliability in software development. Here's why it matters:

DevOps Principle How CI/CD Supports It
Collaboration Developers and Operations teams work together on automated pipelines.
Automation CI/CD reduces manual tasks, ensuring faster and error-free deployments.
Continuous Feedback Developers receive quick test results and production insights.
Faster Time-to-Market CI/CD enables rapid release cycles with minimal disruptions.
Security & Compliance Automated security tests (DevSecOps) ensure secure deployments.

DevOps Without CI/CD?

  • Possible, but inefficient. Without CI/CD, software releases become slow and error-prone.
  • DevOps relies on automation, and CI/CD is a major enabler of this automation.

CI/CD Without DevOps?

  • CI/CD can exist independently, but its full potential is unlocked within a DevOps culture.
  • Without DevOps, CI/CD may lack collaboration, monitoring, and security practices, leading to inefficiencies.

Real-World Scenario: Deploying a Web App Automatically

I have a DemosThemeselection hosted on GitHub that we want to deploy automatically to Themeselection(Offers templates and UI kits for building web applications). We will use GitHub Actions to set up our CI/CD pipeline.

Step 1: Create Your Project

Create a new directory and initialize your project according to your technologies:

mkdir demos-themeselection
cd demos-themeselection
npm init -y
npm install express

Step 2: Push Code to GitHub

Once the app/website is ready, initialize a Git repository and push it to GitHub:

git init
git add .
git commit -m "Initial commit"
git remote add origin https://github.com/yourusername/yourrepositoryname.git
git push

Step 3: Set Up GitHub Actions for CI/CD

Now, let’s configure GitHub Actions to automate the CI/CD process. GitHub Actions allows you to define workflows for different tasks, like testing, building, and deploying your code.

Create a directory .github/workflows in your project, and add a new file called eg: deploy.yml:

name: 🚀 Deploy - Demos ThemeSelection
run-name: ${{ inputs.is_production && '🚀'}} Deploy - Demos ThemeSelection

on:
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest

    env:
      DEPLOY_DIR: "${{ secrets.PROD_DIR }}"
      ORG_NAME: themeselection
      REPO_NAME: platform-demo

    steps:
      - run: echo "Deploying to production"

      - name: Checkout platform-demo repo
        uses: actions/checkout@v4
        with:
          repository: ${{ env.ORG_NAME }}/${{ env.REPO_NAME }}
          token: ${{ secrets.GH_PAT }}

      - name: Remove unwanted folders before upload
        run: |
          rm -rf .git .github

      - name: Upload all files to the server (excluding .git and .github)
        uses: appleboy/scp-action@master
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          port: ${{ secrets.PORT }}
          key: ${{ secrets.SSHKEY }}
          source: "./*"
          target: ${{ env.DEPLOY_DIR }}

      - name: Cleanup old files on the server
        uses: appleboy/[email protected]
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          port: ${{ secrets.PORT }}
          key: ${{ secrets.SSHKEY }}
          script: |
            cd ${{ env.DEPLOY_DIR }}

            # Remove any leftover .git or .github folder if present on the server
            rm -rf .git .github

Explanation of the Workflow:

Workflow Trigger:

  • The workflow is manually triggered using workflow_dispatch. This is a good choice if you want to have more control over when the deployment happens (i.e., you trigger it manually, rather than automatically on a push or other event).

Job: build

  • Runs on ubuntu-latest, meaning GitHub Actions will run the job on a fresh Ubuntu runner, which is appropriate for most deployment processes.

Environment Variables:

  • DEPLOY_DIR: This is the directory on the server where the files will be uploaded. The value is retrieved from GitHub Secrets (${{ secrets.PROD_DIR }}).
  • ORG_NAME and REPO_NAME: These define which repository to clone. In this case, it's the platform-demo repository from the themeselection GitHub organization.

Steps:

  • Step 1: "Deploying to production" message
    • This step outputs a simple message to let you know the workflow is starting the deployment.
  • Step 2: Checkout the repository
    • This step checks out the platform-demo repository using actions/checkout. The repository is accessed using a personal access token (PAT) stored in the GitHub Secrets (${{ secrets.GH_PAT }}).
  • Step 3: Remove unwanted folders
    • This step deletes the .git and .github directories, which are typically not needed on the production server and might contain unnecessary files or configurations.
  • Step 4: Upload files to the server
    • This step uses the appleboy/scp-action to securely copy all the project files (excluding .git and .github) from the GitHub runner to your server. The action uses SSH to authenticate and transfer the files.
    • It references secrets for the server's host, username, port, and SSH key.
  • Step 5: Clean up old files on the server
    • This step uses appleboy/ssh-action to SSH into the server and remove any leftover .git or .github directories that may have been left behind from previous deployments. This keeps your production server clean and free from unnecessary files.

Conclusion

In this guide, we covered the basics of CI/CD and how to automate deployment for your project using GitHub Actions. With this setup, any changes you make to the codebase will automatically go through the build and deployment process, saving time and reducing the risk of human error.

Whether you’re deploying to Github or any other platform, the principles of CI/CD remain the same. By automating repetitive tasks like testing and deployment, you free up time to focus on building features and improving your product.

Remember, this is just the beginning! As you gain experience, you can extend your CI/CD pipeline to include additional features like code linting, security checks, and performance testing. But for now, enjoy the benefits of automation in your project’s deployment pipeline!

Happy coding🚀