Simple Monorepo Workflows with GitHub Actions

Want to Keep Things Clean?

Imagine this: You're managing a project within a project, all nested under a larger one, likely in Git. Meanwhile, other teams are working on their own parts. How do you keep everything organized and ensure a clean, well-structured environment?

This situation mirrors working with Monorepos. In this short blog, I’ll share a practical approach to help you effectively manage Monorepos and maintain order across multiple teams.

But What Is a Monorepo, You May Ask?

A Monorepo is a unified way to manage related projects within a single repository. By centralizing multiple applications and shared libraries, teams can enhance collaboration, streamline development, and maintain consistency.

It sounds great, but how do you optimize your development pipeline?
The answer lies in integrating Continuous Integration and Continuous Deployment (CI/CD) processes. While effective, these can introduce challenges, especially in conflict resolution and scaling.

To tackle these challenges, it’s crucial to thoughtfully structure your CI/CD workflows. A well-organized Monorepo can support automated testing and deployment across all projects while minimizing conflicts between teams.

You can explore my GitHub repository for a practical example of this structure here.

Example Project Folder Structure

Below is a glimpse of how I've set up a simple monorepo structure:

.
├── .github
│   └── workflows
│       ├── camera_app.yml
│       └── zoom_lib.yml
├── apps
│   ├── camera_app
│   │   ├── main.py
│   │   └── test_main.py
│   └── image_compressor_app
├── libs
│   ├── cross_platform_lib
│   └── zoom_lib
│       ├── main.py
│       └── test_main.py
├── .gitignore
└── README.md

To Illustrate How Complexities Arise, Consider This Scenario:

As multiple teams work within the same Monorepo, conflicts can emerge. Imagine the following situation:

Team A updates a shared library (e.g., zoom_lib), while Team B is simultaneously developing a feature that depends on this library.
Both updates trigger CI/CD workflows, leading to potential issues like:
- Runtime Conflicts: If Team A's changes are incompatible with Team B's code, it could result in build failures or unstable deployments.
- Resource Wastage: Unnecessary builds might consume valuable time and resources, especially when Team A's changes don’t directly affect Team B's work.

A workable solution: Isolating CI/CD Workflows

Separate GitHub Actions Workflows
Implementing multiple GitHub Actions workflows that are triggered based on changes in specific file paths can help alleviate these challenges.

For instance:

Workflow A (camera_app.yml) triggers when files in apps/camera_app/** are modified.
Workflow B (zoom_lib.yml) triggers when files in libs/zoom_lib/** are modified.

Creds to ngugi-dev on mathagu-pc

Example Workflows

camera_app.yml

name: Camera App Workflow

on:
  push:
    paths:
      - 'apps/camera_app/**'
  pull_request:
    paths:
      - 'apps/camera_app/**'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Code
        uses: actions/checkout@v2

      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: '3.11'

      - name: Install Dependencies
        run: |
          cd apps/camera_app
          python -m pip install --upgrade pip
          pip install -r requirements.txt || echo "No requirements file."
          pip install pytest

      - name: Run Tests
        run: |
          cd apps/camera_app
          pytest test_main.py

      - name: Deploy
        if: github.ref == 'refs/heads/main'
        run: |
          cd apps/camera_app
          echo "Deploying Camera App..."

zoom_lib.yml

name: Zoom Library Workflow

on:
  push:
    paths:
      - 'libs/zoom_lib/**'
  pull_request:
    paths:
      - 'libs/zoom_lib/**'

Benefits of This Approach

Targeted Testing and Deployment: Only the relevant tests for each team's changes will run, minimizing the chance of triggering unnecessary builds.
Resource Optimization: This strategy reduces CI/CD resource consumption, as only necessary workflows are activated.
Reduced Conflicts: By isolating workflows, teams can work concurrently with less risk of impacting each other, leading to smoother integration and deployment processes.

Conclusion

Addressing CI/CD challenges in Monorepo environments is crucial for effective team collaboration. By implementing isolated workflows, teams can enhance productivity while leveraging the benefits of a Monorepo.

Call to Action

Consider how you manage your CI/CD processes in Monorepos and explore the implementation of targeted workflows to improve your development efficiency. For further insights and examples, check out my GitHub repository here.

Thank you for reading! On to the next one!