Navigating Monorepos: Your Gentle Introduction to Efficient Web Development

A gentle introduction to Monorepo.

Abdallah Yashir Ramsing

Dec 4, 2023 • 11 min read

Introduction - What is a Monorepo?

In this post, I'm going to introduce the topic of Monorepos, and how they can be used to improve your development workflow. I'll also cover some of the downsides, and popular tools on the market that can help you get started today.

Let's first remember what a repository is.

A repository is a central location in which your project files are stored. This can be a local folder on your computer or a remote location such as GitHub or GitLab.

You can have several tools to manage your repository with the most popular one being Git.

A monorepo is a single repository containing multiple distinct projects, with well-defined relationships. -- monorepo.tools^1

Monorepo vs Monolith vs Multirepo

Here's a quick comparison between the different repository approaches. I don't think there's a right or wrong answer -- depending on your team, project and goals.

My personal opinion is Monolith is the simplest one, particularly for small teams and projects. As the team grows and you want to add different technologies, you can use the Monorepo approach.

I would argue that Multirepo is the most complex one. I think it's best suited to large teams and projects working on a multitude of domains -- especially if they are for different customers and/or projects -- without a clear relationship between them.

Here's a quick comparison:

Aspect	Monorepo	Monolith	Multirepo
Code Organization	All projects in one repository	All code in a single application	Each project in its own repository
Dependency Management	Easier, as all projects use the same versions of dependencies	Easier, as there's only one project	More complex, as each project can use different versions of dependencies
Code Sharing	Easier, as all code is in the same repository	Limited, as there's only one project	More difficult, as code is spread across multiple repositories
Build and Test	Can be more complex, as changes in one project can affect others	Simpler, as there's only one project to build and test	Can be simpler, as each project can be built and tested independently
Scalability	Can be more complex, as the repository size can become large	Limited, as all code is in a single application	Easier, as each repository is smaller and independent

Typical Scenario

To better understand the need for a Monorepo, let's have a look at a modern web application such as an eCommerce Site likely having the following components:

Frontend
Backend
Database
Infrastructure
CI/CD
Documentation
Testing
Monitoring
Analytics
Landing Page
Design System

In a monorepo, all of these components would be in a single repository. In a multirepo, each component would be in its own repository as the name suggests.

If they are in a multirepo, then each component would not only have their own repositories but you also need ways to integrate them, which can be a challenge.

Drawing upon my experience, I implemented a pipeline to run E2E Tests using Appium in Java that first launched a dotnet core application for mocking data. In the same pipeline, I had to build both projects, deploy their artefacts and run the tests. Since they were two distinct projects with their own repositories, I found it challenging as they were on different repositories.

In this scenario, a monorepo is a simpler solution -- despite the use of different languages and build systems.

Benefits of Monorepo

Clear Visibility

All the codes are in one place, so it's easy to see what's happening. You can effortlessly trace what's changed, who changed it, and why they changed it.

2. Developer Onboarding

It can be simple to onboard new developers because they can see everything in one place. They don't have to learn how to use multiple tools or repositories. Furthermore, they only need access to one repository instead of multiple repositories which can be a hassle and time consuming.

3. Code Sharing

Furthermore, it's straightforward to share code between projects because they are all in one place. You don't have to worry about copying and pasting code from one repository to another. You can just import the code from one project into another project.

The only challenge might be if you're using different languages for different projects. For example, if you're using JavaScript for one project and Ruby for another project, then you might have to write wrappers around the code so that it can be used in both projects.

One way I've seen teams solve this situation is by creating a shared library containing APIs that do the heavy lifting. An example with our eCommerce Site would be having an internal API that calculates the nearest neighbour for a group of products. This API can be used by the frontend, backend, mobile and data analytics teams -- irrespective of the language.

Moreover, this approach also helps utilise the best tool for the job rather than trying to do everything with one tool; which is like trying to fit a square peg in a round hole.

Having one implementation in the whole monorepo can be extremely useful if you want to follow the DRY (Don't Repeat Yourself) principle.

Documentation is another area where you can benefit from code sharing. You can have a single documentation for the whole project rather than having multiple documentation for each project.

In short, here are the benefits of monorepo:

Dependency Management
Build and Test
Scalability
Collaboration
Onboarding
Documentation

Limitations of Monorepo

Monorepos are no panacea though.

A panacea is a solution or remedy for all difficulties or diseases.

Code Ownership

The biggest problem in my opinion is code ownership. Let me explain with a real-life example.

In one of my previous companies, they employed contractors to work on specific projects. They were essentially broken down into microservices, many having their frontend. Each of them had their repository and specific data contracts. So if you import a library from another project, you have all the information in the contracts. However, the contractors were restricted to a set of projects. I imagine that might be due to security and confidentiality reasons among others.

When you're using multirepos, managing access rights is straightforward. You can give access to the contractor as needed. However, in a monorepo, you can't do that. You have to give access to the whole repository.

One potential solution might be the use of CODEOWNERS. In simplest terms, CODEOWNERS is a file that defines which users or teams are responsible for code in a repository. Only they can approve pull requests that modify the code they own.

Nevertheless, this doesn't prevent the contractors from accessing the code. They can still see the code albeit, they can't modify, create or approve the pull request.

2. Project Size

Another important point to consider is the size of the repository. Having a huge project not only takes a lot of time to clone but also install the dependencies and run the project. The bigger the project, the more powerful a computer you need (which can be expensive).

The big companies solve this by having a remote development environment. For example, Google has Cloud Shell and Facebook has Facebook Code. JetBrains has Space Cloud.

I wonder which technology Microsoft uses to address this. Do you know the answer? Let me know in the comments below or on Social Media.

3. IDE Limitations

An important limitation to note is most IDEs on the market today are optimised to work with a single project. Until recently, the support for multiple projects in a single repository was limited.

A glaring difference is JetBrains. You can load up multiple projects or open a big monorepo without any issue. You however do need a powerful machine.

However, we then have to address the Elephant in the room. Which license should you use?

The Ultimate version covers all the supported technologies and languages; albeit at a relatively high price.

VSCode, on the other side, a few years ago, added a set of features called Workspace to help work with multiple projects. This should work straight away with yarn / npm / pnpm workspaces, Nx, Lerna and Turborepo.

However, during my research, I encountered people complaining about the performance of VSCode when working with multiple projects and also how LSP (Language Server Protocol) doesn't work as expected.

Here's a trick I use to address this: I load the project I'm working on. If I need another one in the same repo, I use Sublime Text which is much faster and lighter than VSCode in most cases.

Tooling - DIY?

What if you want to Do It Yourself?

You need to consider the following:

How to manage dependencies?
How to build and test?
How to deploy several projects esp. if they are in different languages?
How do you manage computation caching?
How to detect only affected projects as you don't want to build, deploy and test everything?
What about code sharing particularly when talking about different languages?
There are multiple packages, libraries and frameworks that generate code and/or configs for you. How do you manage them?

In short, this is non-trivial.

Unless you have a highly specific use case with a solid team of experts, I would recommend using an existing tool.

Popular Tools

If you're already in the JavaScript ecosystem, you can use workspaces, which are built-in in yarn, npm and pnpm. You can get started with minimal fuss without having to change your workflow.

Here's a quick example of jumping in with npm workspaces:

mkdir my-monorepo
cd my-monorepo
npm init -y
mkdir packages
cd packages
mkdir frontend
mkdir backend
cd frontend
npm init -y
cd ../backend
npm init -y

You can then add the following to your package.json:

{
  "name": "my-monorepo",
  "private": true,
  "workspaces": [
    "packages/*"
  ]
}

Moreover, there is an incredible amount of high-quality open-source and paid tools on the market that can help you get started with a new project or seamlessly integrate with your multirepo projects.

Here's a non-exhaustive list of free popular tools:

Lerna from Nxwl
Nx from Nrwl
Rush from Microsoft
Bazel from Google
Buck from Facebook
Gradle Build Tool from Gradle
Moon
TurboRepo from Vercel

You can discover more here.

Looking at a quick Google Trends, it's clear that Nx is the most popular tool on the market.

Note: "NX" is a popular term and might be confused with other brands. Even then, it's still the top monorepo tool on the market at the time of writing (04/12/2023).

A quick introduction to Nx

Nx is a set of tools that help develop monorepo projects. Two ex-Googlers and Angular core team members, namely Jeff and Victor were inspired by Google large-scale monorepo system in action. They decided to build a tool to help developers amass productivity gains with an outstanding set of tooling support.

They started with Angular and the CLI tool. On popular demand, they then added support for React, Vue, NodeJS, Express, NestJS, NextJS, Storybook, Cypress, Jest, Prettier, ESLint, TypeScript and many more.

Nx even supports dotnet and Java. You can find the full list here.

Here are a few tasks you can do with Nx:

nx lint
nx watch --all -nx run
nx test
nx run-many
nx docs

You can also run tasks on affected projects only.

nx affected:lint
nx affected:test

You can further customise the tasks to your needs.

Real World Scenario: Migrating to a monorepo

With all of that mind, let's take a peek at a real world scenario and how we can use Nx to address it.

Here's the breakdown of the project:

Documentation
TypeDoc
Backend
NodeJS in TypeScript
Django
Scraping
Python
Frontend
VueJS
Tests
Jest
Cypress

Actions:

Lint
Tests
Build
Deploy

The challenge here is we have multiple projects in different languages. So how do we manage the actions?

Initial Setup

Create a new directory for your monorepo
You can call it how you like, but a common convention is to call it workspace
Initialise a new git repository
Move one project into the workspace directory
Then Git Merge or Rebase this project so that you don't lose your previous commits
Run npx create-nx-workspace
Run nx@latest init
You will be prompted to select the tools you want to use

You can also choose them later on.

I've skipped some nuances which is out of scope for this post. As a result, the above steps might not be in the exact order.

Rinse and repeat for each project. It's recommended to have one project fully integrated first before adding the others. Therefore, this migration process can take some time.

Nx.json

I've including this gist here illustrating the configuration of the real world scenario.

Note: I didn't test it. This is solely for reference.

Nx Plugins

Finally, we want to install the following plugins, which drastically reduces the boilerplate code and configuration.

JavaScript / TypeScript

@nx/devkit
extend different technos & use cases
@nx/esbuild
@nx/eslint
@nx/js
best DX JS & TS projects
@nx/cypress
@nx/vue
@nx/vite
@nx/typedoc

Python / Django

@nxlv/python
poetry
NxPy
@nx-python/nx-python

From my research though, I no longer see an official Nx plugin for Python. Nevertheless, the ones listed here will do the job. Otherwise, you can still use the devkit plugin to customise the monorepo to your needs.

You can find the full list of plugins here.

Conclusion

In this post, I've introduced the concept of monorepo and how it can help you improve your development workflow. I've also covered some of the downsides and popular tools on the market that can help you get started today. Finally, I've illustrated a real world scenario and how to use Nx to address it.

I hope you've enjoyed this post. If you have any questions, please let me know in the comments below or on Social Media.