Tuesday, 23 July, 2019 UTC
Summary
Splitting a large, monolithic codebase into small, encapsulated modules is usually good practice from an architectural perspective. Modularization is useful for everything from microservices to libraries of reusable components. However, it can also be a nightmare from a publishing and source code management perspective.
- Discovery goes downhill when there are dozens or hundreds of code repositories. When new repositories are created, everyone has to add their existence to their mental model of where code lives.
- Access control becomes tedious and error prone, especially when access is granted to repositories one at a time on a “need to know” basis. New employees tend to go through an endless loop of dependency hell. Each access request simply reveals two more repositories they need to access as well.
- Versioning is also complicated by a highly modular architecture. It often makes sense to “snapshot” a collection of related modules by incrementing their version in lockstep (this is what Babel and React do, for example). However, asking humans to remember to do this across a large collection of packages (including packages they haven’t had to touch) is asking for trouble.
- Duplication of dependencies between packages can vastly increase the time required to install dependencies. Some ecosystems, npm in particular, are highly modular. This encourages reuse but also means that some packages are likely duplicated in every package you develop.
Monorepos are a popular solution to this problem. Instead of one code repository per module, you put all modules in the same code repository. This “monorepo” is then the only thing a developer needs in order to do development with your application. Discovery, access control, and versioning are all simpler by virtue of everything being in the same place.
Obviously, you have to be okay with granting access to all of the code or none of the code. But if this is acceptable, monorepos can give you all the benefits of modularization with very straightforward source code management.
So how do monorepos work with npm? The two most popular solutions are Yarn workspaces and lerna. Since lerna works with npm, let’s take a look at how it implements monorepos.
First, let’s install lerna globally.
$ npm install -g lerna
Next, we need to create new lerna repository:
$ mkdir monorepo_example
$ cd monorepo_example
$ lerna init
If you look at the contents of lerna.json, you can see where the version and packages are defined.
$ cat lerna.json
{
“packages”: [
“packages/*”
],
“version”: “0.0.0”
}
As seen above, the version for all of the packages in our monorepo is 0.0.0. However, there aren’t any packages in our monorepo yet. Before we add them, we should login to our registry so lerna sets the publishConfig correctly on each new package.
If you’re publishing somewhere other than the public npm registry (for example, npm Enterprise), you’ll first need to set your registry.
$ npm config set registry https://registry.npm.yourcompany.com
Next, we authenticate to the registry with a user that has publish permissions. If your packages won’t have a scope, you can omit the –scope flag.
$ npm login –scope test
We now add a few scoped packages to our monorepo.
$ lerna create @test/a
$ lerna create @test/b
$ lerna create @test/c
If you chose the default settings for each package, you’ll now have three packages (a, b, c) in the test scope with version 0.0.0. Before we can increment the version, we need to commit what we’ve got so far and create a remote for git:
$ git add .
$ git commit -m “Initial commit”
$ git remote add origin [email protected]:username/reponame.git
$ git push -u origin master
Now we can bump the version of all packages with one command.
$ lerna version major
This command not only bumps every package to 1.0.0, but also pushes the version update to git for you. The last step is to publish these packages to npm. Lerna lets us do this with a single command.
$ lerna publish from-git
If this command succeeded, you’ve successfully published all packages in your first monorepo to npm. Hooray!
In summary, we:
- Created a new monorepo with Lerna.
- Added three new scoped packages to our monorepo.
- Bumped the version of all packages and committed to git with a single command.
- Published all packages to an npm registry with a single command.
For more on Lerna, including their solution to the duplication problem (hoisting), I recommend their excellent docs at https://github.com/lerna/lerna.
We’re also excited to announce that we plan to bring first-class monorepo support to npm@7. If you use monorepos with npm, we need your feedback! Let us know what you like or dislike about your current monorepo solution.
Happy coding,
The team at npm, Inc.