Friday, 19 April, 2019 UTC
Summary
I'm the kind of guy who really likes to always use the latest and greatest stuff out there. 🌟 Whether it's good or bad is a whole another story. But it is this inner desire that allows me to broaden my knowledge and experience. And this time, it let me to monorepos...
Monorepo as a concept on its own isn't really that new. In fact, it's pretty old. But, with our code getting bigger and bigger, and us wanting structure that is better and better 📈, it started to gain significant traction yet again. So, in this post, we're going to explore what monorepos are, what are their main advantages and other details. Then, tailored at great web development experience, we're going to configure our own monorepo setup, based on awesome tools such as Lerna, TypeScript and Rollup! Let's get started!
So, you say monorepo?I guess you already know, or at least guess what monorepo means and what it's all about. Monorepo (nice wordplay BTW) is a term referring to the way you organize your code-base within a single repository (not a technical definition of any kind 😅). If you haven't ever read any of dozens of articles about why monorepos are great, you may think that, in this way, your on-growing code base can quickly become a huge mess. And, you guessed it! - you're completely wrong.
To understand this better, let's lower our scope-of-interest to a bit more specific territory - JS development. Currently, JavaScript workflow has been dominated by NPM packages. This form allows us to create, share and reuse code easily. Not counting the possibility of malicious code and large dependency trees that can take GBs of space, they're great! 🙃 Form development viewpoint, usually single package = single code repository - logical. But what if you develop an ecosystem of packages that will most likely depend on each other? You may even use an NPM scope to make your packages resemble that. Would you put your code in separate repos? You know that it wouldn't be great for an ecosystem to be decoupled, don't you? Also, the fact of separate issues, pull requests and the whole management process would be a nightmare, as your number of packages continues to grow. As you might expect, the solution to this problem comes in the form of monorepo.
Monorepos combine the best things from both worlds - singular, small and easy-to-manage repository with versatility and capacity of many. 👌 It's nothing more than a repo with good structure - each separate package has its own place, where the code is organized just like you'd normally do.
So, is it worth it?Definitely... under certain circumstances. Naturally, you shouldn't create a monorepo from packages that serve completely different purposes. But, as mentioned earlier, it's great for creating ecosystems of packages that work together or have the same goal. Just a rule of thumb - group only things that should be grouped. So, next time you'd like to actually create multiple separate packages with separate code-bases at once, consider if it wouldn't be better to use a monorepo. To not lure you into complete darkness, as a nice case-study, you can check out source-code and its structure for popular JS libraries and tools, such as Babel, Jest, fan-favorite React, Vue, and Angular... and plenty more.
So, to recap all the info... Monorepo groups similar packages, with a solid structure and dependencies between the packages. Now, things like single issue board, easier cross-package changes and a single place for configs, tests, and examples are obvious. But still, managing multiple packages which have dependencies on their own and are located at different paths wouldn't be so easy without appropriate tooling. In the web development world, such functionality is provided by Lerna. This tool, serving as a higher-level wrapper around a standard package manager (like NPM or Yarn), has been specially designed with monorepos in mind. It gives you access to tons of different config options and specialized commands - e.g. executing given script in every package, installing dependencies in separate packages, manage versioning and publishing to NPM. Literally, all the stuff you need to manage monorepo with ease. ✨
But, with so many different options and commands, it's easy to get quickly get lost in this mono-jungle. That's why I think it's time to create a nice monorepo structure ourselves...
How to do a monorepo?Here, I'm going to guide you through my own, personal monorepo setup. Just as said before, we'll be using Yarn, Lerna, TypeScript and Rollup. But, as we want to focus mainly on monorepo setup itself, we won't be configuring tools such as Rollup from the ground up. Instead, I'll use my favorite, Rollup-based bundler that I use in my projects, that requires much less configuration - Bili. Of course, this is just as production-ready as Rollup can be on its own. Naturally, if you've used Rollup before, you will most likely easily be able to swap it in place of Bili.
BasicsLet's start by creating our core package and installing necessary dependencies. At this point, I expect you to have Node.js and Yarn (or alternatively NPM) already installed.
yarn init
With the command above you will be guided through the basic setup of the package.json file. You can provide the data you want, but an important requirement is to set the
private option to true. This will ensure that given package won't ever be published which, in our case, means that it's just the trunk of our monorepo tree. In the end, your package.json should look similar to this:{
"name": "package",
"version": "0.0.0",
"description": "Simple package",
"main": "src/index.ts",
"license": "MIT",
"private": true
}
Next, we'll install all needed dependencies, so that we won't have to bother with them later.
yarn add lerna typescript bili rollup-plugin-typescript2 --dev
Now, let me talk about Lerna again. At its core, Lerna is a monorepo powerhouse. For most stuff you'd want to use when it comes to monorepos - it has it. All these functionalities have fine and well-written docs. Thus, it would be pointless to cover all of them in this tutorial. Instead, we're going to focus only on those commands that help us set up and work within our monorepo. Just a quick note. 😉
We'll start with creating our lerna.json file in our root directory. It's just a configuration file for Lerna to read from. Your best bet and easiest way to do this is with the
lerna init command.yarn run lerna init
This command will do 3 things - create your config file, create packages folder and add Lerna to your
devDependencies (if not already present, e.g. when using Lerna global installation). Have a look at default config file:{
"packages": [
"packages/*"
],
"version": "0.0.0"
}
packages field is an array of directories or wildcards where our packages are located. I personally consider default location in packages/ best as it is self-explanatory and doesn't require us to e.g. fill our root directory with separate packages' folders. version indicates the current version of monorepo - not necessarily in sync with the one in package.json, but it's a good practice to make it so. There are some more properties available and you can see the complete list here, but I'd like to focus on only one more - npmClient. In our case, we'll need to set it to "yarn". 🧶{
"npmClient": "yarn",
...
}
This will, naturally, indicate to use Yarn instead of default NPM to manage our packages. Now, when using Yarn you have one more important option available -
PackagesuseWorkspaces. This boolean value will let Lerna know that you want it to use Yarn workspaces feature under-the-hood to manage packages. Yarn workspaces are basically a bit lower-level solution to managing monorepos. They work a bit differently than Lerna and doesn't provide the same feature set. But, when used with Lerna they can provide better performance when e.g. linking dependencies. So, why we won't be using them? The simple answer is - they don't have good support for scope packages which, for me personally, is a deal-breaker. Monorepos and scope packages just work so well together, that I consider their support a mandatory.After all the above stuff is done, we pretty much have Lerna ready to go. Quite easy, don't you think? It's time to set up some packages then! Here, you have two options - use
lerna create to let Lerna guide you through steps needed to create your new package (just like yarn init) or get into packages folder, create sub-folders and setup each package individually.lerna create <name>
Of course, with Lerna command, you don't need to create directories or go to the packages folder at all. But still, I prefer using the standard method, as
lerna create additionally setups some boilerplate for you which, in my case, isn't what I'd like to have. 😕Now you've got your packages ready to go. In each one of them, you just proceed to create a structure as you'd normally do in a singular package. But what if these packages should share some stuff? For example, you'd want each of them to be bundled with the same tool, in the same way. For this purpose, we'll set up our Bili config at the root of our monorepo in the bili.config.js file.
But before that, a word on Bili. Bili is just a nice, Rollup-based and zero-config (by default) bundler with built-in support for ES-Next and CSS. I find it to be a very good alternative when not wanting to configure Rollup from the ground up. Even so, Bili still provides a fair amount of options when configuring itself and underlying Rollup (e.g. adding plugins). With that said, everything that, in our case, applies to Bili can be applied to Rollup-only configuration.
Now, we should to take a deeper look at paths in our config file. Consider the fairly complete setup below:
// bili.config.js
// ...
module.exports = {
input: "./src/index.ts",
output: {
moduleName: "Package",
minify: true,
format: ["umd", "esm"],
dir: "./build"
},
// ...
};
From some previous configuration files, you may know that Node.js built-in
TypeScriptpath module and provided __dirname variable is used. Here, it's important to differentiate __dirname and relative paths (always starting with ./). Our config file is at the root of our monorepo, while Bili will be run in different sub-folders separately. This is a nice functionality that Lerna provides us with, and we'll use that in a moment. But now, it's important to make our config work. So, __dirname variable references the directory where the given file is located while paths starting with ./ reference the directory relative to the path currently worked on. That's something to notice, remember and use in our config that will later be used by multiple packages at different dirs.// bili.config.js
const path = require("path");
module.exports = {
// ...
plugins: {
typescript2: {
cacheRoot: path.join(__dirname, ".rpt2_cache"),
useTsconfigDeclarationDir: true
}
}
};
You can find documentation for all other Bili options in its official docs. Here, I'd like to only talk about the plugins property, which we'll use to support TypeScript compilation (just as promised). You may remember that we previously installed, with all other dev dependencies, a Rollup plugin with the typescript2 suffix. And, it's with this suffix that you can make Bili use our plugin-of-choice and configure it freely. Note that typescript2 plugin after installation is supported by default without further configuration. But here, I wanted to change 2 options -
cacheRoot - just for our cache to not be located separately inside every package, but in the root (aesthetic reasons 💅) - and the other long one to output our TS declaration files in the directory specified in tsconfig.json.Speaking of tsconfig.json, we should have a special setup for it too! But this time it will be a bit more complicated. Inside our root directory, we'll set up our base config, for other, package-scoped, configs to inherit from.
{
"compilerOptions": {
"module": "esnext",
"lib": ["esnext", "dom"],
"strict": true,
"declaration": true,
"esModuleInterop": true,
"moduleResolution": "node"
}
}
Next, inside the directory for each of our packages, we'll need to create a separate tsconfig.json file, in which we'll place all our path-related options. For example:
{
"extends": "../../tsconfig.json",
"exclude": ["node_modules", "tests"],
"include": ["src/**/*"],
"compilerOptions": {
"declarationDir": "./typings"
}
}
With all that, we should have nice Bili + TypeScript setup, with bundles outputted to each package's build dir and typings to typings dir. Nice! 😎
UsageNow, to have our monorepo setup complete, it's time to test it! To bundle our code in every package, we can use
lerna exec:lerna exec -- bili --config ../../.bilirc.js
The two dashes (
--) after the main command allow upcoming arguments to be passed to command being executed rather than to Lerna. Now, all of our packages should be properly bundled.But, typing the same method over and over isn't required. Naturally, you can just add the
lerna exec command above to scripts property in root package.json, but I have a bit better solution. Say that you e.g. have different build scripts for each package (not the case in our config but whatever) and you'd still want to be able to run all of them with a single command. For this purpose, you can provide separate build scripts in package.json of every single package, like this (watch is just a nice addition 👍):{
...
"scripts": {
"build": "bili --config ../../.bilirc.js",
"watch": "bili --watch --config ../../.bilirc.js"
}
}
After all, scripts have been set up, you can run all of them in all your packages with
lerna run command:lerna run build
If you want
lerna run, lerna exec or other Lerna's commands (like lerna add) to only apply to certain packages, you should use filter flags i.a. --scope or --ignore. These, when passed the names of your packages (the ones in respective package.json files - not directory names) will properly select packages to apply given operation to.The last thing you must know when working with Lerna is how to make your monorepo packages depend on one another. It's pretty simple too! Just add your packages' names to given package.json
dependencies list and run lerna bootstrap to have all of them properly sym-linked and set up.lerna bootstrap
I may say that we barely scratched the surface, but also learned a lot today. Of course, Lerna still has some commands that we didn't talk about, mainly related to managing NPM publishing and releases. But, for now, the monorepo setup we've talked about has been done. Now you can freely develop your monorepo and think about release management when you'll be read. With such great tooling, it shouldn't take you too long. 😉 Then, remember to check out official (and actually very good) Lerna docs to learn more.
Monorepos... 😵So, what do you think of this article and monorepos themselves? Do you like the idea, and, what's maybe even more important, do you like the post? 😅 Write down your thoughts in the comment section below! Oh, and leave a reaction if you want!
As always, consider following me on Twitter and on my Facebook page to stay up-to-date with the latest content from this blog. Also, if you want, sign up for the newsletter below (finally launched! 🚀). Again, thank you for reading this post, and have a great day! 🖐