Recently, I have published an article where I describe a few tricks about how to migrate your project from multi-repository to mono-repository. Some of you really appreciated the topic:
However, some of you wanted more details about the build process itself. So, I have written about that in depth.
TL;DR on migrations
For those who did not read the article above, I was talking about why we moved from multi-repository to mono-repository:
- Some of our repositories depended on each other, so sometimes, when you made an update in repository A, you needed to make an update of dependency A in every other repository B, C, D and so on — development costs.
- We have micro-services… but these services connected with each other (and it is not through an API, but database). Sometimes, we can not update the only one service because it breaks another. That way, we need to share the release process across all services and pin them to the same revision and test the same revision.
In case you have similar issues, I’d recommend that you consider mono-repository and read the article above.
DISCLAIMER: I’m not claiming to have the best method out there and I’m sure it has problems… But, it works for us, share your solution and I will gladly discuss it with you.
Our source code in Git
From now on, I will assume that you have migrated all of your source code to mono-repository (or you already have it, without migration). Through the article, I will use the following services as an example: api and frontend.
Our source code in mono-repository divided into separate folders under the src folder in the repository’s root. So, source code of api is under src/api folder and the same for frontend — src/frontend.
Each of these services is the separate Node.js project. src/api has its own package.json, Dockerfile, lint rules, etc… So, when you are developing something, you are working under src/api folder, doing npm install there, docker build, well, you know the drill.
We have 17 folders under our src folder (at the moment of writing the article). So, how did I configure CircleCI to build only what changed, without building everything on every commit?
CircleCI configuration
For configuration, CircleCI uses the file named config.yml under the .circleci folder in the root of your repository. Here is the one I’m using to build elastic.io platform and we will go through it with explanations (I strip it for simplicity):
If you are not familiar with YAML, you won’t understand what aliases are and what the &, « and * symbols mean. That is fine. I did not know about them a few days ago. You can read about them here (there is a section about anchors). You can reuse chunks of your YAML configuration.
I made anchors for all the commons parts of our configuration, like, what is our environment, where we need to test our platform, what are the steps, etc… at aliases section.
Common steps in CircleCI configuration are simple:
- Checkout the GitHub repository to elasticio folder
- Restore cache for node modules
- Call a npm install
- Save node modules into the cache
- Setup remote Docker engine on CircleCI, so we can use it for building images
- Run custom Bash script I wrote (we will talk about it later)
Afterwards, I am just iterating over each project we have at elastic.io, and adding it into our build jobs. But I’m changing the working_directory field for each. What does that mean?
working_directory configuration allows you to tell CircleCI, where to run your steps. So, actually, the steps described above are executing not in the repository’s root, but inside the folder with your service, that is, src/api.
Now, imagine that you have pushed a new commit with changes inside src/api. CircleCI triggers the job, gets the configuration file circle.yml and spins the environment for front-end and api jobs (since we configure them in workflows), setting up and running Docker images. It clones the repository into ~/elasticio, switches the context to ~/elasticio/src/api via working_directory configuration and calls common_steps inside of the folder.
As a result, we are getting two environments on CircleCI: api and frontend, where the source code is living. CircleCI called a npm install and configured Docker client to build our images. Our script then — ~/elasticio/.circleci/run.sh.
Our custom runner for tests and Docker images
As you may have noticed, CircleCI spins up the environment for each job, declared in workflows. Even if you did not make a change there and wanted to skip the job. Unfortunately, CircleCI can not (or, I did not find it in their documentation) filter jobs, based on some command result.
Based on that fact, we are spinning the environment for each service, living in our mono-repository, but I’d like to finish building as soon as possible. For these purposes, I have created our custom run.sh script, that is getting called every time CircleCI starts the build. What does it do?
First, it compares the diff between previous commit and the latest commit. If there are changes, it runs tests and builds an image (the script stripped for simplicity):
if git diff --name-only HEAD^...HEAD | grep "^src/${PROJECT}"; then
echo "Changes here, run the build"
npm run test
docker build --tag elasticio/${PROJECT}:${REVISION}
docker push elasticio/${PROJECT}:${REVISION}
else
echo "No changes detected"
exit 0
fi
NOTE: Remember, that we have a working_directory configuration that changes the context, where your script is running. So, actually, the script is running under the project sources, where you can call npm run test, etc.
Results
We have two files: config.yml and run.sh under the .circleci folder.
CircleCI configuration sets up the basic environment, where our Bash script is responsible for changes detection and running tests and building the Docker image.
We have 17 services in our mono-repository and CircleCI plan with 4 parallel containers. The duration of the commit build with changes in one service takes around ~3 minutes (that is for the entire mono-repository, including spinning up basic environment). These are trade-offs we agreed to take, getting in exchange for easier project development.
Ask questions regarding this on Twitter, Facebook, GitHub.
Eugene Obrezkov, Senior Software Engineer at elastic.io, Kyiv, Ukraine.
Comments