So you have written some tests for your project and now you are waiting for the test run to be completed to see if anything breaks due to the changes you have made by your last commit. Finally you can merge your changes to develop when everything seems green on your CI. But when the number of tests increases, their execution time will also increase. Our problem was exactly that. It took more than ten hours to execute one of our testing suites. Since this was notable time duration, we needed a way to distribute tests across multiple VMs as our tests cannot be run in parallel in the same VM. Our CI system Bamboo has built in support for doing such a task by using Bamboo Remote Agents. But these remote agents do not come for free and whenever our tests get increased, we will have to purchase more and more agents. So we wanted to see if there are any existing solutions for this job and my client came up with this excellent open source tool called ClusterRunner.
ClusterRunner is an open source parallel test execution tool does just that. Execute tests in parallel on multiple nodes on your testing cluster. Its master-slave architecture allows easy horizontal dynamic scaling. In other words, it’s a matter of adding more slaves to the cluster to speed up test execution. Furthermore, its intelligent job distribution engine leverages past job execution metrics to make future runs even faster. In this post I will discuss how you are going to break down your testing suite and configure ClusterRunner. But before we go into installation and setup, let’s look at how ClusterRunner works.
How Cluster Runner work
ClusterRunner can be started using two modes: master and slave mode. Master will act as a server and slaves will be connected to the cluster master. When master is up, you can submit a build job to it. This build job will contain information about where your project’s repository is located and in what branch the test suite should be executed. Cluster master will pull your project and will read the clusterrunner configuration file in it. This ClusterRunner config file dictates as to how your tests should be divided (atomized in ClusterRunner terms) among the online slaves and what commands should be invoked to run a test. Once the atomization (creation of atoms) phase is done, slaves will fetch your project and run any configured scripts defined in the clusterrunner config file, before running any tests. Once these prep scripts done executing, master will distribute atoms among the slave nodes one by one. An Atom can be a name of a test class or group of related tests. When a slave receives an atom, it will execute the configured commands on that particular atom and send any generated files by the test (test artifacts) back to the master. This process repeats until all of the atoms done processing and finally the build artifacts will be published so your CI can parse the test results and present nicely! This whole process can be summed up with the following diagram.
Now that you have some idea about what’s happening under the hood, we will see how to setup ClusterRunner under linux environment. You can follow along this tutorial even if you have a single VM, although you will need more than one VM to see parallel execution of tests.
Git and any test related tools and scripts have to be setup correctly in every master/slave machines. ClusterRunner binaries can be downloaded and installed by executing the following command: