/L8Again

Musings about distributed data infrastructure, cloud, high performance computing, and a bit of philosophy.

I am interested in high performance computing in general, and more specifically distributed systems and cloud services (admittedly biased towards AWS). I work in the data infrastructure team of a social media company that handles user generated content (UGC) for major retailer sites.

Maven-Process-Plugin: Run External Processes for your Maven Integration Tests

Maven-Process-Plugin is a simple Maven plugin I developed that would start multiple external processes before your integration tests run (pre-integration-test phase) and then terminate them after the integration tests finish (post-integration-test phase). My company Bazaarvoice has graciously open sourced this plugin.

Simply put, this plugin allows you to improve end to end testing as a part of your maven build by allowing you to start multiple processes in the order you want, in pre-integration test phase, and then stops all the processes after the integration tests are run. Simply add the following plugin to your POM file:

<plugin>
<groupId>com.bazaarvoice.maven.plugins</groupId>
<artifactId>process-exec-maven-plugin</artifactId>
<version>0.4</version>
<executions>
 <!--Start process-->
 <execution>
<id>start-jar</id>
<phase>pre-integration-test</phase>
<goals><goal>start</goal></goals>
<configuration>
 <workingDir>app</workingDir>
 <arguments>
<argument>java</argument>
<argument>-jar</argument>
<argument>app.jar</argument>
 </arguments>
</configuration>
 </execution>
 <!--Stop Process-->
<execution>
 <id>stop-jar-process</id>
 <phase>post-integration-test</phase>
 <goals><goal>stop-all</goal></goals>
</execution>
</executions>
</plugin>

I will let you look at the github readme page for details. Here, I would like to talk about some of the most frequently used applications of this plugin.

  1. Distributed Systems Testing:  Anyone dealing with distributed systems knows well about the challenge of testing and debugging such systems. This plugin allows us to write better integration tests that are closer to the production environment. It makes it really easy to spin up several distinct hosts of our application locally, and then run integration tests specifically designed to test distributed algorithms in the code. Equipped with this plugin, we can have small hooks built into our services for testing fault tolerance, introducing lag, etc. Although these tests may still not be deterministic in some cases, it brings us much closer to control quality and build confidence in complicated distributed algorithms.
     
  2. End-to-End Testing: In a service oriented architecture, services have several external dependencies. Usually testing requires us to mock up these dependencies, and hard code responses from these dependencies. Using maven-process-plugin, you can eliminate the reliance on mocked up objects and actually start dependencies locally allowing you to write richer tests. Of course, depending on your use case, your mileage may vary.  
     
  3. Testing client-side SDKs: Very often, the service you are creating needs to be consumed by other services. Hence, the need for client side SDKs (sometimes in multiple languages). In this case, not only does the plugin help in water-tight test cases for the client SDK, but your consumers now have a running example in the form of your integration tests to easily see how to use your service.

As always, let me know of your comments / suggestions / feedback should you get a chance to use this plugin.