Apart from the validity and correctness of your code units, it is also essential at times to run database jobs to analyze how queries, operations, and the database itself is performing. These include mission critical operations such as banking, financial analytics, and real-time datasets, where errors can be catastrophic. This is, however, not a simple task. A database is a complex ecosystem that incorporates many moving entities, namely transaction sizes, frequency of commits, database content, type, and size of cache. The GraphAware Testing framework also provides the PerformanceTestSuite
and PerformanceTest
classes to make the performance testing process simple.
To deal with moving entities, the tests can define a parameter list that contains the desired entity. The test framework will then proceed to generate every possible permutation and use each of them to run the performance test a number of times. Among other things, in the performance tests you implement, you can specify the following entities as parameters:
Here's a simple example of performance test code that you can write:
public class PerformanceTestDemo implements PerformanceTest { enum Scenario { FIRST_SCENARIO, OTHER_SCENARIO } /**{@inheritDoc}*/ @Override public String longName() {return "Test Long Name";} /**{@inheritDoc}*/ @Override public String shortName() {return "test-short-name";} /**{@inheritDoc}*/ @Override public List<Parameter> parameters() { List<Parameter> result = new LinkedList<>(); result.add(new CacheParameter("cache")); //no cache, low-level cache, high-level cache result.add(new EnumParameter("scenario", Scenario.class)); return result; } /**{@inheritDoc}*/ @Override public int dryRuns(Map<String, Object> params) { return ((CacheConfiguration) params.get("cache")).needsWarmup() ? 10000 : 100; } /**{@inheritDoc}*/ @Override public int measuredRuns() { return 100; } /**{@inheritDoc}*/ @Override public Map<String, String> databaseParameters(Map<String, Object> params) { return ((CacheConfiguration) params.get("cache")).addToConfig(Collections.<String, String>emptyMap()); } /**{@inheritDoc}*/ @Override public void prepareDatabase(GraphDatabaseService database, final Map<String, Object> params) { //create 100 nodes in batches of 100 new NoInputBatchTransactionExecutor(database, 100, 100, new UnitOfWork<NullItem>() { @Override public void execute(GraphDatabaseService database, NullItem input, int batchNumber, int stepNumber) { database.createNode(); } }).execute(); } /**{@inheritDoc}*/ @Override public RebuildDatabase rebuildDatabase() { return RebuildDatabase.AFTER_PARAM_CHANGE; } /**{@inheritDoc}*/ @Override public long run(GraphDatabaseService database, Map<String, Object> params) { Scenario scenario = (Scenario) params.get("scenario"); switch (scenario) { case FIRST_SCENARIO: //run test for scenario 1 return 20; //the time it takes in microseconds case OTHER_SCENARIO: //run test for scenario 2 return 20; //the time it takes in microseconds default: throw new IllegalStateException("Unknown scenario"); } } /**{@inheritDoc}*/ @Override public boolean rebuildDatabase(Map<String, Object> params) { throw new UnsupportedOperationException("never needed, database rebuilt after every param change"); } }
You change the run method implementation to do some real work. Then add this test to a test suite and run it:
public class RunningDemoTest extends PerformanceTestSuite { /**{@inheritDoc}*/ @Override protected PerformanceTest[] getPerfTests() { return new PerformanceTest[]{new PerformanceTestDemo()}; } }
This example code skeleton shows a custom class that implements the PerformanceTest
class of the GraphAware library, which overrides the methods that need to be tweaked according to your requirement. The result is a total of 6 parameter permutations (the product of 2 scenarios and 3 cache types), each executed 100 times, as we have defined. When the test run process is complete, a file with the name test-short-name-***.txt
(***
being the timestamp) appears in the project root directory. The file contains the runtimes of each test round for the parameter permutations. For example, the Test Long Name
result file would contain something like this:
cache;scenario;times in microseconds... nocache;FIRST_SCENARIO;15;15;15;15;15;15;15;... nocache;OTHER_SCENARIO;15;15;15;15;15;15;15;... lowcache;FIRST_SCENARIO;15;15;15;15;15;15;15;... lowcache;OTHER_SCENARIO;15;15;15;15;15;15;15;... highcache;FIRST_SCENARIO;15;15;15;15;15;15;15;... highcache;OTHER_SCENARIO;15;15;15;15;15;15;15;...
It is also worth noting that Facebook has open sourced a benchmarking tool for social graph databases called LinkBench, which makes it possible for the Neo4j community to compare their performance metrics with a real and large dataset. You can check out the details of this system at https://github.com/graphaware/linkbench-neo4j
.