Apart from the validity and correctness of your code units, it is also essential at times to run database jobs to analyze how queries, operations, and the database itself is performing. These include mission critical operations such as banking, financial analytics, and real-time datasets, where errors can be catastrophic. This is, however, not a simple task. A database is a complex ecosystem that incorporates many moving entities, namely transaction sizes, frequency of commits, database content, type, and size of cache. The GraphAware Testing framework also provides the PerformanceTestSuite and PerformanceTest classes to make the performance testing process simple.
To deal with moving entities, the tests can define a parameter list that contains the desired entity. The test framework will then proceed to generate every possible permutation and use each of them to run the performance test a number of times. Among other things, in the performance tests you implement, you can specify the following entities as parameters:
- The number of times to run the tests and get performance metrics
- The number of dry runs to perform so that a warm cache is established to test speed of cache retrievals
- The parameters that are to be used
- When to discard the test database and build a new one
Here's a simple example of performance test code that you can write:
public class PerformanceTestDemo implements PerformanceTest {
enum Scenario {
FIRST_SCENARIO,
OTHER_SCENARIO
}
/**{@inheritDoc}*/
@Override
public String longName() {return "Test Long Name";}
/**{@inheritDoc}*/
@Override
public String shortName() {return "test-short-name";}
/**{@inheritDoc}*/
@Override
public List<Parameter> parameters() {
List<Parameter> result = new LinkedList<>();
result.add(new CacheParameter("cache")); //no cache, low-level cache, high-level cache
result.add(new EnumParameter("scenario", Scenario.class));
return result;
}
/**{@inheritDoc}*/
@Override
public int dryRuns(Map<String, Object> params) {
return ((CacheConfiguration) params.get("cache")).needsWarmup() ? 10000 : 100;
}
/**{@inheritDoc}*/
@Override
public int measuredRuns() {
return 100;
}
/**{@inheritDoc}*/
@Override
public Map<String, String> databaseParameters(Map<String, Object> params) {
return ((CacheConfiguration) params.get("cache")).addToConfig(Collections.<String, String>emptyMap());
}
/**{@inheritDoc}*/
@Override
public void prepareDatabase(GraphDatabaseService database, final Map<String, Object> params) {
//create 100 nodes in batches of 100
new NoInputBatchTransactionExecutor(database, 100, 100, new UnitOfWork<NullItem>() {
@Override
public void execute(GraphDatabaseService database, NullItem input, int batchNumber, int stepNumber) {
database.createNode();
}
}).execute();
}
/**{@inheritDoc}*/
@Override
public RebuildDatabase rebuildDatabase() {
return RebuildDatabase.AFTER_PARAM_CHANGE;
}
/**{@inheritDoc}*/
@Override
public long run(GraphDatabaseService database, Map<String, Object> params) {
Scenario scenario = (Scenario) params.get("scenario");
switch (scenario) {
case FIRST_SCENARIO:
//run test for scenario 1
return 20; //the time it takes in microseconds
case OTHER_SCENARIO:
//run test for scenario 2
return 20; //the time it takes in microseconds
default:
throw new IllegalStateException("Unknown scenario");
}
}
/**{@inheritDoc}*/
@Override
public boolean rebuildDatabase(Map<String, Object> params) {
throw new UnsupportedOperationException("never needed, database rebuilt after every param change");
}
}You change the run method implementation to do some real work. Then add this test to a test suite and run it:
public class RunningDemoTest extends PerformanceTestSuite {
/**{@inheritDoc}*/
@Override
protected PerformanceTest[] getPerfTests() {
return new PerformanceTest[]{new PerformanceTestDemo()};
}
} This example code skeleton shows a custom class that implements the PerformanceTest class of the GraphAware library, which overrides the methods that need to be tweaked according to your requirement. The result is a total of 6 parameter permutations (the product of 2 scenarios and 3 cache types), each executed 100 times, as we have defined. When the test run process is complete, a file with the name test-short-name-***.txt (*** being the timestamp) appears in the project root directory. The file contains the runtimes of each test round for the parameter permutations. For example, the Test Long Name result file would contain something like this:
cache;scenario;times in microseconds... nocache;FIRST_SCENARIO;15;15;15;15;15;15;15;... nocache;OTHER_SCENARIO;15;15;15;15;15;15;15;... lowcache;FIRST_SCENARIO;15;15;15;15;15;15;15;... lowcache;OTHER_SCENARIO;15;15;15;15;15;15;15;... highcache;FIRST_SCENARIO;15;15;15;15;15;15;15;... highcache;OTHER_SCENARIO;15;15;15;15;15;15;15;...
It is also worth noting that Facebook has open sourced a benchmarking tool for social graph databases called LinkBench, which makes it possible for the Neo4j community to compare their performance metrics with a real and large dataset. You can check out the details of this system at https://github.com/graphaware/linkbench-neo4j.