Payments Performance Test Harness.
This is a test harness designed to measure the performance of payments between peers on a blockchain network. This performance is measured against the Emerald benchmark which is specified here:
The test harness should be able to measure the performance of a simple payments network on a number of different blockchain platforms.
Choosing a payments use case enables to not only compare the performance of a variety of blockchain plaforms but also compare against existing payment platforms and networks.
The simple payment network consists of the following:
- A defined number of blockchain nodes/peers.
- Each node has a number of ‘accounts’ in which they can store tokens.
- Each node issues and sends tokens from its accounts to an account on a random peer/node other than itself.
In order for the test harness to measure the performance of a blockchain payents platform. The platform must support the following use cases:
- Create several accounts that can store tokens
- Issue tokens to specific accounts
- Transfer tokens from account to account between nodes.
The third use case is what is measured by this benchmark.
The overall scope is to have a ready made test harness which blockchain developers can use to test and tune their own payment solutions.
The performance benchmark test sends batches of payments to random nodes in the payments network. The benchmark aims to find a stable latency between batches and constantly adjusts a target value to gradually find the optimum latency. This adjustment is ‘smoothed’ by a latency adjustment parameter so that we don’t overload the nodes with too many payments too soon - think of it like turning up your central heating and waiting for the thermostat to click off at the target temperature. The ‘batching’ is designed so that we can measure parallel throughput as well as latency. For this reason we set up a number of test accounts and load each account with 1 million tokens. We then send a payment from each of the test accounts in each batch. We wait for the latency time to elapse before sending the next batch. An instance of the test harness runs against each node in the network and measures the end to end latency by recording the submission time and listening to received payments - this makes it quite different from a jmeter test where there is only usually one instance of the test harness.
Mark Wagner: chair of performance and scale WG & TSC member