The following things were contributing factors:
- I was calling the API quickly adding new transactions as fast as it would respond to queries.
- Now and again a transaction would appear to go wrong (a transaction would remain unprocessed for a while) having talked to the microsoft team, this is likely because it's nonce was out of order with previous transactions.
- Once this happened, the transactions just piled up behind it. Every time a new transaction was added whilst this was unprocessed I think it probably still used the wrong nonce to submit the next transaction
- Even once transactions were cleared, the load on VMSS stayed high.
Things that helped to work around the problem:
- I re-deployed with the newest version of Quorum
- I updated my script which loaded data such that after submitting a new contract instance, it would wait until that contract was mined into a block. This slowed the loading time for 8000 contracts from under an hour to approximately 12 hours however it now works reliably.
Source: MSDN