In May, Ripple configured its validators to vote in favor of activating the XLS-20 amendment on the XRP Ledger (XRPL). In reaching that decision—and building on our ongoing commitment to the XRPL’s health and performance—we conducted extensive testing to determine the XRPL’s capacity to support the extra transaction load brought about by native, on-ledger NFTs at scale.

This post outlines the results of that performance testing supporting the 1.9.1 release of rippled, the reference server implementation for the XRP Ledger. Performed in a distinct environment completely separate from any other XRP Ledger network, our testing sought to determine the peak throughput sustainable by the XRPL against particular workloads, such as native NFT transactions.

The Test Environment

The test environment consisted of 9 servers in a single Amazon Web Services region. Five of these servers were configured as validators, and the remaining 4 were configured to receive traffic from a load-generating benchmarking program located on a separate host. The hardware type for each host was EC2 type z1d.2xlarge, which has specifications roughly in line with those recommended for production: 8 CPU cores, 64GB RAM, an 8GB root volume and 300GB of ephemeral storage. The only significant deviation is the network interface, which is 10Gbps instead of the recommend 1Gbps.

The rippled configuration on each host had the following baseline characteristics (except where noted for specific tests):

• Data and logs stored on ephemeral storage
• The [node_size] is set to huge
• The primary database type is NuDB
• The NonFungibleTokensV1 feature, which is the amendment that adds XLS-20 support is enabled.
• Fee escalation is minimized by using the following configuration file stanza:

[transaction_queue]:
minimum_txn_in_ledger = 4294967295
target_txn_in_ledger = 4294967295
normal_consensus_increase_percent = 4294967295

The test framework is a custom set of tools able to submit transactions of various types at configurable rates. Each test begins with an initial ramp-up period which settles at a fixed initial rate of submission. That fixed rate is maintained for a configurable period of time and then increased by a configurable amount—this cycle continues until a maximum configurable rate is reached. When the test finishes, a report is generated which measures the peak sustained transaction commit rate. This is the peak per-minute value that was sustained for at least 20 minutes during the test. This value can then be divided by 60 for a per-second rate.

The ledger data for each test is based upon a snapshot of the livenet ledger number 70,637,016. This ledger was created on March 29, 2022. The reason for basing tests on this ledger is to simulate the workload on a data set that is similar to the live environment. For each type of workload, additional accounts are created. The test framework randomly chooses sender and receivers among these additional accounts. The types of workload tested were:

• XRP payments
• NFT minting
• NFT offer + buy or sell
• Mixed XRP payments and NFT offer + buy or sell

Test Results

The following tests were based on rippled version 1.9.1-b1, with the initial test measuring sustained throughput for XRP payments. Doing this required 20,000 accounts be created—in addition to those already existing from the snapshot of ledger 70,637,016. A random sender and receiver were chosen from these 20,000 accounts for each transaction submitted during the test. The peak sustained throughput was 2,199 transactions per second.

The next test was configured identically except for the node_database type—which was RocksDB contrasted with NuDB in the previous test. Peak sustained throughput was 1,940 transactions per second. All remaining tests used NuDB.

The next test measured NFT throughput. One million accounts were created in addition to those in the snapshot of ledger 70,637,016. Each account minted 20 NFTs, meaning a total of 20,000,000 NFTs on the XRP Ledger. The test randomly selected an NFT and created an offer against it. The offer was then accepted, with each offer and acceptance constituting a single transaction. To clarify, each NFT offer and acceptance resulted in a pair of transactions. The maximum sustained throughput rate for this workload was 751 per second.

The next test measured a mixture of XRP payments as well as NFT offers and acceptances. The precise mixture was 2 XRP payments for each NFT offer and accept transaction pair. The throughput for this test was 1,064 per second.

The final test—around NFT minting—was based on the livenet ledger 70,637,016 snapshot for which 20,000 additional test accounts were created. Each transaction randomly selected a test account and minted an NFT on it. The maximum sustained throughput of NFTs minted this was 384 per second.

Conclusions

The aforementioned tests are based on synthetic workloads in an ideal environment. Factors that likely affect performance, but which were not tested this way, arise from the live network having hundreds of nodes distributed across the world running different types of hardware. By comparison, the test only had 9 rippled nodes (5 validators) operating in a single site. In addition, there is a continuous flow of existing transactions not tested above—particularly trustline creation and order book manipulation.

Despite the effort to minimize the overhead of XLS-20 based NFTs, NFT transactions are more computationally intensive to process compared to simple XRP payments. This was expected, primarily because more ledger objects are manipulated when processing NFT transactions as opposed to payments, which only manipulate two objects: the accounts of the sender and the recipient.

Database Backend Recommendations

The test results emphasize a need for high-performance disk I/O and strongly suggest that node operators should consider switching away from RocksDB in favor of NuDB.

While previous testing suggested that RocksDB performed better in nodes configured to maintain small amounts of ledger history, that performance advantage is no longer present. The data suggests that NuDB—coupled with a high-performance solid-state storage—is the better solution for all node types, whether full-history or not. In particular, during these NFT tests, NuDB exhibited not only better performance but a lower memory footprint than RocksDB did.

To learn more about XLS-20, follow RippleX on Dev.To or Twitter.