Last Friday, after over two years in the making, XLS-30, the Automated Market Maker (AMM) amendment, went live on XRP Ledger mainnet. The native feature was designed to provide on-chain liquidity and trading capabilities for DeFi developers and users.

Despite the multiple rounds of rigorous testing and audits leading up to the mainnet launch, on Saturday, Tequ and Orchestra Finance, members of the XRPL community, discovered a bug affecting the AMM functionality. Over the past few days, the bug prompted swift action from developers and the RippleX team to ensure the security and efficiency of the XRPL AMM feature. Following thorough testing, the AMM bug fix was released Tuesday evening, and is now available under the 2.1.1 release of rippled.

In this update, we want to provide transparency regarding the recent events surrounding the AMM bug, discuss the work done to address it, and outline the path forward.

What happened
Saturday morning, March 23, Ripple received a report of an inconsistency in an AMM liquidity pool. Alongside community members Orchestra Finance and Tequ, analysis was conducted on the corresponding transactions and an error was identified in how the payment engine interacts with order books and AMM pools. In rare cases, complex payment path scenarios draw an incorrect amount of liquidity from an AMM pool.

This issue was not related or caused by the single-sided deposit feature, which is an intended functionality of the AMM, as it offers a more streamlined user experience, and could result in price impacts when pools have less liquidity. We acknowledge the concerns raised by some users regarding the challenges associated with single-sided deposits. While the feature is functioning as designed, it’s important to address these concerns. As such, we and others will explore and propose potential future improvements for the community to consider to enhance the overall user experience.

Given that this AMM bug fix requires an amendment voting process to correct, the issue remains present on the network in the interim period. Although this appears to be a rare occurrence, out of an abundance of caution, it’s best for users to limit their AMM usage until an amendment addressing the bug is passed by the validators. To prevent exploitation, the in-depth details about the issue are being withheld at this time but AMM UIs have been advised to close deposits. However, after the fix has been activated post amendment voting process, more information will be shared.

It would be remiss not to mention the quick response from many community participants – including Sologenic, Anodos Finance, Orchestra Finance, Dex Finance Pro, XPMarket, and others – in acting quickly to close deposit functionality, as well as many individuals who were vital in spreading the word on the issue and fix. The team will continue to stay in close contact with them as monitoring is still in place.

How the team responded
Once the bug was identified, two parallel workstreams kicked off; the first to assess the scope and impact on AMM pools, and the second to reproduce and fix the bug. Regarding scope and impact, the bug has not affected additional pools since the initial report. There is ongoing monitoring in place to detect if the issue were to re-occur.

The bug was successfully recreated in a unit test environment, facilitating the development of a fix alongside a new invariant check to enforce properties of AMM pools during transaction processing. Since the payment engine rules reside in the protocol layer, implementing the fix necessitates an amendment voting process which is reviewed by UNL validators at minimum of two weeks, requiring over 80% consensus. Extensive internal reviews, performance testing, and quality assurance measures have been conducted on the code changes. The code is viewable on GitHub, but excludes the unit tests to minimize the risk of retriggering the bug on the network. These tests will be incorporated into a subsequent release once the amendment activates.

What is next
The top priorities these last few days have been to (1) mitigate the impact of the bug on the network for XRPL users, and (2) prepare and thoroughly test a fix – thus intentionally limiting the amount of details shared about the bug to minimize the risk of additional impacts while the fix is being released and voted on by the network. All validators are encouraged to commence the voting process to quickly address this issue. Note that the default vote in the source code has been set to “yes” given the importance of the fix, though as always, the amendment must hold over 80% support for two weeks to activate, so there is time for operators to review and change their support as they see fit.

Post-activation, more specific details of the bug, the fix, as well as additional follow-on actions will be shared via a root cause analysis of the incident. A key part of this will be addressing how Ripple and the larger XRPL community can work more closely together to build, test, deliver and improve the XRPL for all its users.

Lastly, a huge thank you to the community members who responsibly disclosed the bug, the projects who quickly responded to limit further deposits into AMMs, the devs who worked around the clock on the fix, and the broader community for the support and patience during this process. It’s been great to see the community involvement in initiating discussions on ways to improve the XRPL and we are on the front lines with you.