Leveraging Idempotence to avoid double payment in a payment system.
Users expect to use your applications in some intuitive ways. For example, they expected to click a button again if it didn’t produce the expected effect when they clicked it previously.
But how can they know that it did not cause the effect they expected, especially if a network is involved?
How can they know that they didn’t debit their account already? As long as you’ve not told them that the transaction succeeded, they expect a retry to be safe. If that’s not the case, someone won’t be happy: it may be you, it may be the user or the customer support personnel that’ll wake up to a nicely worded email.
Anyway, idempotence can save everyone the trouble if you think about it just a little. We’ll see about that in this article; stay tuned.
I’ll talk about something you sadly won’t find in my job description: User Experience (UX). Sadly, because even when I bill myself as a back-end developer, my work contributes to user experiences in several ways.
UX permeates your product beyond its interfaces; every system’s behavior can either make or mar someone’s experience. That person may be a customer, a customer support personnel, a developer, an operator, or even a shareholder.
When we use a piece of software, we expect at least two things from it.
- It does what we expect it to do safely.
- It doesn’t do what we don’t expect it to do.
- It’s intuitive to use.
Yes, that’s an off-by-one error. Make that in your code and see how it affects your UX 😉
You provide various user experiences depending on how well your software meets these expectations. You can make your users feel powerful, understood, productive, intelligent, angry, frustrated, stupid, etc.
We can appreciate the efforts banks make to secure our money. They process lots of transactions daily, and they usually account for every dime. Usually, but we can also agree that they typically scrape at the bottom of user experiences. That one mistake you made using their mobile app can cost you three hours waiting in line for a resolution from their hardworking customer service representatives.
Yes, you made a mistake; do you feel stupid now?
“Fine, I’ll try again.”
Sometime in August (or was it September?) 2020, I tried buying a data plan from an ISP in Nigeria using my bank’s mobile app. It was a cheap 1 GB night-only data plan, perfect for updating all the software that I don’t care about much.
I tried the first time, and it went well. I wanted more data (YouTube hits different at night, too), so I repeatedly tried thrice. Each time I got some cryptic error message that seemed to mean that a network error had occurred.
Then the debit alerts started coming in. Some credit alerts interspersed them. The system was reverting failed transactions; what an intelligent system! Except that they came in out-of-order, and my final balance did not reflect the sequence of failed and successful transactions made.
I looked through the SMS alerts to verify that I saw the correct information. I counted each credit, noted and compared the transaction reference with the others to know whether it was compensating for the debits, counted the digits of my NUBAN (Nigerian Uniform Bank Account Number) to be sure I still know how to count. I noticed that some amounts were missing!
I quickly contacted customer support and even provided a screen recording explaining what happened and highlighting the discrepancies in the transaction alerts I received.
The next day customer support came through. I explained myself the second time, as usual (do they read the messages before reaching out?). They assured me that they would resolve the issue soon. A few hours later, I got a canned response explaining that, in my opinion, that the system has already resolved the issue.
No way. I felt angry, frustrated, and stupid: do they take me for a joke? I couldn’t continue chasing customer support tickets, and maybe if I had stopped at the first error, if I didn’t try again, this wouldn’t have happened. I caused it.
Think of how many times your users say that. “Fine, I’ll just try again.” What happens when they do? Do they get multiple debts? Does your system launch a nuke each time? Do they know that something probably happened? You encourage them to try again, so you should make retries safe.
Let’s see how to do it.
Leveraging Idempotence avoid double payment
In software systems, we say that an action is idempotent if doing it multiple times has the same effect as doing it only once.
If tapping that big, red retry button (it may not read “retry,” but it says “retry”) on my bank’s mobile app did not stream multiple SMS alerts to my phone, and it would be idempotent.
Idempotence is vital for safe retries.
Here’s what the user interface looked like when I got the error message.
This particular screen does not go away after clicking “OK”; it helpfully stays right there so that you can tap that “Confirm” button to retry. Imagine what it would feel like starting back at an earlier screen to retry a transaction that failed.
They were very thoughtful about this design. Look how nice; my bank even gave us puzzle lovers a mystery code, EXTSOC36.
Let’s improve it to support idempotence.
Request IDs
A simple fix that can work here is for the mobile app to generate a unique request ID for each transaction and pass it on to the server. The server can decide whether it has processed this transaction before by comparing the request ID with those it has already seen. If it has processed it before, it sends the saved response from the previous run to the user. Otherwise, it processes it and saves the response just in case the user asks again.
This simple trick can avoid double payment and build trust with your users.
The request ID format can be $USER_ID-APP_ID-TIMESTAMP, making it unique for each user from whatever device they use.
The mobile app would clear the screen and the request ID after a successful transaction. If the transaction failed from the app’s point of view, it simply retries with the same request ID.
I don’t know what goes on behind the scenes of my bank’s mobile banking app, so this may not work for them.
But you get the point.
But I’m Not Handling Payments
Right? You’re not handling users’ money, and you don’t have this problem. I’ll file that under “popular last words.”
Idempotency is vital in the payment scenario I explained above, and elsewhere too.
In early 2020 we worked on a project similar to an online marketplace. Our first implementation for creating a marketplace listing created duplicate listings if the user retried after a network failure.
To remove this annoyance, we let the client devices generate the listing IDs (UUIDs are good enough to avoid collisions), and we changed the HTTP request method to PUT to indicate that it is idempotent.
That solved the problem. Also, because the client already has the full copy of the data, including the ID, they can use the local copy of the data they just sent to the server rather than fetching it from the server.
TL;DR
Idempotence is necessary for safe retries and to avoid double payment. If we can retry an action, then it must be idempotent. Also, both the client applications and the server (if there’s any) must ensure that the idempotency guarantee holds. In this way, both sides provide a better user experience.
Your users should not feel stupid for retrying when your system tells them that it’s safe to retry.
Frustrate the users, and they’ll probably try again. Annoy the users, and the users may get over it. Make them feel stupid, and you’ll trigger all three (frustration, anger, and shame) at once.
Don’t do that. Always consider making user actions safe to retry and to avoid double payment if they can fail.
Further reading
- Stripe deals with payments, so they know this problem well. Read Designing robust and predictable APIs with idempotency.
- In Avoiding Double Payments in a Distributed Payments System, Jon Chew offers an exciting perspective on idempotency, covering Airbnb’s approach from the framework level to the database. I recommend this one strongly.
