Blog

|

engineering

Batching On-chain Sends

Quality of Service (QoS), Efficiency, and Security

Tim Normark

Aug 01, 2024

Strike has a mission to onboard the world to better money. The services we have built are designed for fast, low-cost, and seamless payments over the Bitcoin network. One of the core techniques we leverage to achieve this is batching bitcoin on-chain payments. 

In this article, we'll explore how our on-chain batching system operates, the nuances that make it effective, and some of the technical challenges we've overcome to ensure our users receive top-tier service.

Tiered On-chain Payment Services

At Strike, we offer a tiered service for sending on-chain payments:

  • Flexible (Free) Tier: For those willing to wait 12-48 hours

  • Standard Tier: For faster, but affordable sends

  • Priority Tier: For transactions that cannot wait, targeting confirmation in the next block

This tiered approach allows us to cater to different user needs and preferences, balancing cost and speed effectively. Offering both unlimited free on-chain sends and next-block transactions to all our users poses a bevy of technical challenges. We need a system that can scale to a very large number of users, perform on-chain payments at enterprise-level volumes, keep fee costs affordable, and cleverly utilize the liquidity in our hot wallets.

The Efficiency of Batching

Batching multiple payments into a single large transaction significantly reduces our on-chain footprint and mitigates some of our scaling challenges. This approach benefits not only our users but also the Bitcoin community by minimizing the data usage of our transactions, thereby reducing network congestion and fees. It also means that one of our UTXOs (unspent transaction outputs, sometimes referred to as “coins”) can be used to facilitate several on-chain payments for our users, which is crucial for scaling our operations.

Cost Savings through Batching

By batching payments, we can offer cheaper on-chain sends compared to what users would pay if they were to send transactions individually. On the Bitcoin network, you pay for the data size of your transaction, rather than its financial value. The savings of batching stem from the reduced byte size of batch transactions compared to publishing multiple individual transactions. All on-chain transactions contain both fixed and variable data. The fixed data includes information that is standard for all transactions, whereas the variable data includes the transaction inputs (coins to be spent in the transaction) and outputs (destination of the coins), which can vary in number and size. By combining multiple individual transactions into one large batch transaction, all outputs can share the same fixed data and input coins, thereby amortizing their costs.

Upgrading Our Batching System

Strike has been batching on-chain payments on behalf of our users for quite a while, and it has allowed us the ability to offer on-chain sends at very competitive prices (including a free tier!). However, recently we made a huge improvement to our batching system, which we think will reinforce our position as a leader in Bitcoin. In sharing our newest batching strategy with you, we’d like to set the expectations of what you can expect when you use Strike to make an on-chain payment as well as help other builders by sharing some lessons we’ve learned while iterating on our on-chain payment service.

Append Outputs to Unconfirmed Transactions

The most recent upgrade to our batching system provides the ability to append more outputs (receivers / destinations) to existing unconfirmed batch transactions when feasible. This is done by leveraging a Bitcoin-native feature called Replace-By-Fee (RBF). RBF allows us to “replace” unconfirmed transactions by raising the fee they pay. Importantly, the outputs don’t have to be the same as the original transaction, enabling us to add more outputs to existing, unconfirmed transactions when needed.

Here are some links for those interested in learning more about RBF:

This upgrade allows us to handle a surge in "Priority send" requests (our next block-targeting tier described above) without constantly creating new batch transactions. Instead, we add more outputs to our existing batch transactions that have been broadcast but are still unconfirmed. Here’s why this is a game-changer:

  1. Reduced On-chain Footprint: By minimizing the number of new transactions created, we further reduce congestion on the Bitcoin network.

  2. Fee Cost Savings: Fewer batch transactions for the same number of payments generally means even more savings on Bitcoin on-chain fees - we generally see between a 50-30% cost savings per transaction.

  3. Enhanced Quality of Service (QoS): Existing, unconfirmed batches get fee-bumped to the current market rate as part of the RBF operation. This ensures that our users will experience better service quality as their transactions are more likely to be included in the next block, regardless of fee spikes on the network.

Example of an onchain batch, using RBF to append additional outputs until it gets confirmed

Technical Challenges and Solutions

Implementing this sophisticated batching system required overcoming several technical hurdles:

Prevent Double Send!

Using RBF to add extra outputs requires careful handling to avoid double-sending issues and ensuring that all extra appended outputs are correctly accounted for. There are several reasons double sends could become an issue when you start adding outputs to transactions through RBF. This isn’t because of any problems in the Bitcoin protocol, but because there are some conceptual pitfalls that can arise. One of the main points is that you need to handle the case where miners confirm a non-latest version of your batch transaction. Any candidates you appended after that version need to be rolled over to a new batch. But if you make a mistake here, you might end up double sending!

Guaranteeing Transaction Tracking

Ensuring we don’t double send involves meticulously keeping track of which transactions have been published and monitoring which version of a batch transaction gets confirmed. This careful tracking allows us to roll over candidates from unconfirmed versions to new batches accurately. 

But what if your wallet software crashes exactly at the time of publishing a new RBF transaction to the Bitcoin network? How will you know if the transaction was published? 

Some wallet software will give you the new TxId as a response upon successful publishing of a new transaction. This approach is not good enough here, as it could result in us never learning the TxId of a transaction that might have been published and confirmed on-chain. We might fail to receive the response containing the new TxId because of e.g. network level errors, software crashes, etc, even though a new transaction was in fact broadcast. Since all our transaction versions contain a different set of outputs, it's crucial that we know which version gets confirmed so that we can decide if any appended outputs did NOT get confirmed and therefore need to be rolled over to a new batch.
This general problem can be solved by a 2 step approach:

  1. Build your transaction and store it in your DB

  2. If step 1 succeeded, try to broadcast the transaction to the Bitcoin network

With this approach, step 2 can safely be retried if you encounter any problems. You will also have the ability to monitor the Bitcoin network for the TxId, to help determine if the transaction got published or not. The way Bitcoin transactions work is that building the transaction also means building the TxId. This is a completely offline process which can be done by your software without communication with the rest of the Bitcoin network.

Chain Split & Reorganization

Another case to look out for is when versions of your batch transaction are involved in Bitcoin Chain Splits (temporary blockchain forks) and potentially the subsequent reorganization of the blockchain. Without going into the details, this could mean that a version of your batch transaction initially gets confirmed on-chain, but is shortly thereafter un-done and superseded by another version of your batch transaction!

Since all versions of our batch transactions contain a different set of outputs, and we make decisions on which outputs to roll over to a new batch based on which version got confirmed, this phenomenon also has the potential to make us double send. Chain Splits & Reorganizations are an important part of the blockchain's operation to ensure consensus, so our systems need to be able to handle this. A simple solution is to just do the same as one normally does for on-chain receives; wait until the transaction has X (more than one!) confirmations on-chain before considering it as final. In Bitcoin, you will occasionally see chain splits with the depth of one block, but if your transaction was confirmed 6 blocks ago it's generally considered safe to count it as “final”.

Appropriate Fee Estimations

Using mempool-based fee estimations as opposed to blockchain-based estimations is crucial for QoS and cost efficiency. This approach ensures that transactions can get into the next block without overpaying. Our system continuously monitors our mempool to choose an appropriate fee rate for each batch transaction and each RBF.

Scaling On-chain Operations

Our enhanced batching system positions us to scale our on-chain operations to a level where we can truly grow to be a global company with millions of active users. Our system now allows us to handle huge increases in volume from our users without compromising on speed, cost or quality of service.

UTXOs: A Scarce Resource

One of the biggest challenges with scaling a custodial on-chain send service while still offering the “Priority send” option is the need to keep a stack of UTXOs which can be used as inputs in your transactions to facilitate on-chain payments. Imagine having to serve one “Priority send” request from your users every second, while there is a larger gap (e.g. 1 hour) between blocks getting mined on the Bitcoin network. If you need to instantly publish all these transactions, batching them wouldn’t be possible.

However, let's say we are clever and we add a slight delay of just 5 seconds between each of our batch transactions. This means we can batch 5 requests together every 5 seconds. So, during an hour where no block is mined we would need to publish 720 transactions to facilitate the requests of our users in this example. Since none of these transactions would get confirmed during that hour, the change outputs from these transactions would not be available for safe reuse, necessitating a stockpile of 720 UTXOs of appropriate sizes in our hot wallet. Without this, our service would simply halt until a new block is mined. This situation becomes even trickier during fee spikes, which might cause transactions to remain unconfirmed for an even longer period.

Moreover, maintaining UTXOs of an “appropriate size” adds another layer of complexity. For instance, if all send requests are for 1 BTC each - we would need to maintain 720 BTC in our hot wallet, broken down into UTXOs of 1 BTC each. This not only becomes complex to manage but also operationally inefficient and hard to scale. It’s of course impossible to 100% accurately predict what your users will do, so your stockpile of UTXOs needs to be even bigger than this to be ready for unforeseen spikes in demand. The foundational issue here is that we end up with many unconfirmed transactions in the mempool simultaneously, locking up numerous of our UTXOs.

Running the System on a Single UTXO?

Instead of publishing new batch transactions every 5 seconds as in the previous example, our new approach appends more outputs to our existing, unconfirmed transaction - if there are any. This provides an incredibly efficient solution to the problem mentioned above. Appending more outputs reuses the existing input UTXO of the unconfirmed transaction, provided the UTXO is large enough. This means that, in theory, our entire batching system could run on a single UTXO.

Each time one of our batch transactions gets confirmed on-chain, we receive the change output back as a new UTXO, which can then be used to fund the next batch. This approach essentially scales infinitely (or at least until the Bitcoin block size limit!), allowing us to handle a growing volume of on-chain send requests without being constrained by the number of UTXOs in our hot wallet. It also mitigates the risk associated with fee spikes, ensuring continuous operation even during network congestion.

Through this system we achieve an unprecedented level of scalability and reliability in our on-chain operations. This positions Strike to grow and serve a global user base with millions of active users, offering them the best possible service in terms of speed, cost, and reliability.

Superior QoS

Our commitment to providing our users with the highest quality service in the industry is evident in our approach to on-chain payment batching. Users can trust that their payments will be handled with the utmost care, efficiency, and security, whether they opt for a Flexible, Standard, or Priority send.

Fee spikes on the Bitcoin network are a common problem that plague the quality of other providers. It’s common to publish transactions with a high enough fee to target a confirmation in the next block only to then see the general fee market spike significantly. In these cases, published transactions will likely not get confirmed for several hours and could potentially end up stuck much longer or pruned. The Bitcoin protocol has solutions to let you “fee bump” your transaction to solve for this, but it's not ubiquitous to all apps and wallets, and not common for wallets to automatically fee bump your transaction in such cases without charging you more for it! Strike simplifies all of this for you, and provides flexibility for users with varying time preferences.

Conclusion

At Strike, we continually strive to innovate and improve our services. Our advanced batching system for on-chain sends is a testament to our dedication to efficiency and high-quality service. We aim to be the best at Bitcoin and that means providing our users with the best possible experience that Bitcoin payments can offer. We truly believe that our on-chain send experience is now well on its way to how on-chain payments will, and should work in the future. We will, however, always continue to iterate on our solutions to further enhance them and to show the world how much Bitcoin has to offer.

Feel free to reach out if you have any questions or need further details about our on-chain batching system and its benefits. We're always here to help and provide the best possible service to our users.

announcements

Strike infrastructure update

We now serve customers on our own infrastructure

07 Sep, 2023

© 2024 NMLS ID 1902919 (Zap Solutions, Inc.)

Get off zero.

Strike

BitcoinPaymentsSend GloballyBusinessAPI

Platform