To ensure balances are up-to-date, FRAGMENT seamlessly scales to support high-throughput workloads like ledger migrations, marketplace payouts, and batch payment processing.

This is the second post in a three-part series about the FRAGMENT balance system. The first post describes how the data model allows for efficient updates and low-latency reads. This post describes the architecture of async balance updates.

FRAGMENT's balance system is event-driven. When a Ledger Entry is posted using the API, it's written to DynamoDB, pushed to a change data capture (CDC) stream, and replicated to an event bus. The balance system consumes these events to update async balances.

This is implemented using AWS DynamoDB Streams, EventBridge, and SQS.

To increase throughput, FRAGMENT calculates the latest balance of eventually consistent Ledger Accounts in batches.

If processed individually, each event introduces latency and the probability of conflicting with DynamoDB's concurrency control mechanisms.

With batching, the balance effects of multiple Ledger Entries can be coalesced into a single database update.

As described in the previous post, historical balances are stored as balance deltas at the hour, day, month, and year granularities.

To increase throughput while maintaining deltas at different granularities, the balance deltas are calculated from most to least granular: hours, then days, months, and years.

These additional layers of coalescing reduce the total number of writes, increasing write throughput.

For asynchronous systems to work, they need to update quickly. In production, the async balance system has a p95 staleness of 10 seconds.

If building mission-critical financial infrastructure powering the next generation of financial products excites you, we're hiring.