Hey again 👋

This week I’m sharing one of my favorite recent videos on Sagas in distributed systems, a community comment that sparked a really thoughtful conversation, and a short excerpt from Chapter 2 of The Backend Lowdown, and a tool I’ve been using lately to level up the way I visualize technical concepts.

Let’s get into it 👇

From The Backend Lowdown: Adding Indexes Safely in Production

Adding an index can speed up queries - but if you do it the wrong way, it can lock tables, block writes, and bring production to a crawl.

This is one of those operations that seems harmless until it takes your app down.

The Problem: Locks and Downtime

By default, when you run:

CREATE INDEX ON orders (user_id);

PostgreSQL acquires a lock that blocks all writes to the orders table until the index is fully built. On large tables, that could be minutes (or hours) of downtime - even if the index is eventually helpful.

The Safer Way: Use CONCURRENTLY

To avoid blocking reads or writes, always use CONCURRENTLY in production:

CREATE INDEX CONCURRENTLY CONCURRENTLY_orders_user_id ON orders (user_id);

This tells Postgres to build the index in the background without locking the table for inserts, updates, or deletes.

CREATE INDEX CONCURRENTLY can't run inside a transaction block, and most frameworks (like Rails or Ecto) wrap migrations in one by default. To safely create an index concurrently, you'll need to:

• Disable the transaction (disable_ddl_transaction! in Rails)

• Use the appropriate syntax (algorithm: :concurrently in Rails)

Safely Replacing Indexes

If you're replacing an old index:

1. Create the new one concurrently

CREATE INDEX CONCURRENTLY idx_new ON orders (user_id, created_at);

2. Update your app to use it (if needed)

3. Drop the old one concurrently

DROP INDEX CONCURRENTLY idx_old;

This prevents any locking at every step - and lets you roll forward or back safely.

Weekly Video Highlight

In this week’s video, I break down how distributed systems recover from failure using the Saga pattern.

If you’ve ever wondered how a system undoes part of a multi-step process when one service fails halfway through - Sagas are the answer.

I explain how they work, why compensating actions aren’t just “reverse the last thing,” and where they fall short in the real world.

@stevencodes.swe

In a multi-step workflow, failures can’t be ignored. The saga pattern handles rollback with compensating actions - here’s how it works in ... See more

Community Corner

This question came up on the Saga video and it’s one that gets to the heart of why distributed systems are so tricky. In theory, checks should prevent these issues. But in practice, at scale, timing races and stale reads make perfect coordination impossible. That’s where Sagas shine: not in avoiding failure, but in helping you recover from it.

Dev Tool Highlight - Overleaf

Okay, not technically a dev tool - but I’ve been using Overleaf to generate LaTeX-based diagrams and formulas for recent videos, like the one I did on Count-Min Sketches. I really love the fact it has a visual editor as well, which made it easy to work with and preview what I was doing.

Sometimes code isn’t enough - especially when you’re explaining error bounds, tradeoffs, or memory guarantees. With Overleaf, I was able to show how the sketch’s dimensions impact accuracy and space, and it made the concept click visually in a way that words alone couldn’t.

If you’re explaining technical ideas, especially anything probabilistic or math-heavy, this is a great tool to level up your visuals.

That’s a wrap for this week. If something here made your day smoother, feel free to reply and tell me about it. And if you think a friend or teammate would enjoy this too, I’d be grateful if you shared it with them.

Until next time,
— Steven