Each agent session in AI SRE has one owner. A Redis lock with a 30-second expiry makes sure there is never a second one. That one rule gives us ordered messages, recovery after a crash, and idle sessions that cost almost nothing. The whole runtime needs only Redis and MySQL.
This post shows how it works, what went wrong along the way, and the workloads it doesn't fit.
Part 1 of Inside Flashduty AI SRE, a series about what we learned running AI SRE in production.
What makes agent sessions hard
A web request finishes in tens of milliseconds. If it fails, the client retries. An agent investigation can run for minutes. It makes a dozen model calls that cost real money, it may run commands against real systems, and the user watches every line arrive.
AI SRE runs around the clock on many servers. During those minutes, three things happen often enough that we had to design for them.
- The user sends a follow-up, like "check the slow queries too." The model is waiting on a tool result. If we insert the new message now, the tool-call sequence breaks and the turn fails. We can't drop the message either. It has to wait until the turn ends.
- A server restarts. Every release rolls instances out. Say an investigation is four minutes in and its process gets a shutdown signal. If nobody takes over, the user sees a spinner forever and we waste the tokens.
- A request lands on any instance. Without coordination, two servers can both pick up a session. Then the investigation runs twice, the agent creates the ticket twice, and two outputs interleave in one history. The agent later reads text it never wrote as its own context.
The design
One owner per session
We call the owner a butler. Internally it's an actor: a piece of code that serves one session. Before it does any work, it takes a Redis lock.
- It creates the lock only if none exists, with a 30-second expiry (
SET NX EX). The winner owns the session. Everyone else exits. - It renews the lock every 10 seconds while working.
- After five idle minutes it releases the lock and exits.
If the owner's process dies, the lock expires within 30 seconds and another process takes over. We don't need leader election or a human.
Senders don't wait
Many things send messages to a session: the web UI, a sub-agent reporting back, an automation firing. Each one does two things. It appends the message to the session's Redis list, then publishes a Pub/Sub notification. Then it returns.
The butler reads one message at a time, processes it, and removes it. When the list is empty, it waits for the next notification.
So any server can send, at any rate. Redis decides the order. Only the butler processes.
Idle sessions hold nothing
This decision has the biggest effect on cost. Most sessions sit idle most of the time. A user asks a question and comes back half an hour later. If each idle butler blocked on a Redis read, every idle session would hold a connection. Ten thousand open sessions would mean ten thousand connections.
So an idle butler holds no Redis connection. It waits in process memory until a notification wakes it. Because a notification can go missing, it also checks the list every five seconds. Nobody notices five seconds on a reply they send minutes later. Now the connection count tracks busy sessions, not open ones.

Several senders feed one queue. One butler processes it in order. The ledger keeps events; the broadcast delivers live updates.
When the butler crashes mid-message
There's a gap in the loop above. The butler takes a message, starts work, and crashes. The message has already left the list.
So the butler doesn't take a message off the list. It moves it to a second list, the processing list (LMOVE). It deletes the message from there only after the work succeeds.
A new butler checks the processing list first. If the last owner left anything behind, it moves it back to the head of the pending list (RPOPLPUSH) in its original order. No message goes missing or out of order. The cost is that a message can run twice, so every downstream step has to be safe to repeat.
One more case. The old butler didn't crash. It stalled on a slow step and missed its renewals. The lock expired and a new butler took over. If the old one wakes up and clears the processing list, it deletes the message the new butler was working on.
The fix is an owner token. Each butler gets one when it takes over and checks it before clearing the list. If the token isn't its own anymore, it leaves the list alone. An expired lock tells you the owner stopped renewing. It doesn't tell you the owner stopped running.
Any queue tutorial covers this pattern. We still spell it out because one lost message here is minutes of investigation and a real token bill.
What we left out
We don't use Redis Streams or Kafka. MySQL keeps the full record of every session: each message, tool call, and model reply. We treat everything in Redis as a cache we can rebuild. If we lose the queue, the messages are still in the database. If a live notification goes missing, the client reconnects and catches up from its cursor. A second streaming store would hold the same data twice, and we'd spend our time reconciling the copies.
We don't migrate sessions or balance load. Whoever wins the lock owns the session. A session spends most of its time waiting on model APIs, not on our CPU, so there's nothing worth moving.
What's left depends on Redis and MySQL. An engineer who knows ordinary backend services can read the whole thing in an afternoon.
Where it doesn't fit
Each session processes messages one at a time. One butler's capacity caps a session's throughput. If a single session needs hundreds of messages per second, use something else.
Agent workloads look the other way. There are many sessions. Each has few messages, and each message takes minutes. Butlers are idle most of the time, so cost comes from what idle butlers hold. Fix that, and two ordinary pieces of infrastructure can run a multi-tenant agent platform.
Ordering is half the problem. The other half is Stop. A user can press it at any time, even before the turn starts. During a rollout, the lock and the stop request race each other. We had a real incident: a user pressed Stop, and the agent ran the whole turn anyway. A follow-up post covers how we fixed it with a tombstone.


