Zach Blume

Building an async coding agent you talk to through Slack & GitHub

This week I built a single-threaded agent that consumes a Slack channel, GitHub events, and has a library/document API to manage a design doc's revisions. I dubbed this kind of agent a "captain". It chats with you back and forth in the Slack channel until you approve its design doc, and then it puts up PRs under a bot GitHub identity. When you approve its PRs, it ships them. If you request changes, it makes fixes and re-assigns you. The big deal is that... it kinda just works.

Why build this?

I started building this when Conductor said they were taking their Conductor Cloud offline after they were acquired by Meta (opens in a new tab). I tried a lot of the competitors, and they weren't as good. Capy.ai (opens in a new tab) gave me some of the inspiration for the captain name, but doesn't have this kind of design doc and Slack integration.

The way I used to work with agents was: one task becomes one chat which ends up as one PR. Then, I started re-using chats for many PRs, as "project chats". This was a big improvement because autocompaction has gotten so good that the agent develops a long lived knowledge of the project and can make small decisions much better.

A captain is the next logical leap after that. The agent manages its own project chat. I even added to its system prompt that phrase "No person is watching this chat", so you just communicate with it through async tools like Slack and GitHub. This is a new-ish model where the agent's chat or inner thinking is really an implementation detail and we move to human-like interaction tools.

What is a captain?

A captain is:

  • one long-running agent per project
  • a row in a database table to manage its state
  • GitHub events and Slack messages, which wake it up too
  • a cron job that wakes it up on a timer in case it didn't complete all its work per incoming events
  • a design doc it tries to obey with version control
  • a long system prompt explaining all this to it

The first version was about a thousand lines of code in the tool I already use to run coding agents on my side project. The screenshot up top is a wake up event in the chat, where it's being told that its one open PR is failing CI, and to the right you can see all the listed PRs it's merged.

How does the design doc work?

When you turn a captain on, the first thing it does is ask what the goal of the project is. Then it writes the doc: goal, milestones, what's out of scope, and the order it'll deliver things in. Once you say that you approve, it considers that revision of the design doc approved.

After that it basically only builds what the approved revision covers. If it hits a decision the doc doesn't make, the system prompt instructs it not to guess. It asks in Slack and goes off to work on something else until you answer.

Anything that changes the goal, what the user sees, the data model, or security has to go into a new revision and get approved first, whereas smaller stuff it just does. For example I told it in Slack to change something, and it decided that was new behavior, wrote it into the doc as a new milestone, and asked me to approve it.

Who merges?

The first version actually merged its own PRs as long as they matched the approved doc. I changed that a day later because I wanted a more dev-like flow. Now it uses a bot identity of GitHub,you can't review your own PRs, heh waits for CI, requests my review, and works on something else until I get to it.

Wrapping up

I've been using a captain to build the captain itself, and that project is about thirty PRs in.

A lot of those are small fixes, like getting it to stop waking up over a PR that hadn't changed, or to back off when its sandbox won't boot. It's only a few days old though. I'm still the bottleneck on reviews, and obviously if the doc is wrong it builds the wrong thing.

I'm pretty optimistic that this becomes my new major flow on side projects.

Past Writing