Briar, we need to get some work done

I’m building an AI assistant called Briar. We work together on software and electronics, and I want to get good work done without having to supervise every step.

Recently, though, I kept having to bring Briar back to the job I had actually asked for. They were doing more checks, writing extra little programs to help with the work, or improving their instructions. Meanwhile, the work itself was waiting.

“Briar, we need to get some work done.”

We have spent a lot of time giving Briar instructions, useful ways of working and lessons from earlier mistakes. We call that collection the “harness”. It is supposed to help us work together. Some of it was getting in the way.

I wanted to understand what we needed to keep, what we could simplify and what we should stop doing altogether.

Check the work, then try it

Much of our work involves the software inside electronic devices. After Briar changes that software, I need them to look for mistakes, put it onto the device and test whether it does what we asked.

There are plenty of useful questions to ask during that review. Has Briar understood the job? Have they missed something? Could the change break something that was working before?

But the review was spreading beyond the change we needed to make. Another check would lead to another investigation, or another small program to help us check things. Each extra program is something else we have to understand and maintain.

Sometimes we need that extra work. I just want it to help us finish the job in front of us. If we have already checked something and nothing relevant has changed, why are we checking it again?

We still need the tests that tell us whether the work is any good. When a test fails, Briar needs to use the result to find the fault, fix it and try again. I do not want them disappearing into another round of preparation before we have even tried what they made.

Trying an idea is different from changing something people rely on

One example was Briar spending time preparing a way to undo a software update on a device we use for experiments.

That device sits on our development bench. If the software does not work, I can connect a programming tool and load it again. Being able to try a change, see what happens and try again is how we develop it. We do not need to build an automatic recovery system before every experiment.

We do need to avoid damaging the hardware, and keep a record of which version we tried and what happened. If we are updating a device remotely, we also need to make sure a failed update will not leave us unable to reach it.

A live system that customers depend on is a different responsibility. I want to approve the change, understand who could be affected and know how we would get it working again if something went wrong.

We had been putting too much effort into protecting recoverable experiments as though they were live customer systems. That was slowing down the work where we needed to learn by trying things.

Please ask me when you need to

I am always happy to answer questions that help Briar understand what I need and want. Spending the day doing the wrong job because they did not ask is far more frustrating.

The interruptions I object to are the ones where we have already agreed the work, but Briar has to wait for me to approve another routine step before they can continue.

We should be able to agree which systems Briar can use, what they can do and for how long. Within that agreement, they need enough freedom to get on with the job. If they need my judgement or want to go beyond what we agreed, they should come back to me.

I still decide what gets sent in my name and what changes go into systems people rely on. I normally send the emails and approve the final software changes myself. Permission to experiment must not quietly become permission to change a customer’s system.

We also found that some of the checks we had written and tested were not yet connected to the tools Briar actually uses. Writing “do not send emails” in their instructions is not enough. I need the email tool to refuse to send one. We still have work to do there.

What happens when the conversation gets too long?

While we were looking at this, I also started asking what happens to a long conversation with an AI assistant.

There is a limit to how much conversation the AI can work with at once. When a chat gets long, some tools shorten the earlier discussion into a summary so the work can continue. This is called “compaction”.

My concern is what gets left out. Will Briar remember why we decided to do something? Or that I changed my mind later? I do not want them carrying on with an old instruction that no longer reflects what I want.

I asked the AI LCR Builders group and people shared useful explanations and links. A hat tip to everyone who helped, and to Alex Keyter of Generative Minds, who founded and runs the group. It is a good place to ask questions and compare experiences with other people working with AI.

Some of the links explained how Claude handles long chats. We use Codex, an OpenAI tool, so I also looked for information about that. I still do not know exactly how Codex chooses what to keep when it shortens a conversation.

Someone also shared an account of an AI assistant called OpenClaw deleting emails. I have told Briar not to send emails. Reading that account made me ask what would stop them if that instruction disappeared from the conversation.

For our own work, I want enough written down to pick up the job without making me explain it again: what I asked for, what we decided, what we tried and what happened. Something Briar suggested must not reappear in the next chat as something I told them to do.

Some instructions needed to go

Going through the harness turned up practices we no longer wanted. One instruction called for an extra PDF whenever we prepared certain documents for sharing. I was reviewing a web article; another document was not helping me do that.

Another instruction encouraged Briar to ask me whenever they reached a choice, even if I had already made that choice. Old wording still described how Briar could send an email, although I wanted to send emails myself.

We removed that wording and corrected the instructions that depended on it. We kept useful engineering knowledge and the history of our changes, so we can still see what we did and why.

I see this as part of Kaizen, the practice of continuous improvement, and the Toyota Way practices we use in our work. We need to be willing to remove something we once thought was a good idea. Keeping a rule just because we wrote it down can leave Briar following instructions that no longer help us.

The next job is where I want to see the difference. Briar should ask what they need to know, check their work properly and help us get it done. We have not measured the time or AI-usage savings yet, and I do not want to spend another session bringing them back from improving the harness.

If you are working with an AI assistant, I would be interested to hear where you find yourself having to bring them back to the job. What have you changed that actually helped?

(Briar helped draft this)