As agentic coding tools became the default, the question changed. It used to be “which tool writes the best code.” Now it is “what should this tool be allowed to do in my repository.”
Comparing tools cannot answer that. The same tool carries a completely different accident profile depending on which account it runs as, which repository it touches, and what CI rules sit around it.
Split permissions into three layers
Sort what an agent does by how hard it is to undo, and the boundary becomes obvious.
| Layer | Action | Undo |
|---|---|---|
| Layer 1 | Reading and editing files, running local tests | Easy. Before a commit there is not even a trace |
| Layer 2 | Committing and pushing branches, opening PRs | Possible. The history remains |
| Layer 3 | Merging to the default branch, deploying, DB migrations, outbound API calls, publishing packages | Hard or impossible |
Leaving layers 1 and 2 open is better for productivity. Layer 3 is the problem. That is not something to guard with the tool’s confirmation dialog; the right move is to make sure the permission does not exist in the first place.
Separate the account and the token
The most common mistake is letting the agent use a person’s own credentials as they are. When that happens, nobody can tell who did what after an incident, and the permissions are as wide as a human’s.
The setup recommended in practice looks like this.
- Create a dedicated account or token for the agent. Keep it separate from human accounts
- Give it the narrowest repository scope. A token spanning several repositories multiplies the blast radius
- Turn on default-branch protection. No direct pushes, PR required, review approval required
- Keep deployment credentials inside CI only. In a local environment variable the agent can read them
- Turn on audit logging. Who changed what and when has to be recorded for anything to be traceable afterwards
Decide first what must never be read
An agent sweeps the repository to gather context, and files you would rather it not read come along for the ride.
.env, certificates, key files- Samples and fixtures containing customer data
- Directories mixed with internal documents, contracts or HR material
Most tools support an exclusion list. But exclusion syntax and defaults differ between tools, and switching tools means moving the configuration too. The surer method is not to keep secrets in the repository at all. Move them to a secret manager or CI secrets, and an exclusion rule accidentally left out stops being an incident.
Cost is a permissions problem too
An agent retries when it fails. That trait raises quality and makes cost unpredictable. You cannot know in advance what one task will run to.
There are two controls.
- Limits. Set an organisation-wide monthly cap and per-user caps separately
- Per-task bounds. If there is an option to cap retries or wall-clock time for a single instruction, turn it on
The classic cost blow-up is a large task in a repository with no tests. With no way to verify, the agent cannot judge for itself and keeps wandering. Putting tests in place first is also a cost control.
Incident scenarios and defences
The types actually reported are broadly these three.
1. Quiet sweeping edits. The instruction named one file, but the agent found the same pattern and changed dozens. → Leave a commit per unit of work, and make the number of changed files in a PR part of the review criteria.
2. Added dependencies. It installs a new package to solve a problem, bringing licensing and supply-chain risk with it. → A PR containing dependency changes always gets human eyes. Make lockfile changes a review trigger.
3. Disabled tests. Instead of fixing a failing test, it switches the test to skipped. → Count skipped tests in CI and fail the build when the number grows.
The order to introduce it to a team
- Start with one repository. Pick a low-risk one that has tests
- Grant layer 1 and 2 only. A human presses layer 3
- Put a rules file in the repository. Conventions, prohibitions and directory structure written down means you do not explain them every time — and it helps humans too
- Look at the metrics after two weeks. PR count, review rejection rate, rollbacks, cost
- Then widen the scope
In short
- The question moved from “which tool is good” to “what should it be allowed to do”
- Sort permissions by how hard they are to undo. Deployments, migrations and package publishing get blocked by account permissions and CI rules, not tool settings
- A dedicated agent account, minimal repository scope, default-branch protection and deployment credentials inside CI are the baseline setup
- For secrets, moving them out of the repository beats an exclusion rule
- Runaway cost usually traces back to missing tests. A way to verify is a way to control spend
Frequently asked questions
Do I need any of this for a personal project?
Scale it down, but the principles hold. Keeping secrets outside the repository and leaving a commit per unit of work — just those two — already make most accidents reversible.
How should commits made by an agent be marked?
Committing from a dedicated account separates the author automatically. When a human account is used, noting it in the commit message is the common approach, and it helps later when you are tracing the cause of a regression.
I’m worried about sending internal code to an external model.
Check the contract terms first. Whether your data is used for training and how long it is retained differ by plan, and training exclusion is often guaranteed only in enterprise agreements. If you cannot confirm it, limiting the scope to a locally running model is another option.
If humans review everything, doesn’t that cancel out the time saved?
Review effort scales with the size of the change. Break the work into small pieces and reviews get short; let tests filter pass/fail first and what a human looks at narrows to design judgement. The saving does not come from not reading code — it comes from the time spent writing the first draft.
Leave a Reply