How to Keep Your Judgement in AI-Assisted Software Development

The easiest moment to stop understanding your software is when the code looks finished. I’m James Stevenson, and I run an AI engineering studio where I use coding agents every day. I’ve seen how quickly a sensible habit can shift: first you inspect every change, then you skim, then you approve a confident summary and a green test run. The danger isn’t simply that an agent writes a bug. It’s that a team gradually loses the ability to explain what it has built.

That matters because the job has changed. When implementation is cheap, deciding what to build and proving that it works become the hard parts. Here’s the working practice I use to keep those responsibilities with the people who ship the software.

Notice when review becomes approval by habit

AI output can be convincing without being right for your product. A change may follow familiar patterns, pass the checks it was given and still solve the wrong problem. If the team stops asking why the change exists or what assumptions it depends on, its understanding of the codebase starts to thin out.

Passing tests is useful evidence, not a verdict. Tests can repeat the same mistaken assumption as the implementation. For example, an agent might infer that a cancelled booking should be deleted, then write both the deletion logic and a test that expects the record to disappear. The test is green, but nobody has checked whether the business needs that record for refunds or audit history.

This is why I try to review the decisions behind a change, not only its lines. Did the agent narrow or expand the request? Did it choose a new abstraction, alter an edge case, or rely on an assumption about existing behaviour? Those choices shape what the software does and how maintainable it will be.

A useful test of a review is simple: could you explain the change to the person who will maintain it next? If not, the answer isn’t to trust the summary more. It’s to get the missing reasoning into view.

Put your own thinking before implementation

Before I ask an agent to build something, I write a short brief in my own words. It doesn’t need to prescribe every technical detail. It should name the user or process affected, the outcome we want, important constraints, what is out of scope and how we’ll recognise success.

I also sketch a rough approach. That gives me a starting point to compare with the agent’s proposal. My idea may be wrong; the value is having an independent view before a plausible answer anchors the discussion. If I can’t explain the requirement clearly, I ask the agent to question me about users, edge cases and constraints rather than jumping straight to code.

Then I ask the agent to report its important decisions and assumptions alongside its implementation. For a change to invoice handling, that might include how it treats a missing customer address, whether retries can create duplicate invoices, and which existing rules it followed. I can challenge those points directly instead of trying to infer them from a large diff.

Use a review sequence that fits the work

For a meaningful change, I follow a sequence that keeps implementation and verification distinct:

1. Define the outcome. Write the brief, constraints and acceptance conditions before asking for code. Include the behaviour that must not change.
2. Delegate a bounded task. Ask for one coherent change, not an open-ended instruction to improve a whole area. Request a summary of decisions, assumptions and files changed.
3. Check the evidence. Run the relevant tests and inspect what they actually cover. Where behaviour is visible to a user, exercise that path yourself or inspect appropriate logs and outputs. Add a case that challenges the assumptions, not just one that confirms the happy path.
4. Review for ownership. Read the diff, question the design choices and confirm the result matches the original brief. Ask a colleague or a separate review pass to look for missed risks when the change warrants it, but make sure someone on the team owns the final decision.

For every AI-generated pull request, I use this short checklist:

• Can I explain what problem this change solves?
• Do I understand the agent’s significant decisions and assumptions?
• Do the tests check the intended behaviour, including relevant edge cases?
• What does the evidence fail to prove?
• Would I be comfortable maintaining this change later?

If an answer is no, that is a reason to pause, ask for clarification or reduce the scope. A review checklist only helps when it can stop a merge.

Keep output within your capacity to review

Running several agents at once can feel like progress, but every finished change creates work for a person: understanding it, checking it and deciding whether to own it. If agents produce changes faster than your team can review them properly, you have created a queue, not more usable delivery capacity. Limit parallel work to what reviewers can genuinely inspect, and close or reject stale proposals rather than letting them pile up.

This workflow is most useful now for teams shipping consequential changes, working in unfamiliar code or seeing a rise in agent-generated pull requests. Teams should wait before adding more agents if they cannot define acceptance conditions, have no reliable way to check behaviour, or are already approving work they don’t understand. Start by improving one small change from brief to review.

There are limits to what process can catch. Tests and reviewers can miss subtle faults, and an agent’s explanation can sound coherent while leaving out a crucial detail. Riskier changes need proportionately stronger checks and appropriate human expertise. Better models may help identify defects, but they cannot decide on their own which trade-offs your team accepts or why a feature should exist.

I also make time to solve some problems without an agent. That practice keeps my own debugging and design skills active, so I have a better chance of noticing when an answer is wrong or incomplete. If those skills fade, supervision gets harder precisely when it matters most.

For operators and builders, the aim is not to slow every change down. It is to make sure speed doesn’t outrun understanding. On your next AI-generated pull request, write down the problem it solves and one assumption you need to verify before approving it: hand over the typing, not the thinking; you can delegate the work but not the responsibility.