Debugging code with an AI assistant

Most developers spend part of every week working out why some code does not do what they expected.

AI coding assistants can now help with that work. This page covers ways to use one well. It draws on GOV.UK guidance written for government developers, the National Cyber Security Centre (NCSC) and long-standing debugging practice.

Treat it as a quick but inexperienced colleague

GOV.UK's guidance on AI coding assistants suggests treating one like a programmer who lacks experience. It cannot see the bigger picture of your business logic. It does have perfect recall of programming languages and very good lookup skills.

The models behind these tools are, in effect, sophisticated text predictors. The same question can get a different answer the next time. You stay responsible for any change, so you need to understand the code it suggests.

Give it a small example that shows the bug

Assistants work better with more context. A prompt that is specific to the task gets a better answer.

The advice for asking people about a bug applies here too. Stack Overflow asks for a minimal, reproducible example. Use as little code as still shows the problem, and include everything needed to run it. Then test it, to make sure it does reproduce the bug.

Give the exact error message and the line that produces it. Say what you expected to happen. "It doesn't work" is not enough for a person or a tool.

To make the code smaller, remove parts a little at a time until the problem disappears. Then add the last part back.

Keep secrets out of the conversation

A plugin in your editor may send everything open in your workspace to the assistant's provider. Any passwords or keys in that workspace could go with it. The GOV.UK guidance says never to handle sensitive production data or secrets in a development environment.

When the guidance was written, most free and personal-licence assistants recorded prompts and responses to train their models. It recommends business or enterprise contracts, or running an open source model locally. Read the terms of the tool you use.

Check the fix, not just the explanation

Only commit changes you understand. The GOV.UK guidance recommends review by at least one other person before a merge to the main branch. Small, specific commits make that review easier.

Language models sometimes invent version numbers for libraries. Someone could publish harmful code under a made-up version. So check any dependency an assistant adds against a trusted source.

The head of the NCSC has warned that software produced without human review could spread security flaws. The GOV.UK guidance also recommends good test coverage and vulnerability scanning in your pipeline.

Use the tools that already exist

Use an assistant alongside your debugger. Python's built-in debugger, pdb, lets you set breakpoints, step through code line by line and inspect each stack frame. It can also look at a program after it has crashed.