We've been using the wrong mental model to describe software development AI agents and this might be limiting the value engineers can extract from these tools. I frequently see people referring to AI coding tools as inexperienced developers who can do things for you. I believe this is wrong, these tools are actually senior developers, but they are senior developers without context. These tools know how to write algorithms, they can build complex applications and fix bugs that only experienced engineers can spot. But they don't have the business or the technical context of the thing you are building and that's the main reason why they won't replace human developers any time soon.
Code that produces the right output but that doesn't comply with restrictions imposed by the context at hand is essentially wrong. The most efficient algorithm can increase the complexity of your application which could in the long term lead to bugs and make it hard to make changes to the code. This could be reasonably prevented by using a much simpler algorithm if you know that it's not expected to deal with a high volume of data. Building a highly scalable infrastructure setup is the wrong solution if the constraints are to reduce costs. And it's equally wrong to waste time and resources optimizing the infrastructure if growth is your main goal and money is not a constraint.
Acquiring context is actually one of the main parts of the job of any software engineer and we are quite familiar with the risks of failing to do it. Just as an example of this, as an industry we've recognized that over engineering is a harmful practice and have been learning how to mitigate it for a long time now. Over engineering is essentially a context issue, and is very similar to the problems AI has when it's writing code.
But why is understanding this so important? Changing how you think about AI allows you to improve how you interact with it in order to get better results. For instance, instead of trying to tell it how to do things we can focus on giving it the necessary context that allows it to make better decisions. You can craft your prompts to focus more on the 'what' not on the 'how'. While programming we are constantly making microdecisions that aren't necessarily written, so if we want the AI to also be able to make these decisions it will need more information about the context.
This kind of discussion naturally flows towards the so-called "prompt engineering", but I believe we should aim beyond this. Prompt engineering implies that we are investing a lot into getting the one input that leads the AI to the right decision. But this is far from how we work both individually or within teams. The best solutions come from brainstorming, exploring possibilities and experimenting. That's what we should also be doing with AI. There's no way we will get everything right in one shot, so I think the right path here is that we should instead guide the AI to have a conversation and try itself to acquire more context from us before it actually starts writing the code.
But there's one problem with this approach. The current AI training methods bias models toward producing code immediately so it currently demands some effort in order to keep it iterating with you before committing to an approach. But this behavior is just a result of how LLMs are currently trained, there's nothing preventing future models from being trained differently. I believe this will be the next big change in AI: models that ask more questions before they try to come up with answers. At some point it will feel like a natural conversation we'd have with a colleague and once we've settled all questions and explored solutions we will have a senior engineer with all the context it needs to write all the code in a few seconds.
If you like my writing please consider supporting me by subscribing to my newsletter and buying my book Strategic Software Engineering on Amazon.
