Ever since Stephen O’Grady’s The New Kingmakers, we’ve all been trying to figure out how to appeal to developers. Now we’re going to have to figure out how to appeal to their agents. AI agents, after all, increasingly write our code and decide which tools, libraries, etc., to use in writing that code. This means that although developers still own the application, it’s often the agents that own the decisions that factor into building that app.
So how do we deal with this?
Software vendors and open source project leaders must make their technology understandable and usable by an agent. Developers and their employers, meanwhile, should make sure an agent’s default reflects what they actually want.
A recommendation is already a decision
Richard Seroter recently pointed to research challenging the comforting-but-false assumption that people reserve important decisions for themselves while the agent “writes the code, runs the numbers, and offers advice,” as the research’s authors note. Anthropic studied 50 professional analysts working with agents given secretly randomized preferences. Across four analytical choices, participants followed the agent’s guidance 86% of the time. For people like me who may not be good at math, that translates to “nearly always.”
A technology recommendation can work the same way. By the time a developer inspects an application’s dependencies, the agent may have built enough working code around its initial choice that changing course means discarding progress. At that point, it’s easier to just stick with the agent’s pick (and developers are all about convenience).
This is a plausible consequence of agent-assisted development, but it’s not yet clear how this will impact enterprise purchasing. Supabase, for example, has done interesting work with documentation, skills, and more to ensure agents and LLMs see, understand, and recommend Supabase. But this isn’t the same as signing a purchase order. Even so, technologies can enter a project long before money changes hands and can help steer that money in certain directions.
Just because the agent or LLM recommends certain technology, however, doesn’t mean it’s doing so effectively. As Microsoft developer advocate Waldek Mastykarz argues, first you should determine if an agent picks up on your tech to answer a given prompt. Second, you should check to see whether it’s answering accurately. He therefore suggests you test the first with tasks that don’t name your product and test the second with tasks that do.
Write for the work the agent attempts
There is already evidence that people can influence which tools an agent uses. The more interesting problem, however, combines deep human understanding of a particular task to inform agents how to fix a bad plan.
Microsoft’s Garry Trinder described such a situation in June. An agent upgrading a SharePoint Framework project kept editing package versions manually. The actual upgrade required other changes, which the CLI for Microsoft 365 could handle. A more direct documentation tip got the agent to use the CLI in only one of five runs. Explaining that changing package versions alone would produce build failures got the agent to use the tool in all five runs.
Five runs per condition is a small test, and this was tool selection within an existing Microsoft project, not a contest among cloud vendors. Even so, it’s a concrete example of documentation changing an implementation choice. The writer understood why the agent’s approach was wrong and put that explanation where the agent encountered it. It’s the best kind of agentic-era documentation.
Supabase has reported a related problem. Agents knew about its platform but still missed security requirements and guessed at commands. In its account of building agent skills, the company says agents sometimes skipped separate reference files so it moved critical security guidance into the main instructions. It also directs agents to current documentation rather than relying on remembered APIs.
These are changes to what agents read and when they read it. They require people who know the product well enough to recognize a plausible but dangerous implementation. Producing another hundred pages of fluent product description wouldn’t fix that problem.
For a database vendor, the useful material might be a tested path from a customer’s existing schema to a working application, including permissions and failure recovery. For an API company, it might be an example that handles retries without charging twice. The commercial value comes from removing a specific reason the developer’s project would fail, so that it doesn’t fail.
Treat content as part of the product
The human work also continues after the first instructions ship. Products and projects change, yet a new model may be stuck interpreting old information.
Google’s Remigiusz Samborski recently described the company’s approach. Changes to Google
[…]
Content was trimmed to protect the source. Please visit the original article for the full text.
Read the original article: