Before you buy an AI agent, make it pass an exit test

I recently moved my own setup from one coding agent to another. It took about 10 minutes and I thought it went well.

The first agent had a lot to carry. Today it has a global instruction file, 114 skill folders (most installed, a few written by me), 189 small memory notes spread across 24 projects and 73 scripts that shell out to its command-line tool. To give a second agent the same footing, I copied the instruction file to the name the other tool reads, AGENTS.md, and ran a find-and-replace on the product name.

I didn’t read the result properly until this month. It tells the new agent to make every model call with Codex -p. In that CLI, -p is short for –profile. Followed literally, my instruction loads a configuration profile named after whatever prompt it was handed. The same file still warns the new agent, in bold, never to use the old vendor’s API key, a key it had no reason to go near. Nothing errored. The file sat in my home directory for almost three months looking like a finished migration.

That is the exit problem in miniature. I could switch models in an afternoon. The work didn’t come with it, and nothing told me.

Portability claims stop at the model

Vendors have learned to answer the lock-in question. Ask about switching and you hear about model choice, standard protocols and conversation export. InfoWorld has already made the case that the new AI lock-in lives in the layers around the model rather than in the model. The standards side is moving too. MCP and AGENTS.md now sit under the Linux Foundation’s Agentic AI Foundation, which is good news for the plumbing.

None of that tells you whether a half-finished job survives a change of agent.

An agent that has worked in your codebase for six months has picked up things that are neither the model nor the transcript: decisions it was told to respect, approaches it tried and dropped, changes it made, changes it only planned and the permissions and hooks wired around it. Exporting a conversation gives you a record of what was said. It doesn’t give you the state of the work.

Regulators have noticed the general shape of this. The EU Data Act’s cloud switching rules have applied since September 2025, and from January 2027 providers can’t charge customers for the switch itself. Those rules were written for data processing services. How they’ll be read for an agent’s working state is anyone’s guess, and I wouldn’t plan a procurement around it being settled in your favor.

The exit test

So test it yourself before you sign or renew. The drill takes a day and one realistic task.

Pick maintenance work that needs memory. The example I’d use, and this is illustrative rather than a study I’ve run, is upgrading a logging library across a service with a dozen modules. Partway through, a human decision changes the plan: the team agrees to keep the old log format for one downstream consumer, which reverses an assumption the agent made on day one.

Let the first agent get about halfway. It has migrated seven modules, skipped an eighth on purpose because of that decision and left a note saying why. Stop it there. Ask the product to export whatever it says represents the work, meaning requirements, decisions, completed actions and remaining steps. Hand that export, and only that export, to a second agent. A different vendor’s agent is the real test. A fresh session with no access to the first one’s memory is the minimum.

Then watch what the second agent does with it. Four things matter.

1. Does it redo finished work? If it re-migrates the seven modules, the export recorded intent and lost completion. Repeated changes are cheap in a logging upgrade. In a schema migration, they are an incident.
2. Does it bring back a retired assumption? This is the failure I care about most. If the second agent helpfully “fixes” the module the team deliberately skipped, the export carried the original plan and dropped the decision that overrode it. A handoff has just undone a human call, and nothing in the build will flag it, because the code is perfectly consistent with the plan that no longer exists.
3. How long does a human spend rebuilding context? Time it with a stopwatch. Every question the engineer has to answer from memory is state the export didn’t hold.
4. What broke on the way over? Instruction files that name one vendor’s commands. Skills written in one tool’s format. Hooks that fire on one tool’s events. Credentials scoped to one product. Write each of these down. That list is your real switching cost, and in my experience it is longer than anything in the contract suggests.

Run the drill twice. Once inside the same product, a new session picking up the old one’s work, and once across vendors. The first tells you whether the product’s own continuity is real. The second tells you what leaving costs.

Don’t let the vendor pick the task. Every product has a demo path where its memory looks flawless, and it is

Read the original article:

Before you buy an AI agent, make it pass an exit test