One of the reasons programmers “got” agentic AI first, in the sense that it could meaningfully alter their productivity and effectiveness, is because code could be checked. They turn agents loose on a function, read the diff, run the tests, and decide whether to keep it. Output comes back to them in a form they already know how to read, so trusting the machine didn’t necessarily require a ton of faith.
The folks who work with data all day long sit one step back from the same door. People in business and financial operations roles outnumber software developers in the US by about six to one, and they’ve thus far picked up a small fraction, at best, of the productivity gain that engineering has absorbed. Yes, they can describe the analysis they want, which is what requires their specialized knowledge. What comes back to them is a number or a chart, or maybe two hundred lines of code, none of which shows an analyst how the results were produced.
The output, in other words, cannot automatically be assumed to have used the right pieces of data, combined it correctly, and finally computed with the right formulas. Or maybe the agent missed a bunch of the data. The result is a black box, which isn’t something that should breed confidence.
The question then becomes why, and there are three solvable variables.
Problem 1. The user’s own data is out of reach
A business user’s data sits in spreadsheets, files in SharePoint, applications such as Salesforce, and data platforms such as Databricks, Snowflake, or BigQuery. Getting to it means filing a request with a data platform team whose responsibility is governing everything. Of course, as just about any data platform engineer will tell you, the team’s headcount is undersized for the requests it receives, and what the business user gets as a workaround is often a copy of a copy. Maybe it’s a CSV that was pulled a couple of months ago, or a workbook that has been handed off four times with untracked changes. Then when an agent goes to work on that data, the business user has something fast… working on a shadow version of the truth. The data lineage is long gone before the agent takes its crack, so no one is really sure where certain numbers came from, and the trust has been lost. Not good regardless of use case, and probably a career event if you’re at a financial institution.
Problem 2. There’s no data fact-check
Chatbots hand business users a result with relatively little context for the user to validate it. Showing them the generated code isn’t the answer, since these users aren’t expected to read it. The right response is not signing off on decisions made with data that cannot be checked, but, like programmers shipping code, the pressure to move quickly and deliver supersedes “right” much of the time. And when an auditor or higher-up asks how a figure was produced, “the AI said so” isn’t the answer a business user will want to give.
Problem 3. Tools stop short of the job that matters
A lot of the products available for the business users assume that the data is clean and ready, sitting neatly in a table somewhere, waiting to be agent-ified. These products, based on text-to-SQL queries, assume ready data; they don’t prepare the data for AI. A business user’s day starts before that, with vendor files arriving in myriad formats and data stored four different ways. Skipping past that AI data prep work and just going straight to the query is why so many of these tools demo gorgeously and then go underused in practice.
When AI preps data, it must show its work
When a business user has an artifact that they can understand, you’ll see the same trust unlock that played out in engineering. When an agent prepares data, results should be visual workflows that show step by step how the data was read and transformed at each stage. You should want to know without a hint of doubt where a revenue figure came from or what happens when a date range moves. Business users can walk through it themselves, fixing the steps where agents misunderstood the business logic. They can put their name on the output with a lot more confidence. This audit capability means having guardrails in the plain meaning of the word, where a human in the loop is, well, actually positioned to be in the loop rather than just a tacit nod at an agentic black box.
Running this inside a governed platform (on a schedule someone monitors, under existing access controls) lets the work queue drain without losing oversight on data egress. “Shadow spreadsheets” shrink when governed paths are also the fastest ones.
AI is here, and self-serve AI will become available to business users. But let’s not repeat the mistakes of previous waves of self-service. Desktop tools put workflow-building on a thousand machines, with silos and a decade of technical debt. In this era, let’s get the trust and governance right.
What comes next isn’t solved with another round of
[…]
Content was trimmed to protect the source. Please visit the original article for the full text.
Read the original article: