Before you buy an AI tool, map the decision it will influence

An AI demonstration can make weak assumptions look finished. A model summarizes a customer file, predicts a maintenance event, or drafts a schedule in seconds. The meeting moves quickly to licensing and integration.

Slow down. Before evaluating the tool, map the decision it is meant to influence.

That single step changes the discussion from “What can this product do?” to “What will a person do differently, using what evidence, with what consequences?”

Name the decision precisely

“Help sales” is not a decision. Neither is “improve forecasting.” A useful statement names the person, the choice, and the timing.

For example: each Monday, a branch manager decides which late orders need customer contact. Or a maintenance planner decides whether to inspect, monitor, or schedule work on a machine before the next production run.

The decision may be smaller than the original AI idea. That is usually helpful. Smaller decisions are easier to test, measure, and stop if the tool performs poorly.

Write down what happens today. Who makes the call? What information arrives too late? Where does judgment matter? What is the cost of a false positive and a false negative? The answers tell you whether prediction, classification, summarization, search, or no AI at all is appropriate.

Map input, output, and action

Trace the chain on one page:

1. The input available at decision time
2. The output the tool will produce
3. The person who will interpret it
4. The action that person can take
5. The result recorded for later review

This prevents a common pilot failure: producing an interesting answer that does not fit into work. A risk score has little value if the supervisor receives it after the schedule is locked. A contract summary may save reading time but create risk if no one knows which source passage supports a conclusion.

Include the current alternative. A spreadsheet, standard report, threshold, or experienced employee may already perform well enough. AI has to improve the decision or reduce the effort at an acceptable level of risk. Novelty is not a business case.

Decide where human judgment stays

Some decisions can tolerate automation. Others require review because errors affect safety, employment, contractual commitments, customer trust, or substantial spending.

Do not settle for “human in the loop” as a control. Define what the person reviews, what evidence the system presents, when the person can override it, and whether workload makes careful review realistic. If a tool produces hundreds of alerts, nominal human approval may become a click-through habit.

The NIST AI Risk Management Framework organizes AI risk work around govern, map, measure, and manage. Its map function asks organizations to establish intended purposes, context, requirements, users, and possible impacts before deployment. That is a useful discipline even for a modest internal tool.

Test the evidence, not the demo

Ask vendors to demonstrate the decision using representative company data under agreed safeguards. Include ordinary messy cases and cases where the right answer is unclear. A polished example selected by the vendor tells you little about daily performance.

Define the measure before the trial. Depending on the decision, useful measures could include review time, missed exceptions, unnecessary interventions, corrections, or the share of outputs with adequate source support. Model accuracy by itself may not describe the operating result.

Test against the current process. If experienced planners make acceptable decisions in fifteen minutes, a tool that cuts five minutes but requires extensive data preparation may not be worthwhile. If the current process misses serious exceptions, investigate whether the tool finds them early enough for action.

Examine data rights and retention

The decision map shows what information the tool needs. That opens practical questions for security, legal, and operations.

What data will leave company systems? Can the provider use prompts, files, or outputs to train models? How long is data retained? Who can access it? Can records be deleted or exported when the contract ends? How are incidents communicated? Which subcontractors or model providers are involved?

Answers should appear in the agreement and the configured service, not only in a sales conversation. Use less sensitive data when it will do the job. If the decision can be supported with equipment readings and work-order history, it may not need employee names or customer pricing.

Plan for change and failure

AI performance can change when products, customers, processes, data sources, or model versions change. Decide who will notice.

The operating owner should review corrections and exceptions. IT or a data owner should monitor whether inputs remain available and consistent. The vendor should explain update practices and service changes. Leadership should set a condition for pausing the tool.

Record the outcome after action so the company can learn whether the recommendation was useful. Without feedback, a pilot can remain “promising” indefinitely because nobody can show what happened.

Also plan the manual path. If the service is unavailable or confidence is low, how will the decision be made? A tool used in production needs an exit that people understand.

Buy for one real decision

A broad platform may offer dozens of capabilities. Evaluate it first against one recurring, consequential decision with a willing owner and accessible data. The initial use does not have to be the largest opportunity. It should be meaningful enough to test the operating model.

If the map reveals no clear action, weak data, no owner, or an error cost the company cannot control, do not force a pilot. Fixing the process or information flow may be the better investment.

AI can improve how a company sees patterns and prepares choices. It does not remove the need to define the choice. EdgePoint can help facilitate a decision-mapping session when several functions need to agree, but a good first draft often starts with the manager who will use the output and the employee doing the work today.