Is your AI strategy improving the business, or just making the replies sound better?
A beautifully written email is useful. An accurate quote that reaches the customer sooner is a stronger business case. When I look at an AI investment, I want to know whether the whole job is getting better.
Usage figures tell you whether people are using the software. I would also like the business case to include something that happened after the login screen.
My view is straightforward: start with a piece of work that matters, understand why it is difficult, and measure whether the whole job improves. That gives you a much more useful brief than asking everyone to find something to do with a new subscription.
Follow the quote all the way through
Consider a fictional industrial distributor. A customer needs a replacement component and sends an enquiry with an old part number, a photograph and half the information needed to confirm suitability.
An AI assistant can help the salesperson produce a polished reply. But the quote still waits for someone to identify the current replacement, check the application, confirm pricing and availability, and approve an exception.
The email now sounds excellent. The customer is still waiting. At least the delay is better written.
I would look at that whole chain before buying more licences. Which missing details can we collect earlier? Where is the approved product information? Who can confirm suitability? What needs a specialist, and what is simply waiting for a clear owner?
A useful design might extract details from the enquiry, flag missing information and organize approved sources for the salesperson. It could prepare a proposed price using current records and established calculation rules, with the inputs and exceptions visible for review.
Sometimes the first improvement will be a better enquiry form, current records or a clearer approval route. I am quite comfortable with that. The customer is unlikely to object because the solution was insufficiently futuristic.
Give the human reviewer something worth reviewing
Have you asked AI to write an email or put together a complex presentation, then found yourself changing the tone, the emphasis or the recommendation?
The draft may be competent. It may also miss why this customer is cautious, which trade-off matters or what you are prepared to promise. More context can improve the result, but someone still needs to decide whether it fits the situation.
Those edits are useful design information. If reviewers repeatedly correct the same assumption, examine the source, instruction or process behind it. Otherwise, you risk automating the first draft and creating a permanent editing department.
For a manufacturer or distributor, I would build that learning into the workflow. Give the reviewer an organized decision brief: the customer's need, relevant product information, proposed pricing or analysis, sources, assumptions and anything unresolved.
The aim is to give that person more time to assess the recommendation and less work assembling it. Their value includes the commercial context, practical judgement and authority to make the decision.
The level of oversight should match the consequences. A routine internal summary and a quote committing the business to a price deserve different controls. Where an error could create a safety issue, a customer commitment or a liability concern, define what needs a qualified person's approval before it moves forward.
A human in the loop needs the information, time and authority to challenge the answer. An approval button alone is a rather optimistic risk strategy.
Measure the finished job
For that quoting workflow, I would agree a small set of measures before a pilot:
End-to-end time: from the customer’s initial enquiry to a usable, approved quote. Record waiting time as well as active work.
Quality: how often the quote needs correction, which important details were missed, and whether the customer received an accurate answer.
Effort: the total staff time spent preparing, checking, correcting and handing over the work.
Cost: the software, integration, training, support and review effort required to produce that result.
Customer progress: whether the response helped the buyer take the next step. Track sales separately, without assuming every change in orders came from AI.
Keep usage figures alongside these outcomes. They can help identify a training gap or a tool nobody finds useful.
Compare a representative set of enquiries before and during the pilot. Include the awkward ones, record changes in volume or complexity, and count the work passed to another person. A faster draft offers little help if someone else spends longer checking it.
Time released is valuable, but it does not automatically become a cash saving. Be clear whether the team is using it to handle more demand, improve service, reduce overtime or spend longer on important decisions. Those are different benefits, and they deserve honest descriptions. The accountant may ask for something more specific than “everyone feels quicker”.
Write down the decision before the pilot
A useful pilot should end with a decision. I would put these questions on one page before starting:
- What counts as finished? For our distributor, it is a usable, approved quote, rather than a first draft.
- What are we comparing? Choose comparable enquiries and record complexity, missing information and unusual cases. Decide the evaluation period before looking at results.
- Who checks the result? Name the reviewer and record their time, including corrections and work handed to another team.
- What would make us stop? Set the boundaries for unacceptable errors, unsupported commitments or information being used outside its approved scope.
- What would justify continuing? Agree the required improvement and acceptable cost with the people responsible for the work.
Keep a short record for each enquiry: received, ready for review, approved, corrections required, total staff effort and any unresolved issue. Use only the information needed for the assessment, with appropriate access and retention controls.
Then compare the result with the starting point. If turnaround improves but total checking effort rises, investigate the trade-off. If the main improvement comes from fixing the form, record that honestly.
A small pilot can help you decide what to test next. Treat its results as evidence for that decision, with the limits of the comparison made clear.
Make the process usable for the people doing it
Bring the people doing the work into the design. Decide who owns the process, who handles exceptions and what happens when the system cannot provide a reliable answer. Include a practical fallback when the usual reviewer is unavailable.
Start with one bounded workflow so the people doing the work have a chance to improve it before you expand the pilot.
Choose where the model runs deliberately
For some businesses, a locally hosted model is worth considering. For example, Ollama documents a local-only mode that disables its cloud models and web search. That demonstrates one option for local processing. Whether it fits your business depends on the whole application, its connections and your requirements. Ollama documentation
A server in your building still needs access controls, secure configuration, updates and a clear policy for logs, backups and retention. Any external tools or services must be accounted for. Local hosting also brings hardware and maintenance costs.
Effectiveness needs its own test. Use realistic tasks, including missing information and unusual cases, to check accuracy, response time and the amount of human review required. Privacy and performance are things to design and validate. Neither comes free with the word “local”. NIST’s generative AI risk profile identifies risks including confabulation, data privacy and information security, and recommends evaluation suited to the deployment context. NIST Generative AI Profile
Start with a proper review
This is why I put consultation and review at the start. At OrangeFin, we can help you examine how the work actually moves, identify the bottlenecks, review the information and risks, and define a practical pilot with clear measures of success.
Where it is the right fit, we can build locally hosted AI systems for your business, using models selected or adapted for your workflows and designed for private operation. The security controls and real-world performance need to be validated before wider use, then reviewed as the system and your business change.
That consultation and review process is how we make useful, measurable improvements possible. Bring us one workflow that keeps getting in the way. We can start there.