Start with a decision, not a demo

The fastest way to build an AI product that nobody uses is to begin with the technology. A demo can make a model look capable in a controlled setting. A product has to survive real users, incomplete data and the consequences of being wrong.

A better starting point is a decision or task that already consumes time. Maybe a support team repeatedly classifies the same enquiries. Maybe an operations manager spends an hour assembling a weekly summary. The question is not "where can we use AI?" but "where would a faster, grounded answer change what someone does next?"

Write that decision down in plain language. If the team cannot describe the current workflow, the failure mode and the person accountable for the outcome, the project is not ready for a model.

Choose a bounded first problem

AI products become difficult when they are asked to solve an entire department on day one. A bounded problem has a clear input, a recognisable output and a way to measure whether the output helped.

For a first release, look for work that is repetitive but not trivial, where a human can review the result cheaply. Good candidates include summarising documents, drafting a response from approved material, classifying requests or extracting structured information from unstructured text.

Avoid starting with work where a wrong answer is expensive, invisible or difficult to reverse. Those use cases may become viable later, but they need stronger guardrails and more evidence.

Ground the product in your data

A model does not know your policies, products or customers unless the product gives it access to reliable material. Grounding is the difference between a fluent guess and a useful answer.

That means treating source content as a product asset. Documents need owners, versions and a clear scope. Access rules need to apply to retrieval just as they apply to the underlying systems. The model should be able to cite the source it used, and the interface should make that visible.

If the source material is contradictory or out of date, the model will make the confusion look confident. Fixing the source is usually more valuable than changing the model.

  • Define which sources are approved for use.
  • Keep permissions aligned with the systems users already trust.
  • Show citations and make uncertainty visible.
  • Test with the awkward questions real users ask.

Design the human loop deliberately

Human oversight is not a fallback for a weak model. It is part of the product design. The important questions are where a person reviews, what they see and what happens when they disagree.

For low-risk drafting work, review can be light: the user edits and sends. For higher-impact decisions, the system should present evidence, require an explicit action and keep an audit trail. The person accountable for the decision should remain obvious.

A good interface also gives users a way to report a bad answer without leaving the workflow. Those reports become the evidence you use to improve prompts, retrieval and source material.

Evaluate before and after launch

Evaluation is not a one-off accuracy score. Start with a small set of representative cases, including difficult and ambiguous ones. Record what a good answer looks like and how often the product meets that bar.

After launch, measure the operational change: time saved, rework reduced, escalation rate, user correction rate and whether people keep using the product after the novelty fades. A model can look impressive while making the workflow slower.

Keep a feedback path from production back into evaluation. The cases users struggle with are the most useful test set you will ever have.

Treat it as a product, not a model

The model is one component. The product includes the workflow, permissions, source content, interface, evaluation and support model around it. That is where most of the value and most of the risk live.

A useful first release is small enough to learn from and complete enough to be trusted. Ship it to a real team, watch how the work changes and improve the parts that matter. The goal is not to prove that AI can answer a question. It is to make a decision easier without making accountability harder.