Your AI Works in the Demo. What About Everyday Use?

An AI application answers the presentation questions well. Its writing is clear, responses arrive quickly, and the team can picture where to use it.

Everyday work brings incomplete questions, outdated documents, exceptions, and situations nobody included in the demo. In that environment, the company needs to find out whether the system helps people complete their work to an acceptable standard.

To decide whether a pilot can move forward, turn the impression that it “works well” into criteria the team can check.

Start with the work the application needs to deliver

Consider a hypothetical example: an assistant that drafts replies for a customer support team. It consults internal procedures, suggests guidance, and leaves the decision to send the reply with the specialist.

The expected outcome includes finding the correct rule, preserving exceptions, identifying the source, and recognizing missing information. A well-written answer can fail on any of those points.

Define the task, users, available data, and limits of use. Also record what the application should do when a question falls outside its scope.

The NIST AI Risk Management Framework recommends evaluating systems under conditions similar to deployment and documenting their limitations. Quality needs to be judged in the context where the company intends to use the application.

Build an evaluation set from everyday situations

Collect questions and cases that represent the work. Use records the company is authorized to use, removing or replacing sensitive data when necessary.

Include frequent situations and less common cases with significant consequences. A sample consisting only of easy questions can hide failures that appear precisely in the exceptions.

For the support example, the set could include:

Situation	Expected outcome
Policy with an explicit rule	Apply the rule and identify the relevant source.
Rule with an exception	Account for the condition that changes the guidance.
Question with insufficient information	Ask for the missing detail before reaching a conclusion.
Conflicting documents	Flag the discrepancy and route the question for review.
Content outside the specialist's access permissions	Preserve the access restriction.
No evidence to support an answer	State the limitation without inventing guidance.

People who know the process need to review these expected outcomes. Using another AI to generate questions and automatically accepting its answers as references is insufficient.

Separate cases used to adjust the system from cases reserved for evaluation. If instructions are repeatedly adapted to the same examples, the result may reflect that adaptation rather than performance on new questions.

Define an acceptable answer before measuring

Open-ended tasks may have several correct answers. Comparing the generated text only with a reference sentence penalizes differences in wording and can overlook errors in meaning.

Create objective criteria: is the guidance correct? Does it include the applicable exception? Does the reference support the claim? Does the answer ask for clarification when needed?

Classify errors by impact. A lengthy sentence and the disclosure of restricted information should not carry the same weight. Watch for excessive refusal too: a system that routes every question to a person may avoid some incorrect answers while providing little benefit.

Record which criteria are mandatory and which allow correction during review. This makes evaluation more consistent across reviewers and helps the team decide what to fix first.

Use automation with criteria and human checks

Software can perform some checks directly: output format, required fields, the presence of a reference, and authorization to access a document.

Other checks require assessing meaning and context. A model can help with this analysis, but the scores it assigns also need checking.

The study by Zheng and colleagues on models used as judges identifies limitations such as a preference for longer answers and sensitivity to the position of alternatives. Automated evaluation can be useful without being an absolute measure of correctness.

Compare its assessments with human reviews, especially for higher-impact cases and disagreements. Keep the criteria and evaluator configuration on record so you can compare versions.

For applications that consult documents, as explained in our article on RAG, evaluate retrieval and generation separately. The Ragas paper distinguishes dimensions such as the relevance of retrieved context and how faithfully an answer uses that context. This separation helps identify whether a failure began in retrieval or in the use of the information.

Measure time saved, including corrections

Generation time is only one part of handling a support request. The specialist still needs to read the answer, check the source, correct the text, and decide what to do next.

Compare the total time for the current process with the time for the assisted process, using cases of similar difficulty. Include requests that failed and had to be repeated.

Track correction effort, the frequency of human escalation, cost per completed task, and error severity. Analyze results by case type, since an overall average can hide poor performance in an important category.

An application that saves time on common questions and requires extensive review for exceptions may still be useful. The scope of use and review requirements need to reflect that difference.

Set criteria for moving forward and keep evaluating

Before expanding the pilot, establish approval conditions: minimum quality by category, time and cost limits, verified access controls, and a clear way to stop or escalate a support request.

No single accuracy percentage qualifies every application for production. The decision depends on the task, the consequences of an error, and the ability to detect and correct failures.

A gradual rollout lets the team compare performance with everyday work and collect problems the initial set did not cover. The absence of failures in a sample does not prove the system is safe in every case.

Repeat evaluations after changes to the model, instructions, sources, or integrations. Because answers can vary between runs, critical cases deserve repeated checks. Identify versions and test conditions so the team can understand what changed.

Evaluation becomes part of operations: new errors feed into the tests, and the tests help determine which changes can be released.

How Fidalgo IT Solutions can help

Fidalgo IT Solutions can help define quality criteria, build evaluation sets, and analyze failures in AI applications.

The work can include version comparisons, measurement of review effort, and the design of a pilot with technical oversight. The goal is to determine where the application provides a concrete benefit and which conditions must be met before expanding its use.

Explore our Artificial Intelligence services for business to learn how to begin an assessment with your team.