Designing a reliable business AI agent: Gold/Silver Retrieval architecture

I – Introduction

LLM-based conversational agents have achieved an impressive level of fluency. However, as soon as they are exposed to a real business environment, a limitation quickly becomes apparent: reliability. In many projects, the approach is to inject more and more data into the prompt or into a RAG system.

The result is often counterintuitive: more data in the prompt -> more errors (hallucinations).

In this article, I will present an AI agent architecture solution that I have created, which yields interesting results for this use case.

 

II – Classic architecture

Data is often treated as a single block of text. However, this information does not all serve the same purpose:

• some is used for reasoning,
• while other is used solely to answer direct questions.

Mixing these two levels is one of the main causes of:

• hallucinations,
• contradictions,
• ambiguous answers.

Of course, it depends on the use case. If the solution is only intended to answer factual questions, it is acceptable not to distinguish between different types of information.

On the other hand, if you want a solution that can reason, compare and relate different pieces of information, it is essential to classify the data and build a clean, structured knowledge graph.

III – Proposed solution

1) Data classification

The Gold/Silver Retrieval architecture is based on a strict separation of knowledge according to its role in the system. Silver data is used in a traditional RAG context, while Gold data is used in a GraphRag context.

Gold data – the foundation of reasoning

Gold data defines the core of the business: rules, constraints, dependencies, relationships between concepts – in other words, everything that can explain the business.

They are never used as simple text; they are:

• normalised,
• transformed into explicit concepts,
• structured in the form of a knowledge graph.

This graph becomes the source of truth for the AI agent, an AI agent that is an expert in its field.

Silver data – the response knowledge

Silver data mainly comprises supplementary data for a particular profession, for example:

They follow a classic RAG approach: vector indexing, semantic search, enrichment of the generated response. Their role is to respond, never to reason.

2) Intelligent orchestration

The true value of a Gold/Silver architecture lies not only in the separation of data layers, but above all in their intelligent orchestration.

This orchestration is at the heart of the decision-making system. It aligns the model’s reasoning with business rules, while organising controlled and relevant use of data.

When a question is submitted to the system, it initiates a multi-step process:

1. Analysis of the nature of the question in order to determine its type:
   • factual (search for specific information),
   • business (requiring rules, constraints or domain logic),
   • hybrid (combining facts and business reasoning).
2. Selection of the appropriate execution path, based on this typology.
3. Application of a fundamental rule: Reasoning is never performed directly on Silver data.

In this architecture, Silver data retains a strictly informational role.
It provides the facts, values and contextual elements necessary for the response,
but never serves as a basis for reasoning.

Reasoning is guided exclusively by: business rules, semantic relationships and
the structure of the knowledge graph (Gold layer).

This separation guarantees: consistency of responses, control over model behaviour and
reproducibility of reasoning.

Through appropriate orchestration, the LLM receives, on the one hand, the business rules from the
knowledge graph, which define how to reason, and, on the other hand, the relevant Silver data, which defines what to respond to.
The graph thus acts as a cognitive framework, guiding the
model’s reasoning without imposing factual content on it.

 

IV – Example: a bank advisor

An analogy helps to clarify how this works. The system behaves like a bank advisor :

the business rules and knowledge graph represent its professional training,
telling it how far it can go in its answers and what logic it should use to reason;
the Silver data corresponds to customer and product information, which is necessary to
provide concrete answers to direct questions.

The advisor does not deduce their rules from customer data; they apply their business rules to the
data.

 

V – Conclusion

The innovation of this architecture therefore lies not in the simple separation of layers, but
in the system’s ability to:

analyse queries in detail,
dynamically orchestrate reasoning,
and ensure a strict separation between business logic and operational data.

This approach makes it possible to build robust AI agents that are explainable and aligned with
business requirements, while fully exploiting the potential of LLMs.

Written by Brahim KERDAD