A retrieval-augmented assistant combines a model with documents or business records selected for the user's question. This article is a reference design for scope planning, not a claim that one architecture meets every project's accuracy, latency or privacy requirements.
Start with access and source ownership
Identify the source owner, update process, allowed users and documents that may be sent to the model. Record source IDs and revisions so a returned answer can link to its evidence. Enforce permissions before retrieval and again before returning records or executing an action.
Retrieval choices to evaluate
| Need | Candidate approach | How to evaluate |
|---|---|---|
| Exact part numbers or policy names | Keyword or structured lookup | Test spelling variants and absent identifiers |
| Questions expressed in different words | Semantic retrieval | Review relevant passages on real questions |
| Several plausible passages | Candidate reranking | Measure answer support and additional latency |
The number of chunks, candidate limit and ranking threshold should follow evaluation. A vector database or reranker does not by itself establish that an answer is correct.
Separate answers from actions
Retrieved documents and user messages can contain instructions that conflict with the application. Treat them as untrusted data. Allow only named tools, validate arguments in the backend, enforce record access and require human confirmation for consequential changes. Log outcomes without retaining unnecessary personal data.
Choose models against a common test set
Compare candidate models using supported answers, missing-evidence behavior, tool correctness, response time and cost per completed task. A fallback must retain the same access and evidence requirements. Validate the fallback separately; switching providers is not proof of equivalent quality or data handling.
Define the delivery evidence
Request a reviewed question set, permission tests, source citations, stale-source handling, failure routing and operating cost observations. Review AI API integration and send a redacted knowledge and workflow brief before agreeing on the architecture.