Private-data AI agents: assistants that answer from your own documents and data
By Vishal · published 25 October 2026
A private-data AI agent is an assistant that answers questions using your company's own documents, databases and systems rather than general internet knowledge, and shows where each answer came from. VAI Five builds them to run inside your cloud account or on your own hardware, with an evaluation set so you know how often they are right.
What it does for a company
The problem is rarely a lack of information. It is that the information sits in forty places and three people know where. An agent grounded in those places lets anyone ask in plain language and get an answer with a citation.
| Team | Question they ask | Sources the agent reads |
|---|---|---|
| Sales | "What did we promise this client on SLA, and what is in the renewal?" | Contracts, CRM notes, email threads |
| Support | "Has this error happened before, and what fixed it?" | Ticketing system, runbooks, release notes |
| Operations | "Which suppliers were late more than twice this quarter?" | ERP tables, purchase orders |
| Founders and finance | "Explain the margin drop in August." | Accounting exports, board decks, dashboards |
| HR and compliance | "What is our policy on notice period for contractors?" | Policy documents, HRMS |
How it is built
The pattern is retrieval-augmented generation (RAG) with agent tools where a question needs more than reading.
- Ingestion — connectors to your document stores, databases and SaaS tools; chunking and metadata that respect your permissions model, so a person only gets answers from what they may see.
- Retrieval — hybrid search (keyword plus embeddings) with reranking; the difference between a demo and a system people trust is mostly here.
- Generation — a language model instructed to answer only from retrieved material and to say "not in the documents" when it is not. Citations are mandatory, not optional.
- Tools — when the question needs a live number, the agent queries the database or the API rather than guessing from stale text.
- Interface — a web app, a Slack or WhatsApp bot, or an endpoint inside your product.
Where it runs
Three deployment options, chosen for your data sensitivity and budget.
- Your cloud account (AWS, GCP, Azure in Indian regions) with a hosted model API under a data-processing agreement. Fastest to build, lowest upfront cost.
- Your cloud, open model — an open-weight model served on your own GPU instance so no prompt or document leaves your account.
- On-premise — the whole stack on a workstation or server in your office. Our own reference stack runs a 30-billion-parameter open model with tenant-scoped retrieval on a single high-memory workstation; it is the setup we recommend for law firms, clinics and manufacturers with strict data rules.
How we know it is right
Before building, we collect 100–300 real questions from your team with the correct answers and sources. Every change to the system is scored against that set: answer correctness, citation accuracy, and refusal rate when the answer is not in the documents. You see the numbers each week of the pilot. An agent that scores 70% is a research project; one that scores 95% on your own questions is a tool people use.
How a project runs
- Feasibility note (free) — a call about the questions people actually ask and where the answers live; a one-page note with the architecture, the deployment option and the pilot price.
- Two-week pilot (fixed price) — one team, one or two source systems, the evaluation set, a working assistant and the scores.
- Build and handover — remaining sources, permissions, interface, monitoring, documentation and a handover session. Your repository and your infrastructure.
Questions people ask
What is the difference between RAG and fine-tuning?
RAG retrieves your documents at question time and lets the model read them, so it stays current and can cite sources. Fine-tuning changes the model's weights to learn a style or domain, but it does not reliably memorise facts and cannot cite. Most company assistants need RAG; some benefit from both.
Can the agent see everything in our company?
Only what you connect, and only what the asking person is permitted to see. Permissions are enforced at retrieval, not left to the model.
Does our data get used to train anyone's model?
No. With hosted model APIs we use business tiers under data-processing terms that exclude training; with open models in your account or on-premise, nothing leaves at all.
How long until a team can use it?
A pilot assistant for one team is usable within two weeks. A company-wide rollout with permissions and several source systems typically takes six to ten weeks.
Tell us the problem, not the model.
A 30-minute call and a one-page feasibility note, free. If a two-week pilot makes sense, we scope it with a fixed price. If it does not, we say so.
Related