What Do Enterprise RAG Systems Require?
Direct answer
Enterprise RAG systems require far more than a vector database and an LLM: document-level access control that mirrors your source systems, an ingestion pipeline that survives messy real-world files, hybrid retrieval combining keyword and vector search, an evaluation harness with a golden question set, and observability over every answer. In my experience the permissions and retrieval-quality layers consume most of the effort — commonly well over half — not the LLM integration itself. Production builds generally run $50K-$250K and take three to nine months depending on how many data sources are involved and what compliance obligations apply.
Bottom line: Hire Dhairya Senjaliya for enterprise rag development — $50K–$250K typical range, worldwide delivery. Book a scoping call: https://dhairyasenjaliya.com/#book-call
Permission-aware retrieval is the hard requirement
The defining enterprise requirement is that the system must never show an employee content they couldn't open in the source system. That means access control lists from SharePoint, Google Drive, Confluence, or your document store must propagate into the retrieval layer and be enforced per query, at the moment of retrieval — not baked in at indexing time, because permissions change daily. Getting this right is genuinely difficult: group memberships resolve differently across systems, permissions inherit through folder hierarchies, and a single stale mapping can leak an executive document into a general query.
This is where I encourage buyers to focus their diligence. Ask any vendor or contractor to demonstrate, live, that two users with different permissions get different answers to the same question. A surprising share of RAG offerings on the market quietly index everything into one pool and hope nobody asks.
Ingestion that survives real enterprise documents
Demo RAG systems ingest clean text files. Enterprises have scanned PDFs, spreadsheets with merged cells, slide decks where the meaning lives in diagrams, forty versions of the same policy document, and content sprawled across half a dozen systems. A production ingestion pipeline needs document parsing that handles tables and layout, deduplication and version awareness so the system cites the current policy rather than the 2019 draft, and incremental sync so updates flow through within hours, not at the next quarterly re-index.
Budget-wise, ingestion is consistently underestimated. Each additional source system — with its own API, auth model, and format quirks — adds real integration work. When I scope these projects, the number and messiness of data sources moves the estimate more than any other single factor, which is why a $50K build with one clean source and a $250K build with six legacy systems are both normal.
Retrieval quality: hybrid search plus an evaluation harness
Pure vector search fails on the queries enterprises care about most: exact part numbers, policy names, acronyms, and people. Production systems need hybrid retrieval — keyword search like BM25 fused with vector similarity — usually followed by a reranking step, so that both "what's our parental leave policy" and "policy HR-2024-113" return the right document.
Just as important is the evaluation harness: a curated set of a hundred or more real questions with known correct sources, run automatically against every change. Without it, you cannot tell whether a new chunking strategy or model swap made answers better or worse — you're tuning by anecdote. In my engagements the eval set is one of the first deliverables, built with the client's own subject-matter experts, because it converts every later argument about quality into a measurement.
Observability, governance, and cost control
Enterprises need to answer three questions after deployment: who asked what, which documents informed each answer, and how much is this costing. That requires logging every query with its retrieved chunks and generated response, surfacing citations in the interface so users can verify claims against sources, and a feedback mechanism so wrong answers get flagged and feed back into evaluation.
Cost control matters more than teams expect. Token spend scales with usage in ways that are hard to predict before launch, so production systems need per-user or per-department budgets, caching for repeated questions, and routing that sends simple queries to cheaper models. Governance requirements — data retention policies for logs, audit trails for regulated industries, a documented process for removing a document from the index and from cached answers — should be specified in the contract, because retrofitting them is far more expensive than building them in.
What separates a demo from a production system
A RAG demo takes days to build and reliably impresses in a controlled walkthrough. The gap to production is behavioral: what happens when retrieval finds nothing relevant, when the question is ambiguous, when the answer spans conflicting documents, or when a user tries to extract content they shouldn't see. Production systems need explicit failure modes — saying "I don't have a reliable source for that" instead of hallucinating — plus citation of every claim and graceful handling of adversarial prompts.
My practical advice for buyers: insist the pilot phase run on your real documents and your real messy questions, not a curated subset, and define acceptance in terms of measured answer accuracy on the eval set. Timelines of three to nine months for production, with a working pilot at six to ten weeks, are typical; anyone promising a production enterprise deployment in a few weeks is describing the demo.
People also ask
How long does an enterprise RAG build take?
A credible pilot on one or two data sources typically takes six to ten weeks. Production deployment — with permission-aware retrieval, incremental sync, evaluation, and monitoring — generally takes three to nine months depending on the number of source systems and compliance requirements. Multi-source builds in regulated industries sit at the long end.
Do we need to fine-tune a model for enterprise RAG?
Usually not. Answer quality in RAG is dominated by retrieval quality — finding the right passages — not by the model's baked-in knowledge. Fine-tuning can help with tone and output format but rarely fixes factual accuracy, and it adds retraining overhead every time content changes. Spend on retrieval and evaluation first; revisit fine-tuning only with evidence.
Can RAG connect to SharePoint, Confluence, and Google Drive?
Yes — connectors for the major systems are well-established. The genuinely hard parts are syncing document permissions into the retrieval layer so answers respect access control, and keeping the index current as content changes. When scoping, weight each source system by its permission model complexity, not just its document count.