Guides

RAG and semantic search implementation

Prepared by: improveIT.pl

How to build an assistant that answers from company documents.

RAG and semantic search for business: how it works, stages, permissions, answer quality and costs. An improveIT guide with prices excl. VAT.

In brief

RAG (retrieval-augmented generation) means a language model answers from retrieved passages of company documents, not only from its own knowledge. Semantic search finds passages by meaning, even when the question uses different words than the document. A RAG implementation at improveIT costs USD 2,800–6,325 excl. VAT (PLN 11,200–25,300).

How RAG works

Documents are split into passages and stored in a vector database. When a user asks a question, the system retrieves the best-matching passages and passes them to the model together with the question. The answer links to its sources.

  • Document import and chunking
  • Semantic and keyword search
  • Answers with source citations
  • Document permission checks

Semantic search without a chat

Not every company needs an assistant. Sometimes a search engine that understands the meaning of queries and returns the right documents is enough, e.g. procedures, contracts or product sheets.

  • Faster access to procedures and contracts
  • Search across several systems at once
  • Lower cost than a full assistant

Implementation stages

The first stage covers one document set and one user group.

  • Choosing documents and a content owner
  • Permissions: who can see which document
  • Building search and answers
  • Testing on a set of real questions
  • A pilot with users and fixes

How to measure quality

Before launch, prepare a few dozen questions with correct answers. Compare each system version on this set.

  • Are the right documents in the results
  • Does the answer match the source
  • Does the system admit when it does not know
  • Answer time and cost

Common mistakes

Most problems come from data, not the model.

  • Outdated or duplicated documents
  • No permission control
  • No test question set
  • Loading every document at once instead of a pilot

FAQ

How is RAG different from plain ChatGPT?

ChatGPT answers from what it was trained on. RAG first searches company documents and answers from them, with links to sources.

Can RAG run on our own server?

Yes. The document store and the model can run on company infrastructure, although this raises implementation and maintenance costs.

How many documents do we need to start?

One well-maintained set is enough, e.g. customer service procedures. Start with a smaller, current set rather than every file in the company.

How much does a RAG implementation cost?

At improveIT USD 2,800–6,325 excl. VAT for the first stage, plus monthly model and hosting costs.