An employee asks the company assistant about a customer agreement. The answer is accurate, fluent and based on a real document. It can still be a serious failure if that employee was never allowed to read the agreement.
A useful internal assistant needs both relevant knowledge and a clear access boundary. Uploading more documents improves neither automatically. Start with one group of users and one task, such as finding approved operating procedures, before connecting a broad company archive.
Define what the assistant is allowed to do
Separate answering questions from taking actions. A read-only pilot has a smaller operational scope than an assistant that changes customer records or sends messages. If actions are added later, each needs its own permissions, validation and confirmation rules.
Choose documents with a named owner and an identifiable current version. Define how corrections, withdrawals and access changes reach the assistant. A procedure that was correct at import time can become misleading when the original is replaced.
Check access before information reaches the model
Retrieval-augmented generation, or RAG, looks up material and supplies selected passages to a language model. Retrieval can find relevant text; the application must also enforce the user’s permissions. Use the authenticated identity and current access rules to exclude disallowed material before it becomes model context.
Telling the model “do not reveal confidential information” is insufficient as an access-control design. The same boundary must cover stored conversations, cached results and diagnostic logs. A shared cache that ignores user access can bypass a correctly filtered document search.
Evaluate refusals as well as answers
Prepare representative questions with expected outcomes reviewed by the document owner. Include a question the material answers clearly, one that requires missing information, an outdated procedure and a request for another team’s restricted document. Run the access cases with different test identities.
Score factual support, completeness and appropriate refusal separately. An eloquent answer can fail the access test; a cautious refusal can be the correct result. Record the document versions used so later changes in quality can be investigated.
If the approved material does not support an answer, the assistant should say what is missing and offer an appropriate next step.
Treat retrieved documents as input, not authority to run commands
A document can contain misleading instructions directed at the assistant. Retrieved text should not be allowed to change system rules or grant tool access. Restrict available actions and validate their inputs outside the model. For a read-only pilot, keep write-capable tools out of the workflow.
Decide what conversation data is retained, who may inspect it and how sensitive content is handled by each provider. Local hosting changes the operating responsibilities; it does not automatically solve access control or data quality. Review the actual configuration and agreement rather than assuming privacy from the deployment label.
Include the identity-management interface in that review. A separate admin-console security checklist covers the control plane beside a login system; checking the assistant alone leaves that neighbouring surface unexamined.
Expand only after the boundary holds
Compare the pilot with the existing way of finding an answer. Measure completion time and correctness for the same task, including review and correction effort. Also track unsupported answers and permission failures. Expand the document collection only after the team can explain failures, update content and maintain the access rules. A smaller assistant with dependable limits is easier to trust in daily work.
