Department-level access control when rolling out internal AI means: each department (finance, HR, sales…) only gets AI answers based on documents it is allowed to see — no cross-leakage. The standard approach is RBAC (role-based access control) at the application layer, combined with enforcing access right at the RAG layer (the AI only retrieves the document chunks a user is permitted to see). This article gives you two diagrams: a department map and a role × permission matrix, plus how to roll it out gradually.
Quick summary
- Why access control: avoid internal leakage — finance shouldn't be able to query HR files, staff shouldn't read board-level documents.
- RBAC: assign permissions by role (Admin / Department head / Staff / Guest), not per individual.
- The real checkpoint is the RAG layer: the AI may only retrieve documents a user is allowed to see; otherwise access "leaks" through the answer itself.
- Technique: tag each document chunk with department metadata, filter by SSO group when querying the vector DB.
- Rollout: department by department, use SSO + groups, turn on audit logging from day one.
Why access control by department?
Internal AI is only useful when it can "read" the organization's real documents. But one AI assistant serving every department introduces a new risk: if everyone can ask anything, the information boundaries between departments disappear. Finance shouldn't be able to query HR's detailed payroll; a salesperson shouldn't be able to extract another team's customer contracts; and no department should read the board's strategy documents.
This isn't a hypothetical risk. In OWASP's risk list for LLM applications, sensitive information disclosure is its own category: a model can inadvertently surface data the user isn't allowed to see in its answer. For internal AI, department-level access control is the first fence against that kind of leak — and it must be designed into the architecture, not patched on later.
The foundational principle is least privilege: each person has exactly the access their job needs, no more. Role-based access control (RBAC) is the most common way to realize this principle at organizational scale.
The department map for internal AI
The diagram below shows a single internal AI assistant at the center, sitting inside your own infrastructure boundary. Each department connects to the AI, but the document scope the AI is allowed to retrieve for that department differs — that's each department's own "field of view".
The key point: even though every department uses one AI system, each department's "document scope" must be kept separate. When someone in Finance asks about an HR employee's salary, the AI should respond as if that document does not exist — not "you don't have access" (which already reveals the document exists).
The access-control matrix (RBAC)
RBAC assigns permissions to roles rather than to individuals, then assigns users to roles. So when someone changes department, you only change their role instead of re-auditing every permission. The RBAC model is standardized by NIST and underpins access control in most enterprise systems.
The table below illustrates four typical roles and their permissions. Higher roles inherit the permissions of lower ones — in the spirit of hierarchical RBAC.
| Role | Q&A on dept docs | View whole org | Manage users | Export data | View audit log |
|---|---|---|---|---|---|
| Admin | ✓ | ✓ | ✓ | ✓ | ✓ |
| Department head | ✓ | — | — | ✓ | ✓ |
| Staff | ✓ | — | — | — | — |
| Guest | — | — | — | — | — |
The diagram below shows the same idea as a role → permission mapping: the higher the role, the more permissions it accumulates.
Access control at the RAG layer
RBAC at the application layer (who can click which button, open which screen) isn't enough. For internal AI, the most important checkpoint is the RAG layer — where the AI goes to find documents to answer with. If the retrieval step doesn't filter by permission, then no matter how tightly the UI locks the "view payroll" button, the user can still ask and the AI will weave payroll content into the answer. Access has "leaked" through the answer itself.
The right approach: enforce access right at retrieval time. Each document chunk is tagged with department / sensitivity metadata; when a user asks a question, the system only searches within chunks their group is allowed to see. The AI never "sees" out-of-scope documents, so it cannot accidentally disclose them. This is also a direct control for the sensitive information disclosure category in the OWASP Top 10 for LLM applications.
Details on building the RAG layer are in the RAG for internal documents article; the overall defense layers (blocking outbound connections, access control, logging) are in Internal AI security system.
Two layers to combine: RBAC at the application and document-level filtering at the RAG layer.
- Tag department metadata onto each chunk at ingest time:
department,sensitivity, the list of allowed groups. - Filter by SSO group at retrieval: take the user's groups from the SSO token, turn them into a filter condition before the vector search.
- Don't trust the client: the filter condition must be enforced server-side, never accepted from the browser.
- Audit-log every query: who asked, which scope was retrieved.
A conceptual example — querying the vector DB with a filter based on the user's groups:
# department groups come from the user's SSO token (NOT from the client)
groups = user.sso_groups # e.g. ["finance"]
results = vectordb.search(
query_embedding = embed(question),
top_k = 6,
filter = { # filter BEFORE ranking by similarity
"department": {"in": groups},
"sensitivity": {"lte": user.clearance},
},
)
# the AI only receives the filtered chunks -> cannot leak out-of-scope docs
answer = llm.generate(question, context=results)Rolling out department by department
Don't enable access for the whole organization at once. The safe path is a phased rollout, one department at a time:
- Pick a pilot department with clear documents (e.g., one department's internal procedures), ingest documents with metadata, and test the retrieval scope.
- Use SSO + groups as the single identity source: users sign in once, department groups decide document scope — no scattered account management.
- Turn on audit logging from the start: record who asked what and which documents were retrieved — to review on suspected leaks and to prove compliance.
- Review access periodically: when someone changes department, lock/change their role immediately; periodically re-check who has what (least privilege).
- Expand gradually to the next department once the pilot runs smoothly.
The Namtech view
On the internal AI platform Namtech deploys (running 100% on-site on Apple Silicon with open-source models), department-level access control sits at two layers: RBAC for the application and document filtering right inside RAG by SSO group. We turn on audit logging by default and roll out department by department so you can clearly see the information boundaries before expanding. The goal: a shared AI, but each department only "sees" exactly its own part — and data still never leaves the organization.
Frequently asked questions
How is RBAC different from assigning permissions per person?
RBAC assigns permissions to roles (Admin / Department head / Staff / Guest), then assigns people to roles. When someone changes department, you only change their role instead of re-auditing each permission — fewer mistakes and easier to audit than per-individual assignment.
Why isn't locking the UI enough?
Because the AI answers based on the documents it can retrieve at the RAG layer. If retrieval doesn't filter by permission, a user can still ask and the AI will weave out-of-scope content into the answer. The checkpoint must be at the retrieval layer, not just at the buttons on screen.
How do I stop the AI from disclosing out-of-department documents?
Tag each chunk with department / sensitivity metadata at ingest time, then filter by the user's SSO group at retrieval time. The AI only receives filtered chunks, so it cannot disclose out-of-scope documents — even if the user tries to ask.
Do I need audit logging?
Yes. Recording who asked what and which scope was retrieved lets you review on suspected leaks, detect privilege abuse and prove compliance (e.g. PDPL). Turn it on from the pilot deployment.
Want internal AI with tight department-level access control?
Namtech deploys private internal AI platforms with RBAC + document filtering at the RAG layer — each department sees only its own part, data never leaves the organization.
Book a free consultationNote: This is a guide on the access-control model, updated 02/07/2026; adjust roles and scopes to your organization's actual structure.