The security system of internal AI: defense in depth

Security is the core advantage of internal AI: because the model and data run entirely on your own infrastructure, prompts and documents are never sent to third parties. But "on-site" isn't automatically safe — you need defense in depth: multiple independent layers protecting the center, your data & model. This article lists the 8 defense layers, the AI-specific risks (prompt injection, RAG leakage), and how security makes proving PDPL compliance easier.

Why security is the core advantage of internal AI

With cloud AI, every prompt and document you paste in leaves your organization and is processed on the provider's infrastructure — you have to trust the terms, the server location and how they store it. Internal AI flips that: the model runs on your own servers, so all prompts, documents and answers stay inside your network. The attack surface shrinks to exactly what you control.

But "runs on-site" is necessary, not sufficient. A misconfigured internal AI system — an open port to the internet, no access control, no encryption — can still leak data. The correct principle is defense in depth: don't rely on a single protective layer, but stack multiple independent layers so that if one is breached, the next still holds. This is a foundational principle in cybersecurity generally, and it's especially apt for AI because it adds new attack surfaces (prompts, RAG documents).

The defense layers (defense in depth)

From the outside in, an internal AI system should have the following layers. Each addresses a different class of risk and does not replace the others:

1. Network isolation & egress blocking: the inference path needs no internet — block outbound connections at the firewall so prompts/documents can't be pushed out even if software is compromised. This is the strongest evidence for the claim "data never leaves the organization."
2. Authentication & authorization (RBAC/SSO): each user logs in through the corporate identity (SSO), and access is granted by role (RBAC) — who can ask what, who can read which documents.
3. Data encryption (at rest & in transit): data on disk is encrypted (at rest), and every connection uses TLS (in transit) — even inside the internal network.
4. Full audit logging: record who asked what, which documents were accessed, and when — to investigate incidents, detect abuse and support compliance proof.
5. Prompt-injection & jailbreak defense: filter and isolate untrusted content fed into the prompt, limit the model's privileges, and avoid letting instructions inside documents control the system.
6. PII handling / sensitive-data masking: detect and redact/mask personal information before it enters the context or logs, minimizing exposed sensitive data.
7. Secrets/key management: keys, tokens and passwords don't live in source code or arbitrary environment variables — use a secrets manager, rotate keys regularly, grant least privilege.
8. Physical security of the servers: because machines sit on-site, control who can touch the hardware — locked server room, access control, full-disk encryption so a lost machine isn't lost data.

See the companion post to understand where these layers sit in the architecture: Internal AI system architecture diagram.

AI-specific risks: prompt injection & RAG leakage

Beyond ordinary cybersecurity risks, AI systems have two new threats that a security team used to working on infrastructure may not fully anticipate:

• Prompt injection: untrusted content (an email, a web page, an uploaded document) contains hidden instructions that make the model do something unintended — ignore system instructions, disclose data, or misuse a tool. This is the number-one risk in the OWASP Top 10 for LLM Applications.
• Data leakage through RAG: when the AI "reads" internal documents to answer, if there's no per-document access control a user may unintentionally (or deliberately) extract content they shouldn't see — the system answers from documents outside their permission scope.

How to defend, at the level of principle:

• Control document sources: only ingest trusted, vetted sources into RAG; treat all user-supplied content as untrusted.
• Filter & isolate: clearly separate system instructions from data content, don't let text inside documents be interpreted as commands, and limit the privileges and tools the model may call.
• Per-document access control: each document (and each embedding chunk) carries a permission label; on retrieval, only return chunks the current user is allowed to read.

# Linux: deny all outbound, allow only loopback + internal LAN
sudo ufw default deny outgoing
sudo ufw allow out on lo
sudo ufw allow out to 10.0.0.0/8 # internal network only
sudo ufw enable

# verify: a call to the internet must FAIL (no egress route)
curl -sS --max-time 5 https://api.openai.com/v1/models # expect: timeout / connection refused

Compliance (PDPL) is easier to prove

Personal-data protection law (PDPL) requires enterprises to control where and how data is processed, and to prove it under audit. With internal AI, two factors make that proof far easier: (1) data stays on-site — you can point to the physical servers and network boundary where the data lives; (2) controlled security — audit logs, access control and encryption are concrete evidence of protective measures, instead of depending on a third party's terms.

In other words, security isn't just technical — it produces the evidence record of compliance. For details on Vietnam's AI legal framework, see Decree 142/2026 on AI.

The Namtech view

Namtech deploys private internal AI platforms running 100% on-site, and security is designed to the exact defense-in-depth principle above: the inference path needs no internet so egress can be blocked entirely, access is bound to corporate identity with role-based authorization, and every query is audited. We treat security as a continuous process — hardened at deployment, monitored in operation, updated as new threats appear — not a one-time "install and done." The goal is for the claim "data never leaves the organization" to stand on evidence, not just a promise.

Frequently asked questions

Note: This article presents security practices grounded in public reference frameworks (OWASP, NIST, MITRE ATLAS), updated 02/07/2026; apply them according to your organization's specific context and compliance requirements.