Banning generative AI is difficult to enforce and rarely sustainable. Opening access without policy, however, creates an unmanaged channel for documents, source code and commercially sensitive information.
A better alternative is a governed AI environment in which the choice of model and permitted actions depends on the user, the data class and the business scenario.
Start with classification, not a model
The architecture should distinguish at least four information classes:
- public data that may be processed by an external service;
- internal material without critical restrictions;
- confidential data available only to approved models and groups;
- information that must not be sent to an AI system or used for training.
Classification may come from DLP, MDM, the document catalogue or rules inside a business application. The important point is that the decision is not left entirely to the user at the moment they submit a prompt.
One AI gateway
Cloud and local models are easier to govern through a shared gateway. It performs several functions:
- Authenticates the user and obtains their roles.
- Determines the class of the request and attached data.
- Selects an approved model and processing route.
- Applies masking, filtering and tool restrictions.
- Creates a technical audit trail without unnecessarily copying sensitive content.
The gateway does not replace DLP, IAM or SIEM. It turns their decisions into AI-application behaviour.
RAG also needs boundaries
An enterprise knowledge base should not become a single corpus available to every agent. Indexing must preserve the source, owner, information class and access policy. Retrieval filters should run before passages are provided to the model, not after an answer has already been generated.
This matters when documents look similar but belong to different projects, legal entities or confidentiality domains.
Give an agent fewer rights than a person
An AI agent that can act as well as read introduces a new category of risk. Access to ERP, CRM, file stores and service systems should be exposed through narrowly scoped tools.
Critical actions require human confirmation. Every invocation should record the initiator, objective, input parameters, outcome and policy decision.
A local model does not solve everything
Running an LLM inside the perimeter reduces third-party data exposure, but it does not remove excessive permissions, cross-department leakage, prompt injection or uncontrolled agent actions.
The choice between a cloud and local model is therefore only one architecture decision. The maturity of the environment depends on policy, observability, data quality and the ability to investigate an incident.
A useful first stage
A practical pilot needs only one valuable scenario, two data classes and a limited set of models. It should test:
- correct request routing;
- access enforcement during RAG retrieval;
- masking of sensitive data;
- completeness of the audit trail;
- clear refusals and warnings for users;
- cost and latency for each route.
This pilot answers more than “how well does the model respond?” It shows whether the organisation can safely govern a new technology environment.