Every CISO I have worked with over the past year has faced the same uncomfortable realization at a different pace than the rest of the organization: generative AI did not just create a new category of business value, it created a new category of attack surface — and most security programs were designed for a world that no longer exists.
The traditional cybersecurity playbook assumed a relatively closed system: known applications, known data flows, known user behavior. Generative AI breaks each of these assumptions. A large language model can be manipulated through natural language rather than code. An AI agent can be tricked into leaking sensitive data through a cleverly worded prompt rather than a technical exploit. And employees, eager to be productive, connect unapproved AI tools to corporate data faster than any security team can inventory them.
This article is for CISOs, CIOs, Chief Data Officers and Digital Transformation leaders who need to understand, concretely, what generative and agentic AI actually change in the threat landscape — and what a credible security posture looks like in 2026.
Table of Contents
- Understanding the new AI threat landscape
- Enterprise architecture: where AI meets the attack surface
- Business perspective: why this is now a board-level risk
- The human dimension: shadow AI and the insider risk
- Technical expertise: securing models, prompts and agents
- Project methodology: building an AI security program
- International programs: securing AI across jurisdictions
- Why these projects fail
- Executive recommendations
- Executive conclusion
1. Understanding the new AI threat landscape
Generative AI introduces at least four attack categories that did not meaningfully exist five years ago. Prompt injection manipulates a model’s behavior through crafted natural-language input, sometimes hidden inside a document or web page the model is asked to read, causing it to ignore its original instructions. Data leakage through model outputs occurs when a model trained or grounded on sensitive data reproduces fragments of it in a response to an unauthorized user. Model and supply chain risk arises when an organization relies on a third-party model, plugin or fine-tuned checkpoint whose provenance and integrity cannot be fully verified. Agentic overreach happens when an autonomous agent, given broad tool access, is manipulated into taking a harmful action — approving a transaction, exfiltrating a file, modifying a record — without a human ever approving that specific step.
What makes these risks structurally different from traditional cybersecurity threats is that they exploit the model’s core function — following instructions expressed in natural language — rather than a coding flaw. A firewall does not stop a well-crafted prompt. An antivirus does not detect a model quietly leaking a customer record it was never supposed to disclose.
The point for an executive to retain: securing generative AI is not an extension of traditional application security. It requires a distinct set of controls, built around the specific ways language models can be manipulated, not around the vulnerabilities traditional security tooling was designed to catch.
2. Enterprise architecture: where AI meets the attack surface
Positioning AI security within a complete enterprise architecture means recognizing that generative and agentic AI touch nearly every layer of the technology stack, each with distinct exposure.
Data layer and RAG pipelines. A retrieval-augmented generation pipeline connected to a poorly governed document repository can surface confidential content to users who were never authorized to see it — a risk directly tied to the data governance principles I detailed in Data Governance in the Age of Agentic AI.
Identity and access management. Every AI agent needs its own identity, scoped to the narrowest possible set of permissions. Treating an agent as a generic, broadly privileged service account is the single most common and most costly architectural mistake I see in AI deployments.
API and integration layer. An agent that writes to a CRM or ERP inherits every validation and consistency constraint of the underlying system — and every gap in those constraints becomes an avenue an attacker can exploit through the agent rather than the system directly.
Model hosting and inference infrastructure. Whether models run on Azure OpenAI, a dedicated cloud provider, or self-hosted infrastructure changes the security perimeter significantly — data residency, encryption in transit and at rest, and the provider’s own security posture all become part of the organization’s attack surface.
Observability and logging. Every prompt, every tool call, every model response should be logged in a way that supports forensic reconstruction after an incident — the same principle that underpins any credible AI governance program.
Existing security stack integration. AI security should extend the organization’s existing SIEM, identity provider and endpoint protection rather than create a parallel, disconnected system — otherwise security teams end up monitoring two disconnected worlds instead of one coherent one.
3. Business perspective: why this is now a board-level risk
AI security has moved from a technical footnote to a board-level agenda item for concrete financial and reputational reasons.
Direct financial exposure. An agent manipulated into approving a fraudulent payment or leaking a customer database represents an immediate financial and legal liability, not a hypothetical scenario reserved for security conferences.
Regulatory exposure is expanding rapidly. The EU AI Act imposes differentiated obligations based on risk level, with stricter transparency, human oversight and documentation requirements for high-risk systems — obligations that are difficult to demonstrate without the security and traceability controls described in this article.
Shadow AI is already a live liability, not a future risk. Employees are already using unapproved AI tools with corporate data, often without malicious intent but with real exposure — a fact that surfaces in almost every honest security audit I have been involved in over the past year.
Customer and partner trust increasingly depends on demonstrable AI governance. Enterprise customers now routinely ask vendors for evidence of how AI systems handling their data are secured, not just whether they exist.
The cost asymmetry favors attackers. A single successful prompt injection or data leakage incident can cost far more in remediation, legal exposure and reputational damage than the entire security investment that would have prevented it — a calculus every CFO understands once it is framed this way.
4. The human dimension: shadow AI and the insider risk
No AI security program succeeds on architecture alone. The human dimension is where most programs, however well designed technically, actually break down.
Shadow AI is a response to unmet needs, not a discipline failure. Employees who use unapproved AI tools with sensitive data are almost always trying to solve a real productivity problem that the organization has not yet addressed through approved channels. Banning the tools without providing an alternative rarely works durably.
The insider risk changes shape with agentic AI. An employee no longer needs deep technical skill to cause serious harm — a well-crafted prompt directed at an internal AI agent with broad permissions can achieve what once required genuine technical intrusion.
Security teams and AI teams often speak past each other. Security professionals trained on traditional threat models sometimes underestimate prompt-based attacks because they do not resemble conventional exploits, while AI teams sometimes underestimate security concerns because they are focused on model performance rather than adversarial robustness.
Training must go beyond a one-time awareness session. Employees interacting with AI-powered tools need to understand, concretely, what data should never be shared with an unapproved model and how to recognize a system behaving unexpectedly — an ongoing effort rather than a single onboarding module.
Executive sponsorship must translate into real budget and authority. An AI security program without a named executive sponsor able to allocate resources and settle disputes between security and product teams tends to stay a set of recommendations rather than an enforced standard.
5. Technical expertise: securing models, prompts and agents
At the technical level, securing generative and agentic AI rests on a distinct set of controls that traditional application security does not fully cover.
Input and output filtering. Systems that screen prompts for injection attempts and screen model outputs for sensitive data leakage before they reach the user or downstream system provide a critical, if imperfect, first line of defense.
Least-privilege access for every agent. Each AI agent should hold the minimum permissions required for its specific task, with time-limited credentials and clear audit trails — the same principle that governs human access, applied to non-human actors whose behavior is probabilistic rather than deterministic.
Segregation between experimentation and production environments. AI systems should be tested in environments where potential damage is contained, with clear promotion criteria before any system touches production data or critical systems.
Red teaming specific to AI systems. Traditional penetration testing does not adequately cover prompt injection, jailbreaking or data extraction techniques specific to language models — dedicated AI red teaming exercises should be part of any pre-production validation.
Continuous monitoring for model drift and anomalous behavior. An agent’s behavior can degrade or change unexpectedly as the underlying model, its context, or the data it retrieves evolve — monitoring must be continuous, not a one-time validation at launch.
Vendor and supply chain due diligence. Understanding how a third-party model provider handles data, what its own security certifications look like, and what contractual guarantees exist around data usage should be part of procurement, not an afterthought.
Useful reference frameworks include the OWASP Top 10 for Large Language Model Applications, the NIST AI Risk Management Framework, and ISO/IEC 42001 for AI management systems — each offering a structured starting point rather than a definitive checklist.
6. Project methodology: building an AI security program
Building an AI security program requires a structured approach distinct from a traditional security project, precisely because it touches architecture, governance and behavior simultaneously.
Honest discovery of existing AI usage. The first step is mapping every AI system already in use, including shadow AI — an exercise that almost always reveals more usage than leadership expected.
Risk-based prioritization. Not every AI use case carries the same risk. A formal grid crossing data sensitivity, autonomy level and business criticality prevents security investment from being allocated based on visibility rather than actual risk.
Baseline controls before expansion. Establishing least-privilege access, logging and basic prompt filtering across existing AI systems should precede any expansion to new use cases, rather than being retrofitted after incidents occur.
Formal criteria for production promotion. No AI system should reach production without documented adversarial testing results, defined escalation paths, and verified logging — criteria that should be as non-negotiable as any other security gate.
Agile delivery with security built into each sprint. Security requirements should be part of the Definition of Done for AI features from the start, not a separate review inserted at the end of development.
Structured incident response specific to AI systems. Playbooks for AI-related incidents — a data leak through a model response, a manipulated agent action — should exist before an incident occurs, with clear roles for security, AI, and legal teams.
Continuous improvement as a standing practice. A monthly review of AI security metrics, new attack techniques, and emerging regulatory requirements keeps the program current rather than static.
7. International programs: securing AI across jurisdictions
Organizations operating across multiple countries face additional complexity when securing their AI deployments at scale.
Regulatory fragmentation. The EU AI Act, evolving US state-level AI regulations, and data protection regimes that vary by jurisdiction create a patchwork that a security program must absorb without multiplying uncontrolled local exceptions.
Data residency constraints shape architecture choices. An AI system that must respect data residency requirements in multiple jurisdictions cannot always route all requests through a single centralized model endpoint — requiring regional hosting or careful data filtering by geography.
Uneven security maturity across subsidiaries. A headquarters with a mature AI security program often finds that subsidiaries use unapproved AI tools locally, simply because the central program has not yet been deployed there.
Incident response across time zones. An AI security incident spanning systems deployed across continents cannot follow the same escalation rhythm as a single-site operation — requiring clear 24/7 coverage or well-defined handoff procedures.
Shared centers of expertise. Pooling scarce AI security expertise into a shared services model, rather than duplicating it unevenly across countries, accelerates maturity while still allowing local teams to adapt to regional regulatory specifics.
8. Why these projects fail
The real causes of failure in AI security programs are more specific than the generic explanations usually offered.
Security is addressed after AI deployment, not before. Use cases are already in production when the security program starts, turning what should have been a design exercise into a costly retrofit.
Traditional security tooling is assumed to be sufficient. The organization applies existing application security controls to AI systems without recognizing that prompt injection and model manipulation require distinct detection mechanisms.
Shadow AI is discovered, not prevented. The security program learns about unapproved AI usage during an incident rather than through proactive discovery, leaving no time to remediate before damage occurs.
Least-privilege access is never actually enforced for agents. Agents are granted broad permissions for convenience during development, and those permissions are never tightened before production deployment.
No formal criteria exist for production promotion. Systems reach production because they « seem to work, » without documented adversarial testing or defined escalation paths.
Security and AI teams operate in silos. Without a shared vocabulary and a joint governance structure, security concerns are dismissed as overly cautious and AI risks are dismissed as theoretical, until an incident proves otherwise.
Vendor risk is never assessed. The organization adopts a third-party model or AI tool without any due diligence on its data handling practices, discovering the gap only when a customer or regulator asks.
9. Executive recommendations
Map shadow AI honestly before building new controls. You cannot secure what you do not know exists — a proactive discovery exercise should be the first step of any program.
Treat every AI agent like a new employee with the narrowest possible access. Least-privilege access, scoped credentials and clear audit trails should be non-negotiable from day one.
Build AI-specific red teaming into your validation process. Traditional penetration testing does not cover prompt injection or jailbreaking — dedicated exercises are required.
Formalize production promotion criteria before the first incident, not after. Documented adversarial testing and verified logging should be as mandatory as any other security gate.
Give security and AI teams a shared vocabulary and governance structure. Joint training and a shared risk framework prevent the silos that let real risks fall through the gaps.
Assess vendor and model supply chain risk as part of procurement. Data handling practices and security certifications should be verified before adoption, not discovered during an audit.
Communicate AI security as an enabler of safe adoption, not a brake on innovation. Teams adopt security controls far more readily when they understand these controls are what allows the organization to deploy AI with confidence rather than anxiety.
10. Executive conclusion
Generative and agentic AI have not just created new business opportunities — they have created a new attack surface that traditional cybersecurity programs were never designed to cover. Prompt injection, data leakage through model outputs, agentic overreach and shadow AI are not theoretical risks reserved for security conferences; they are present, active exposures in most organizations already using these technologies.
The organizations that get this right are not the ones with the most sophisticated security tooling. They are the ones that treated AI security as a design requirement from the start, gave security and AI teams a shared vocabulary, and built least-privilege access and continuous monitoring into every AI system before it reached production — not after an incident forced the issue.
AI does not just need to be secured. It needs to be secured differently.
Frequently Asked Questions
What is prompt injection?
Prompt injection is an attack technique that manipulates an AI model’s behavior through crafted natural-language input — sometimes hidden inside a document or webpage the model processes — causing it to ignore its original instructions or leak information it should not disclose.
Does traditional cybersecurity tooling protect against AI-specific risks?
Only partially. Firewalls, antivirus and traditional application security controls do not detect prompt injection, model data leakage, or agentic overreach, which exploit the model’s core function rather than a coding vulnerability.
What is the biggest AI security risk organizations underestimate?
Shadow AI — employees using unapproved AI tools with corporate data — is consistently underestimated because it rarely triggers a technical alert, yet it represents one of the most common and costly exposures in practice.
How should least-privilege access apply to AI agents?
Each agent should hold only the permissions strictly required for its specific task, with time-limited credentials and full audit trails — never a broad, generic service account granted for convenience.
Where should an organization start if it has no formal AI security program today?
With an honest discovery exercise mapping every AI system already in use, including shadow AI, followed by risk-based prioritization before building new controls.
Expert Perspective
Having built and deployed my own generative AI platform in production with Diagnoz® — including tool-use orchestration, cost controls and access gating — I have learned firsthand how easily an AI system’s behavior can diverge from what its designers intended, even without any malicious actor involved. Refactoring a single monolithic API call into sequential steps to respect hosting constraints, or replacing an « unlimited » access tier with explicit monthly caps, were not security decisions in the traditional sense — but they were exactly the kind of engineering discipline that prevents an AI system from behaving unpredictably at scale.
The organizations that get AI security right are not necessarily the ones with the biggest security budgets. They are the ones where security and AI teams sat in the same room early enough to design constraints together, rather than security being asked to review a system after it was already built and deployed.
The most consistent lesson I take from this: an AI system is only as secure as the assumptions its builders never questioned. Naming those assumptions early is worth more than any tool added after the fact.
Take Action
If your organization is deploying generative or agentic AI and you want an independent assessment of your security posture, I would be glad to discuss your specific context.
Book a strategic session with Notoriti — or explore our other analyses on AI and enterprise data governance in the Notoriti Knowledge Center.
References
- OWASP — Top 10 for Large Language Model Applications: https://owasp.org/
- NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
- ISO/IEC 42001:2023 — AI Management System: https://www.iso.org/standard/81230.html
- EU Artificial Intelligence Act: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
