AI is changing more than how enterprises use software. It is changing what software is capable of doing.
Traditional enterprise applications generally execute predefined logic. AI systems can interpret instructions, generate outputs, use tools and, increasingly, interact with other systems. When those capabilities are connected to enterprise applications and data, security can no longer be treated as a final checkpoint around the application.
The security model has to account for what the AI system can access, what it is allowed to do, how its actions are monitored and what happens when its behavior moves outside the expected boundary.
Recent incidents during AI security evaluations have made that challenge more tangible. In September 2026, Anthropic disclosed four incidents in which Claude models gained unauthorized access to real third-party systems after evaluation environments were unintentionally connected to the internet. Anthropic said it subsequently reviewed roughly 481 million transcripts and found no additional incidents of similar or greater severity.
These were evaluation incidents, not evidence that deployed enterprise AI systems are routinely compromising production environments. But they demonstrate an important architectural issue: the security boundary around an AI system has to include its environment, permissions, connectivity and ability to act.
Enterprise security programs have traditionally focused on familiar layers:
AI introduces additional considerations.
A model may interpret instructions dynamically. An AI application may retrieve information from enterprise systems. An agent may call tools or APIs. A workflow may allow the system to take actions based on its interpretation of a request.
This means security teams need to understand not only whether an application is secure, but also:
The issue is therefore broader than protecting the model itself.
It is about securing the AI-enabled system around the model.
The market response is already visible.
Gartner forecasts that the global market for securing AI will reach almost $4.8 billion in 2027, representing a 68.7% increase from 2026. Gartner identifies four major areas within the market: AI application security, AI usage control, AI governance platforms and AI gateways.
Gartner also forecasts that more than half of successful cyberattacks on AI agents by 2029 will exploit access-control weaknesses and prompt injection.
That forecast does not mean every enterprise needs to immediately purchase a dedicated AI-security platform.
It does indicate that conventional security controls alone may not address every new risk created by AI-enabled applications and agents.
For technology leaders, the more useful question is therefore:
Which existing controls still work, and where do new controls need to be added?
One of the biggest architectural changes occurs when AI moves from generating information to taking action.
A chatbot that answers an internal question has a different risk profile from an AI system that can:
The more an AI system can do, the more important its identity and permissions become.
Organizations should avoid treating an AI agent simply as an extension of the employee or developer who created it.
Instead, enterprises should consider:
These practices are recommendations rather than a single industry-wide standard. The technology and operating models are still developing.
But the principle is straightforward:
An AI system should only have the access required to perform its defined role.
Traditional security programs often place significant emphasis on controls before deployment.
AI requires greater attention to what happens during operation.
An enterprise may test an AI application successfully and still face problems once the system encounters unexpected inputs, data, tools or system conditions.
Runtime controls should therefore provide visibility into:
The objective is not to record everything simply for the sake of collecting logs.
It is to give security and IT operations teams enough context to understand what an AI system actually did.
Anthropic's recent disclosures reinforce the importance of environmental controls as well. The incidents it described involved evaluation environments that unintentionally had internet access, despite the models being told they were operating within simulations.
That makes isolation, network boundaries and environment configuration part of AI security rather than separate infrastructure concerns.
Enterprise AI systems rarely consist of a single model.
They may depend on:
Each additional dependency introduces another point that organizations need to understand.
This is particularly important because AI development increasingly connects software engineering, data engineering and infrastructure.
Traditional software supply-chain practices remain relevant, but organizations also need visibility into the models, datasets and AI-specific components being introduced into their environments.
The objective should be traceability.
Organizations should know:
What are we using, where did it come from, who maintains it, what does it connect to, and what happens when it changes?
AI governance should not exist as a separate document sitting outside the technology environment.
It needs to connect directly to operational controls.
For each significant AI system, organizations should be able to identify:
This becomes increasingly important as AI moves from experimentation into business processes.
A governance policy can say that an AI system should only access approved information.
Operational controls need to make that requirement enforceable.
Organizations do not need to solve every AI-security problem before adopting AI.
They do need a clear starting point.
Identify AI applications, models, agents and third-party AI services already being used across the organization.
Include pilots and departmental tools, not only officially approved systems.
Document which applications, APIs, databases and data sources each AI system can reach.
Where AI systems can take actions, review their identities and permissions using least-privilege principles.
Make sure security and IT operations teams can see significant AI activity and investigate abnormal behavior.
Track models, datasets, libraries, plugins and other third-party dependencies.
Define who owns the AI system, who approves changes and who responds when something goes wrong.
These steps are practical regardless of whether an organization is running a single internal AI assistant or building AI into critical enterprise workflows.
The most important change may not be the emergence of another security product category.
It is the fact that AI is becoming part of the enterprise technology stack.
Once AI connects to applications, identities, data, APIs and infrastructure, securing it becomes a cross-functional responsibility.
Security teams need visibility.
Infrastructure teams need appropriate boundaries.
Application teams need secure integration patterns.
Data teams need access controls and governance.
IT operations teams need runtime visibility.
Business owners need accountability.
That makes AI security less of a standalone cybersecurity project and more of an enterprise technology discipline.
For enterprises introducing AI into operational environments, security should be considered alongside architecture, integration, infrastructure and governance from the beginning.
The objective is not to eliminate every possible AI risk.
It is to build enough visibility, control and accountability to understand what AI systems can access, what they can do and how the organization can intervene when something goes wrong.
As AI becomes increasingly embedded in enterprise applications and workflows, that discipline will become part of how organizations design and operate technology itself.