Agent Security and Agentic Security Are Different Things

Inventory

An exhaustive inventory goes a long way in reasoning about security. It is very foundational in this regard and a coverage gap in this step may often show up as a missed diagnosis later on in the pipeline.

• How does it differ from service inventory? A service inventory can often be determined from its config file or deployment definitions (like Helm Charts). A quick check with the person owning the pipeline usually gives the right answer. AI agents, however, can be spawned ad hoc either by a developer to perform data analysis, a SaaS tool to manage deployment, or even by another agent. These are not always declared and can run through their entire life-cycle without anyone knowing.
• Declared inventories are transient. A config file and a dependency manifest can only describe deployment intent. For AI agents, it doesn’t tell you which workload called which model, with what credential and at what time. Without runtime evidence, one cannot identify shadow agents (undeclared but active) or zombie agents (declared but inactive). Our article on AIBOM goes through this in detail.
• Beyond the code. Static code analysis only goes so far in terms of inventory. The capabilities of an agent extend far beyond that. It should also include all tools used, MCP servers called, and other agents invoked. An agent can pick up a whole set of artifacts during its runtime. An exhaustive inventory should be able to list all of them.
• Classification after listing. An agent, an MCP client and an MCP server have different blast radii when they’re compromised. Merging all of them into a single “AI app” label doesn’t do much to make the list actionable. Having this classification early on helps narrow down the list of possible outcomes in case something goes wrong.

The recent ClawHavoc incident is an interesting example here. As reported by Trellix, the public skill registry ClawHub was poisoned with over 350 malicious skills carrying stealer payloads and indirect prompt injections. Package-focused inventories (traditional SBOMs) can miss skills loaded at runtime. On top of that, organizations may not have concrete policies with respect to skill usage and which channels they are acquired from. This is made worse by the fact that a user told by an agent to run a complicated script will most likely run it. Due diligence nowadays has been replaced by the convenience of pressing Enter on the keyboard.

Identities

A lot of the work that AI agents do is often described as “on behalf of” the human. As a result, these agents use delegated credentials to achieve their goals. This works reasonably well for your local Claude or Codex sessions. However, the same practices do not translate well when deployed to production.

An example of that is using shared access tokens, like Slack user tokens, GitHub PATs, etc. For actual systems, especially when automations kick in, it becomes difficult to attribute an action back to its source by investigating the separate pieces. Multiple agents might be using a common PAT to achieve their goals. Agent development has been so rapid over the last few years that the identity mechanism to support an agent fleet is still not widely in use.

Identity Attribution becomes very important in such scenarios. The current industry standard has various takes on agent identity:

• Identity providers extending to agents. One way is to give first-class identities to each agent and trace that throughout its activity cycle. Okta Agent SSO went GA recently and is a step in the right direction for solving this problem.
• Protocol-based auth mechanisms. Cross-App Access (XAA), the underlying protocol behind Agent SSO, has been incorporated as the Enterprise-Managed Authorization mechanism for MCP. This is more future-proof and vendor-agnostic. Similar ideas can be transferred to other protocols that may come up in the future.
• Beyond IDPs. The space is not just IDPs: non-human identity governance, secret-less actions and workload identity federation also bring interesting ideas into the mix. SPIFFE is the open standard in this regard and can be used for both AI and non-AI systems.

However, evidence-backed identity chains are always more useful to reason with instead of saying “the agent did it”. Check our article on Runtime Identity Attribution which goes deeper on this. We also discuss different tiers of evidence quality in AI Agent identity attribution here.

Permission Rightsizing

When we talk about permission management, we are usually faced with two major questions:

• What was granted but never used? A simple enumeration of all the permissions granted to an AI Agent. Either via an AWS IAM role or via a database user with elevated permissions. Then validate that set against actual usage patterns. Then trim the difference to reduce the attack surface. Each permission is different from the other in terms of potential risk it poses, and should be handled accordingly.
• What changed? This speaks more of the agent evolution over time. Some AI agents start with very innocuous job descriptions like “send a weekly email summarizing all on-call alerts”. Then over time they end up having bloated permissions that rarely fit the task. Each extra added permission adds new scope of risks especially when used in tandem with the existing permission set. Tracking such evolution over time is necessary to kill the problem before it becomes unmanageable.

For answering the first question, it’s important to rely on observed behavior rather than declared permission sets. Rightsizing against the policy document means depending on the same artifact which produced the over-grant. Someone auditing every action of the identity with finer granularity is much more useful here for identifying the working permission set.

Behavioral Analysis using LLMs

Fixed rulesets and policies can only detect what has been predicted by someone. LLMs can help figure out patterns and anomalies which were never encountered before. It does not, however, give the right to act. These findings can either be discarded or incorporated into the fixed ruleset for future detections.

Another area where AI agents can work remarkably well is finding relations across different layers of activity. An identity appearing across multiple unrelated applications, a new prompt leading to a new tool call, a sensitive read followed by an unusual transfer. Each such event might be harmless on its own, but correlating them to paint the larger picture is something an AI security agent can help with.

Something to mention here is the false positive rate. It’s necessary to strike a balance between bumping up the precision of these models and letting them loose to find whatever they can. Having too many false positives early on can be overwhelming for the human engineer to prioritize and act on.

The AI SOC

The AI agent output must be able to give enough information for the security engineer to act on:

• what happened?
• what is the cause?
• what evidence supports the finding?
• what is still unknown?
• what are the next steps?

The agent should be explicit about the gaps (unknowns) in the current investigation without over-claiming it. That helps the engineer take necessary steps while accounting for the missing pieces.

AI agents, as of now, cannot act as completely autonomous SOC engineers. Reading telemetry, correlating actions and explaining are cheap operations to get wrong and reverse. Blocking a user or shutting down an application is a much more expensive operation and should require human intervention before going through. With time, this boundary should move as trust develops and LLMs get better. The SOC team should be able to decide what responsibilities can be delegated to an agent as the boundaries shift.

These constraints, however, limit what the agents can actually report. They exclude the truly out-of-the-box findings from ever appearing for the sake of lowering false positives. One way to incorporate this is by having multiple tiers of agents responsible for different levels of incidents. A low-priority exploratory agent can have more of a free rein and report incidents with a lower frequency (weekly) for the SOC team to investigate. Relevant findings from that research can then be added to a higher-tier agent for future occurrences.

Intent Analysis and Drift

This is where LLMs can provide real value over standard security tools.

We should define a few key terms first:

• Declared purpose. This belongs in the same category as declared inventory or policy. For AI agents it can be an AGENTS.md file, or an initial prompt which specifically states what the AI agent is supposed to do.
• Observed intent. The declared purpose doesn’t take into consideration the runtime behavior. This can paint an entirely different picture and needs to be taken into consideration for any kind of drift analysis in the future.

When it comes to observed intent, the baseline period is critical. We define drift to be a departure from observed behavior. If the baseline period itself has some compromise, it will be encoded in its baseline and the security agent will defend that behavior forever. Human intervention and labeling are necessary here to prevent such scenarios.

A few things need to be taken into consideration while building a drift detection engine:

• Statistical and semantic drift. Departure from one’s own historical patterns is statistical drift and can be obtained by comparing distributions. Straying from its declared purpose is semantic in nature and requires a language model or human intervention to deduce. The first one is cheaper to detect but the second one is more interesting.
• Two-step pipeline. The reasonable middle ground to detect intent drift without throwing all telemetry data to an LLM is to use the statistical filter as a first step. Use the distributional comparison (cheap) to find candidates, then send the candidates to an LLM adjudicator (expensive) for final output.
• Baseline quality. The output of this kind of system depends a lot on the quality of the baseline. A good baseline learned from observed behavior and the declared purpose are good inputs for an intent drift system. We discuss this further in the article on Runtime Intent Drift.

History surfaces change; an approved purpose gives semantic adjudication a reference.

Intent analysis and drift are both examples of agentic security applied to agent security. There is a certain level of non-determinism involved and scope for the unpredictable to happen. That raises the obvious next question.

Guardrails that do work

We’ve talked about LLM guardrails in a previous article; a lot of them apply here:

• Model should not compose data access. The design should be made in such a way that it does not rely on a model generating direct SQL queries that it can run. Prompt injection can steer model-generated SQL towards unauthorized reads.
• Keep read surface enumerated. Usually, for an AI agent to give usable output, it needs read access to a lot of data sources (S3 bucket, OpenSearch logs, database, etc.). Having an approved list of such sources is required here, so that an open question doesn’t lead to the agent accessing a data source beyond what is expected.
• Model output should be treated as untrusted evidence. Whatever output is given by a model should not be treated as the gospel. Human validation post model output is almost always required.
• Use a scoped, short-lived identity. Discussed briefly in the identity section. Applies to AI security agents as well.

Limiting non-determinism

The guardrails mentioned above are just examples of the general principle to keep non-determinism away from expensive irreversible steps. That way, you get the good parts of AI (the research and wide net approach) without falling victim to its shortcomings.

The deterministic parts of the system should do the most work before passing on the curated data to the LLM. Cheap steps like deduplication, normalization, statistical filtering can reduce the sample to a few interesting candidates. Then a strict prompt to the LLM with validated input-output patterns would reduce the possibility of guesswork in the process.

The agent should be given strict instructions (in the form of prompts or code validations) to withhold conclusions when proof is insufficient, instead of presenting a made-up conclusion. That way we can reduce further mistakes down the chain which can be dependent on said conclusion.