An AI Inventory Built From Runtime Evidence

The problem: declared vs observed

Most teams already have two artifacts that look like an AI inventory and neither serve the complete purpose.

The first is a software bill of materials listing an AI client library as a dependency. While that’s a ground truth, it’s practically useless. The library being present in a build tells you nothing about whether anything called it, from which workload, against which model, or with whose key. Plenty of services carry a dependency they import once in a code path that never runs in production.

The second is a configuration inventory listing declared model endpoints. Also true, also insufficient on its own. It describes intent at deploy time. It doesn’t describe behavior at runtime, and that gap creates some of the risks described here.

This results in two gaps, and they’re worth keeping separate, because they’re found by comparing different things.

Shadow AI compares declared against currently observed. It’s a workload calling a model that no inventory, config file, or architecture diagram mentions. Finding it requires knowing what’s running now and comparing it against what was written down.

Zombie AI compares historically observed against recently observed. It’s a workload with AI activity in its history and none in the recent past. Disclaimer: it does not establish that the credential still works, that a network path is still open, that anyone abandoned the integration, or that the workload stopped running. It says the AI calls stopped showing up, and leaves the reason to whoever picks up the record.

They are different comparisons over different evidence: one against declared state, the other against a workload’s own past. A written inventory cannot detect when it has drifted from either.

An inventory built from observed behavior can be incomplete or ambiguous, but every entry is anchored to something that was actually seen. When an entry stops matching recent behavior, the zombie flag records that change and makes the evidence gap visible.

This post explains how the inventory is constructed. The closing section separately covers the limits on trusting an individual edge.

What the record holds

Before the mechanism, the artifact. Each workload in the inventory carries:

Identity: the cluster, namespace, and the service its pods group into. Deliberately not the pod IP. Pods are ephemeral; an inventory keyed on them is a list of ghosts within a day. IP resolution is a fallback for when nothing better is available.

Component type: whether this is a service calling AI, an agent, an MCP client, or an MCP server. These have different blast radii, and collapsing them into a common “AI app” throws away the distinction that makes the row actionable.

Destinations: the external endpoints reached and the named models used on those calls.

Data path: database protocols and how many distinct destinations were seen on each, plus internal service and third-party links. This records where the workload was observed to go, not everything it was permitted to reach; the next section explains why it’s in an AI record at all.

Volume: call counts and data volume, recent and prior. The comparison shows the direction of travel needed for a decision.

Zombie flag: seen before, not seen recently.

Ownership: a credential fingerprint resolved to owner metadata.

Review status: whether a human has looked at this and decided something.

Those last two might seem irrelevant but are doing more work than they appear to. A list nobody owns or has adjudicated isn’t a credible bill of materials.

What classifies as AI?

The promotion rule admits only two types of evidence and rejects several common false signals.

A workload enters the inventory on exactly two kinds of evidence: a destination host recognized as an AI provider, or an AI protocol signature on the request. The signature can come from the protocol itself or a typed entity on either end of the call.

Machine-learning evidence is used only for enrichment. If a workload is already in the inventory, ML-derived component evidence adds detail to its record. It cannot, by itself, pull a workload in. The reason is that the costs are asymmetric. Adding detail to a record that already exists is cheap and reversible. Admitting a workload on a soft signal puts a false positive into an inventory a security team is supposed to trust, and they only need to chase two of those before they stop reading the whole thing.

Ordinary third-party traffic never promotes. Source control, chat, and databases are where a broad rule goes wrong. If every external call is treated as possibly AI, nearly every service becomes “AI-relevant” and the flag loses its value. An exception that we allow here is that users can configure their own custom “ML endpoints”. Once configured they are treated as any other known ML endpoint.

The general principle: a detector’s precision is set by what it refuses. For an inventory a human reviews one row at a time, a false positive costs more than a miss. This trade-off reduces recall. We accept that gap to keep the inventory useful.

on observed_request:
    if ai_provider_host(request) or ai_protocol_signature(request):
        mark_ai_relevant(resolved_workload)     # starts or extends the window

    if is_marked(resolved_workload):
        retain(request)                          # including db, internal, third-party

Zombie AI is a evaluated differently

It would be tidy if the expiring window also gave us zombie detection. But we deliberately kept it separate for several reasons.

The window lives in the collection path, in memory, and it governs what gets forwarded. It is short-lived by design, and it can go quiet after a restart, an eviction under capacity pressure, or a redeploy under a different identity. Treating “no longer marked” as “no longer using AI” would misclassify ordinary infrastructure events as zombie activity.

Zombie AI is derived from persisted history. The inventory query looks at a workload’s stored AI activity, takes the timestamp of its most recent AI event, and compares that against an inactivity threshold. If the event is older than the threshold, the record is flagged zombie.

The window asks is this workload currently in scope for collection? and needs to be short. The threshold asks has this workload’s AI activity been absent long enough to be worth a look? and needs to be long enough that a quiet weekend doesn’t trip it. One value could not serve both.

While the second question surfaces an absence, it does not give an explanation. Whether the integration was retired, the workload was rebuilt under a different identity, or the traffic simply moved somewhere we don’t see is exactly the work the flag hands to a human ( or an AI-SOC agent ).

Two shapes of inventory

The behaviour record above is one view: one row per workload, describing what it did.

The dependency view has one row per detected AI component and lists the workloads where it was observed. It pivots the same underlying evidence to answer: which of our workloads was this component seen in? That is a useful starting point for investigation.

ML component evidence cannot pull a workload into the inventory. Once a workload is in, that evidence produces its component list. Those components are grouped across the promoted population to create the dependency view.

It resembles an SBOM in shape but has narrower coverage. It lists components observed in workloads that were running, not everything installed. A component absent from this view may still be installed and simply never surfaced. A component present in it was seen in a running production workload, which narrows where to look first.

Owner Resolution

An inventory row that says “some workload called a model with some key” isn’t actionable. Someone owns that key, and finding out who is the difference between a ticket that gets closed and one that gets forwarded four times.

What the inventory surface exposes: responses carry a credential fingerprint, which is a stable, non-reversible handle, alongside a masked form of the value rather than the raw token. Cached owner resolutions hold the fingerprint, provider, lookup result, and status. They do not hold the token.

The chain: fingerprint → internal secret lookup → provider-specific owner lookup → cached resolution.

It’s provider-specific because it has to be. OpenAI, GitHub and Slack each expose identity through different APIs returning different shapes, and there’s no generic “who owns this token” call. Each needs its own resolver, which means the set of providers we can attribute ownership for is smaller than the set we can detect traffic to.

Providers with no resolver resolve to an explicit unsupported state, not an empty one. “We can’t answer this” and “there is no owner” are different facts. An empty owner field can look like a resolved answer that happens to be blank. This is also the case where for some providers, there does not exist a resolution method.

Caching follows the same logic, with three outcomes on different clocks. A successful resolution is trusted for about a day; owners change, but not hourly. A failed lookup is retried soon after, because failure is usually a transient API problem rather than a fact about the world. An unsupported provider is parked for much longer, because that answer only changes when we ship a new resolver.

Resolutions are also versioned, so answers cached by an older resolver aren’t trusted indefinitely after the logic behind them changes.

Edge Cases worth noting

Three limits worth stating plainly.

Gateway attribution. AI traffic arriving through a trusted internal gateway is attributed to the backend workload, never the gateway. Otherwise the gateway becomes the most AI-active service in every cluster and the inventory says nothing. When the real source can’t be resolved, the evidence is retained and the attribution is explicitly marked unresolved.

Coverage. Providers and frameworks we don’t yet recognise aren’t in the inventory. Recognition is incomplete and will lag new providers and frameworks.

Inference. Components identified by inference are labelled as inferred, and stay labelled downstream.

How much to trust any individual edge is its own topic, with its own post: The Attribution Graph Is Only as Good as Its Evidence.

Practical use-cases

Policy. A durable record of what AI is in use is the precondition for any statement about what’s allowed. It’s a good starting point for implementing AI usage policies.

Review. The approve/reject decision on a discovered workload makes the inventory usable. The human decision lives in the record rather than in a parallel spreadsheet that is prone to getting stale.

Incident context. When something fires, the expensive part is working out which team owns the workload, which credential it used, and which datasources and destinations it was seen talking to. Those three questions are already answered before the alert exists. The observed traffic ties it all together to give the full picture.