LLM Security Triage with Deterministic Guardrails

The problem: signals arrive without a narrative

Runtime security monitoring produces discrete, low-context facts. “A new actor sent data externally.” “Service A now talks to service B.” “A database permission changed.” Each is true in isolation and nearly useless alone. An engineer looking at forty rows like this doesn’t want more rows. They want to know what it is, whether it’s bad, and what to do about it. That’s a narrative, and raw signals don’t carry one.

This post starts after that signal stream exists; getting there is covered in runtime identity attribution with eBPF, IAM, and CloudTrail and why an attribution graph is only as good as its evidence.

Turning signals into a narrative is exactly the step a language model is good at. It’s also where letting a model speak directly to a security engineer is most dangerous. A fabricated “malicious” incident costs trust the first time it’s investigated and found empty, and gets the whole feature muted the second time. Once a team stops trusting triage output, they stop reading it, and the tool is dead regardless of how good the underlying signals are.

A single model call that groups, classifies, and narrates leaves no checkpoint between inference and verdict. A wrong interpretation can become a malicious label before deterministic code has a chance to reject it.

The design requirement is to use model interpretation without granting the model authority over the verdict.

Why this matters

OWASP LLM09:2025 Misinformation covers two relevant failure modes: hallucination, where a model “generates content that seems accurate but is fabricated,” and overreliance, where users “place excessive trust in LLM-generated content, failing to verify its accuracy.” NIST’s Generative AI Profile (NIST AI 600-1) lists confabulation among its twelve generative-AI risk categories.

Security triage is unusually fertile ground for both. The output is read under time pressure, it arrives pre-formatted as a verdict, and the reader often has no independent way to check it in the moment. The interface itself invites over-reliance, and the more fluent the narrative, the stronger that invitation gets.

Prompt instructions alone do not provide enforceable constraints. The design therefore uses checks that hold regardless of what the model returns. Where no such check exists, the limitation is stated directly rather than hidden behind prompt wording.

The design: deterministic constraints around model output

Model calls handle grouping, inference, and phrasing within predefined output structures. Deterministic code validates what those outputs may become.

Four sequential steps run on a 4-hour cron per company:

runtime evidence (anomalies, service-to-service risk, permission deltas, actor purpose)
  → normalized signal stream
  → Step 1  (LLM) grouping + treatment
  → Step 2a (LLM) behaviour profile
  → Step 2b (LLM) classification        ← locked from here on
  → Step 3  (LLM) evidence story, high-priority only
  → user-visible incident

What “guarded” means varies across those steps, and describing it as uniform would be the same kind of over-claim this post argues against. JSON Schema runs on all four, rejecting malformed shape and constraining closed-value fields. Hand-written structural validators run on three of the four, encoding cross-field invariants a schema can’t express. The profile step is the exception. Its main protection is a gate that runs before its model call, rather than a validator afterwards.

Step 1: grouping without inventing

One call bundles the window’s signals, up to a hard cap I’ll return to in the limitations. Each actor’s signal group becomes a candidate pack, and the model’s only lever here is a treatment value from a fixed set: focus_now escalates the pack into the later steps, while review, group_only, and suppress are all terminal, differing only in whether the pack is kept for inspection, counted toward totals, or dropped as noise.

Pack identity is not something the model names:

pseudocode

pack_key = hash(run_identifier + sorted(all signal identifiers in the pack))

# when a pack carries no traceable signal identifiers, fall back to
# hashing the run identifier together with the pack's name and anchor

A rerun of the same window therefore produces the same key rather than duplicate work.

The second rule at this step works differently. Narrative fields are supposed to describe an actor, action, asset, and destination in plain language, keeping internal identifiers in separate machine-readable fields. That rule lives only in the prompt. No validator enforces it, so the pipeline cannot guarantee that an internal identifier never reaches narrative output. It is a convention we request rather than an invariant the system holds, and treating it as enforced would be exactly the mistake this post is about.

Step 2a: what the app is for, in two shapes

This step produces two outputs from one call: a full behaviour profile that the classify and narration steps consume, and a compact, product-facing intent summary served on the application page. The audiences differ. One is analyst-grade context for a later decision; the other is a short human-readable answer to “what does this app do.”

"application_intent_summary": {
  "intent":        "short Title Case name for the workflow",
  "what_app_does": "a single sentence",
  "expected_to":   ["action-verb bullets"],
  "details":       "a short paragraph, phrased tentatively"
}

Presence and structure are schema-enforced; the finer stylistic constraints live in the prompt, so prose that runs slightly long degrades gracefully rather than failing the build. Schema for what must exist, prompt for how it reads.

Before any of this runs, a minimum-activity gate checks whether the actor has enough distinct days of history. Below the threshold, the step returns an insufficient-evidence result without calling the model at all, and the frontend waits for more data rather than receiving a guess dressed as a conclusion.

That gate is this step’s main protection, and it is worth being clear what that implies: this is the one step with no post-output structural validator. Its output is shape-checked by schema, and its consumers apply their own validators to what they build from it. Nothing here asks whether the profile contradicts itself. Refusing to profile thin evidence is a weaker protection than the ones downstream, and it’s where I’d add the next validator.

Step 2b: the classification is a closed set, and it has to be earned

Classification is a closed seven-value set: malicious_intent, purpose_drift, misuse, expected_behavior, low_signal_noisy, insufficient_evidence, needs_scope_validation. Schema constrains the field to those values, so the model selects among predefined options and cannot introduce a new one that passes validation.

A closed set is weak on its own. It stops invented vocabulary; it does nothing to stop the model reaching for the most dramatic available label. So the validators here also check that the evidence offered supports the label chosen, and they are strictest about the two that matter most:

A response labelling something malicious_intent while offering neither fails validation. That is the difference between an encoded threshold and a prompt asking the model to be sure.

This classification then becomes authoritative for everything downstream. The narration step inherits it and has no route to revise it.

Step 3: narrating evidence without rewriting the verdict

Narration runs only on incidents that are both high priority and already evaluated, producing a structured evidence story: a top card, one of three evidence shapes, a blast-radius summary, and a set of remediation options.

On screen, the analyst sees a short statement of what happened and why it was escalated. The evidence chain follows as discrete entries, each tagged with the kind of signal it represents: destination risk, data sensitivity, or policy violation. The rule against duplicate signal types keeps that list from repeating itself, so three entries mean three different reasons rather than one reason phrased three ways.

The clearest constraint is shape. A story declares a type, and that type dictates which evidence container must be populated and which must be empty. A drift story carries a mismatch chain and nothing else. A temporal story carries a timeline of more than one event and nothing else. An independent story carries a stack of separate corroborating signals and nothing else. A story claiming to be temporal with an empty timeline is rejected regardless of how convincing its prose is.

The second constraint is the classification lock. Before validation runs, code stamps the incident’s decided classification, priority, and execution mode onto the story. If the story diverges from what was stamped, that indicates the caller failed to stamp it correctly. The model’s disagreement carries no weight here.

The remaining validators check story type, evidence shape, remediation structure, summary shape, the locked incident fields, and duplicate evidence types.

What happens to a story that fails validation

A rejected story is never persisted to the incident's narrative field or exposed through product read paths. What remains is the incident record the classify step already created, carrying the specific validation errors, tagged by which step produced them. One qualification matters here, because an earlier draft of this post got it wrong: schema validation and the response-cache write both happen inside the LLM client, before the structural validators run in the caller. So the schema-valid raw response may also remain temporarily in the response cache until it expires. It is never served to a user, though claiming it exists nowhere would be inaccurate.

Keeping the errors rather than dropping the record is deliberate. A silently dropped record is a debugging dead end, while retained violations make the failure rate countable per validator: which one trips, and how often. A rejected story also degrades to no story instead of a wrong one, which is the behaviour you want when the alternative is showing a security engineer something incoherent.

Prompt-injection limits

These controls do not prevent prompt injection. Runtime evidence carries attacker-influenced strings by construction, including application names, table names, and destination hostnames, and some of those are attacker-chosen in exactly the scenario the product exists to detect. That is the setup for indirect prompt injection.

What the constraints bound is the consequence. An injected instruction cannot produce a label outside the closed set, unlock a classification an earlier step already fixed, or surface a structurally incoherent story. What they do not do is sanitize model input or prove semantic correctness: an attacker who successfully steers the model can still influence grouping, phrasing, and the choice among legal classifications.

Enforced properties and remaining limits

These constraints enforce specific properties without guaranteeing correct interpretation. Output cannot carry a label outside the set, claim the two strongest labels without the required evidence structure, revise a decision an earlier step owns, or reach a user as a story whose declared type contradicts its contents.

None of this removes the human. Triage output opens a review rather than closing one. The pipeline's job is to make sure that what reaches a reviewer is internally coherent and honestly labelled, leaving the conclusion to them. A validator can tell you that a story contradicts itself; it cannot tell you whether the story is true, and the design assumes someone will still ask that question.

I'm not putting numbers on any of this. A validator-failure rate means something only once it's been measured over a long enough window with the method written down, and quoting a soft figure would undercut the argument of a post about not over-trusting confident-sounding output. Retaining the violations is what makes proper measurement possible later.

Three implementation limits remain. Step 1 truncates a busy window's signals, so tail signals can be lost before grouping ever happens. Permission changes derive from a last-modified timestamp rather than a dedicated audit trail, so we can see that a permission changed but not reliably who changed it. Marking an incident as expected also doesn't yet suppress future similar ones.

These guardrails limit the consequences of incorrect output without making the model correct.

Key takeaways

Common questions

Related reading

• Runtime Identity Attribution with eBPF, IAM, and CloudTrail: how six data sources get joined so a runtime signal can name the workload, identity, and data behind it.
• The Attribution Graph Is Only as Good as Its Evidence: why the grade of the underlying telemetry decides whether an identity graph can be trusted.