AI Runtime Security Tools

How we score AI runtime security tools

One research plan covered all 16 vendors on 16 Nov 2026 and produced 1,469 facts across 32 vendor files, each checked against a quote. One scorer scored blind, a second blind reviewer checked, and weights were committed before any evidence or score existed.

Facts read on .

The weights

The weights were fixed on , before any score existed, and have not changed since.

Criteria and weights (out of 100)
CriterionWhat it measuresWeight
Action types stopped before they completeWhich harmful actions it stops before they complete, named: shell command, file read or write, data paste or upload, package install, network call, credential access, unsafe tool call, prompt injection or harmful model output30
Layers coveredWhich layers it enforces at, as documented: model (prompts and outputs), application (an AI app's requests), agent (tool calls and plans), endpoint (actions on the machine)20
Latency and failure modePublished latency of an inline decision and the published behavior when the control fails (fail open or fail closed, timeouts)15
DeploymentHow it deploys (API, SDK, proxy or gateway, agent, agentless), what it needs, time to deploy as published15
Detection and responseAlerts, an audit trail of blocked and allowed actions, investigation, and export to SIEM or SOAR5
TransparencyPublic documentation readable without sign-in, a published price, published research10
ProofNamed customers, funding, independent analyst coverage5

Weights by situation

Weights used for each situation view
SituationAction types stopped before they completeLayers coveredLatency and failure modeDeploymentDetection and responseTransparencyProof
Developers run coding agents with shell, file and package access352010151055
A company shipping a customer-facing LLM application25202515555
Staff use agents inside SaaS and cloud workspaces that act on company data30305151055
A regulated company that must prove what was blocked and respond to incidents251510103055
A lean team that must have runtime control in place this quarter25151035555

Score anchors

Rules

What research was done?

A single research plan was applied to every vendor on 16 Nov 2026. The same questions were asked of each: which actions are stopped, at which layers, with what published latency and failure mode, how it deploys, what it records, what is public and who backs it.

The result is 1,469 facts across 32 vendor files, because the files also serve other comparisons; the 16 files for this site hold 896 of them. Each fact holds a claim, a quote, a source URL, a read date and a kind: vendor, third-party, observed or none-found. Every quote was checked against the page it came from.

How does blind scoring work?

One scorer scored every vendor from the evidence file only, working from coded packets with every vendor name removed, so the scorer did not know which vendor was which. One blind reviewer then checked each score against its cited facts. Disagreements were settled by returning to the quote.

Scores are whole numbers from 0 to 10. A score changes only when a sourced fact changes, and the change is logged with the fact.

Why are weights fixed first?

The weights, the five situation weightings and the scoring anchors were committed on 16 Nov 2026 before any vendor evidence or score existed. After the first score, no weight and no criterion changes.

Breadth counts such as module numbers, integrations, headcount and funding inform only the proof criterion, which weighs 5, so they carry at most 10 percent of any ranking.

How are latency and failure mode scored?

Only as published. A millisecond figure on a vendor page, a documented timeout or a stated fail-open or fail-closed default earns credit. A figure seen only in a demo panel is treated as a mention. Silence scores 0 and is written as none found on 16 Nov 2026.

This favors vendors that document their control, which is the point: a buyer cannot plan around a failure mode nobody has written down.

How are errors corrected?

Send a correction with a source to editor@airuntimesecuritytools.com. A confirmed fact change updates the fact, the score and the changelog on the same day, and the old and new values stay visible.

Vendors may supply public documentation at any time. A claim without a public source stays at the claim ceiling of 7.

Changelog