blog/llm-guardrail-tools-not-interchangeable-nemo-llm-guard-llama-guard-hero
← Back to blog

LLM Guardrail Tools Are Not Interchangeable: What Each One Actually Catches

Technology•17 November 2026•TREEO Indonesia•5 min read

Search for an LLM guardrail and you meet a long list of names: NeMo Guardrails, Guardrails AI, LLM Guard, Llama Guard, Lakera Guard, Azure Prompt Shields, OpenAI's moderation endpoint. They are often compared as if they were competing versions of one product. They are not. Comparison articles written in 2026 keep returning to the same warning: these tools are different kinds of control, and picking the wrong kind means fighting it for months. This guide sorts them by what they actually do, so a team can ask the right question before choosing one.

Guardrail tools in numbers

4

distinct shapes among the open-source projects: a rails language, a validator library, a classifier model and a scanner pipeline

9 July 2026

date one source gives for the LLM Guard repository being archived and made read-only

5

rail types in NeMo Guardrails: input, dialog, retrieval, execution and output

14

hazard categories in the Llama Guard 3 taxonomy, which does not include prompt injection

Four shapes of open-source guardrail

Practitioner comparisons describe four archetypes. They are not competing implementations of one idea.

ProjectShapeLicenceStatus in the sources
NeMo GuardrailsProgrammable runtime: five rail types steered with the Colang languageApache-2.0Actively maintained by NVIDIA
Guardrails AIValidator library with a hub of validators, typed output via Pydantic or RAIL schemasApache-2.0Maintained; one comparison lists a change of ownership in September 2026
Llama Guard 4A single 12-billion-parameter safety classifier for text and images, scored against 14 hazard categoriesLlama Community License: open weights, not an OSI open-source licenceActively maintained by Meta
LLM GuardPipeline of roughly 35 chained input and output scannersMITRepository archived about 9 July 2026

The licence column matters more than it looks. Apache-2.0 and MIT are permissive. The Llama licence allows use of the weights but carries its own terms, so legal review is a different exercise.

NeMo orchestrates conversation flow and tool calls. Guardrails AI is aimed at applications that must return validated, typed data. One comparison puts it bluntly: Guardrails AI catches a malformed JSON schema, but it will not catch a prompt injection, and confusing the two is how teams ship a validated response that is also a data-exfiltration route. Llama Guard is a model, not a framework: it classifies content. LLM Guard historically covered prompt injection, personal data, toxicity, secrets and code detection, composed by choosing which scanners run and in what order.

What a taxonomy mismatch looks like

Different tools answer different questions, and a useful illustration comes from a recent preprint benchmarking safety classifiers. In that study the small Llama-Guard-3-1B model scored an F1 of zero on jailbreak corpora. The authors explain this is a taxonomy mismatch rather than a malfunction. Llama Guard 3 is a content moderator with 14 hazard categories, and prompt injection is not one of them. Most jailbreak prompts ("ignore previous instructions, pretend you are...") are not harmful on their surface, so the classifier correctly calls them safe. A content-safety guard does not stand in for an injection guard. A related architectural point appears in one comparison: it states that Meta places a separate prompt-injection classifier in front of Llama Guard because a language-model guard is itself injectable.

The archived repository

LLM Guard is the cautionary tale of the group. Sources report that Palo Alto Networks completed its acquisition of Protect AI on 22 July 2025, and that the llm-guard repository was later archived, with the repository described as read-only from 9 July 2026. One comparison that predates this recommended LLM Guard as the gentlest starting point; another source argues it should no longer be adopted. If you are reading an older "best guardrails" article, check the repository status before you read the feature table. Maintenance is a property of the project, not of the comparison page, and it changes.

Cost deserves the same scepticism. Every model-based check is an extra inference on top of the real model call. In NeMo's common self-check configuration, one source notes, the guardrail doubles model calls on allowed traffic, while blocked messages cost one call because the refusal is canned. NeMo has since added parallel rail execution and caching to reduce the penalty. Regex and deterministic validators are cheap but brittle.

None of the three open-source frameworks compared in one source ships a built-in monitoring dashboard. Events must be piped into whatever observability stack you already run; NeMo exposes OpenTelemetry and an OpenAI-compatible endpoint, which helps.

Hosted APIs: different design, different blind spots

ServiceWhat the sources say it targetsFree tier as reported
Lakera GuardPrompt injection, jailbreaks and system-prompt extraction, plus PII and content checksSources conflict: a comparison table lists a free self-serve tier, a profiling site says no free plan
Azure AI Content Safety, Prompt ShieldsSeparates direct attacks (jailbreaks) from indirect ones (injection through documents); four harm categories scored 0 to 6Free tier with a hard transaction cap and no overage
OpenAI omni-moderation-latestCategory classifier for text and images: hate, harassment, self-harm, sexual, violence, illicitFree for OpenAI API users

The only head-to-head latency figures in the sources are Azure at 52 ms and OpenAI moderation at 192 ms. These are a third party's report of TrueFoundry benchmarks, not a first-party measurement, and nothing comparable was found for Lakera. A related report from the same benchmark family found no single provider winning every task type: one led on injection recall, another on PII precision, another on content categories.

Two further names appear in gateway plugin lists: Google Model Armor and Cloudflare AI Gateway. The sources did not document their capabilities well enough to compare them, so treat them as unassessed here rather than as inferior. Data-residency terms for every hosted service were also missing from the sources. If prompts contain personal or sensitive information, read each vendor's current terms and ask where processing happens.

The gap packaged tools leave

One point recurs in the sources: a packaged tool ships a public taxonomy, such as toxicity, PII or the MLCommons hazard categories, or a generic injection pattern. An organisation's own refund rules, prohibited topics or escalation thresholds are not in the box. Some tools are configurable, since NeMo's Colang can express custom flow and policy rules and validators can be written by hand, but someone still has to write and test those rules. A tool can block harmful content and still fail to answer whether a response was supported by the right evidence, for the right user, at the right time. Treating a single moderation API as a complete defence is described in one source as the most common failure mode.

So the practical sequence is: write the policy in plain language first, decide which parts are content safety, which are structured output, which are injection and which are business rules, then choose a tool for each part and write the business rules yourself. Layering is the recommendation in nearly every source. Whatever you choose, test it on your own traffic: see how to measure a guardrail for the method, and why prompt injection needs architectural defences for what filters cannot do.

Frequently Asked Questions

Not according to the comparisons reviewed. The tools differ in kind: conversation control, output validation, safety classification and scanning. Sources recommend layering them and writing your own business rules, in a configurable tool or by hand.

Turn climate goals into a verified portfolio

TREEO connects the full carbon cycle — eligibility, simulation, real-time monitoring and registry-ready reporting — combining expert consulting with dMRV technology so the evidence exists before anyone asks for it.