HomeProjectsArticlesPK BoardAbout
Explore Projects

NeMo Guardrails

Active
GitHub Python NOASSERTION

Description

NVIDIA NeMo Guardrails is an open-source toolkit for adding programmable guardrails to LLM-based conversational systems, supporting topic control, safety enforcement, and dialog guidance.

Key Features

  • Programmable guardrails for LLM-based conversational apps
  • Input, dialog, and output rails for comprehensive control
  • Protection against jailbreaks and prompt injections
  • Support for multiple LLMs including GPT-4, LLaMa-2, Falcon
  • Colang programming language for defining rail flows
  • Async-first design with sync and async APIs

Use Cases

💡 RAG applications requiring fact-checking and moderation
💡 Domain-specific chatbots staying on designated topics
💡 Safe LLM endpoints for customer-facing applications
💡 LangChain chains with added safety layer
💡 Enterprise AI assistants with compliance requirements

Strengths & Limitations

✅ Strengths

  • • Actively maintained, recent updates
  • • High community interest (7.0k stars)
  • • Established track record (3 years in production)

⚠️ Limitations

  • • No clear open-source license

Tags

guardrails llm-safety nvidia conversational-ai safety

Categories

📊 Observability ⚡ Agent Tools 🛡️ Security & Guardrails

Quick Start

Install with pip install nemoguardrails. Load a guardrails configuration and create an LLMRails instance. Use rails.generate() to make guarded LLM calls. Supports Python 3.10-3.13 and multiple LLM providers.
Visit GitHub Visit Website View Docs

Related Projects

Guardrails AI

7.3k · Python
Active A+

Guardrails AI adds programmable guardrails to large language models, ensuring reliability and safety through input/output validation, structured data extraction, and custom validators.

guardrailsllm-safetyvalidation +2
  • · Input/Output Guards — detect, quantify, and mitigate specific risk types in LLM applications
  • · Guardrails Hub pre-built validators — RegexMatch, CompetitorCheck, ToxicLanguage and more out of the box
  • · Pydantic structured output — enforce LLM output format via function calling or prompt optimization
View full details → View on GitHub

UQLM

1.2k · Python
Active A+

CVS Health's open-source uncertainty quantification library for language models, providing UQ-based hallucination detection with confidence scoring and mitigation tools to identify and reduce unreliable LLM outputs.

hallucination-detectionuncertainty-quantificationllm-evaluation +2
  • · Five categories of uncertainty quantification scorers: Black-Box (consistency), White-Box (token probability), LLM-as-a-Judge, Ensemble, and Long-Text
  • · Black-Box scorers measure response consistency through multiple generations and comparisons, compatible with any LLM, off-the-shelf
  • · White-Box scorers leverage token probabilities for uncertainty estimation, single-generation scoring with minimal latency and cost
View on GitHub

OpenAI Evals

19.4k · Python
Stale B

OpenAI's framework for evaluating LLMs and LLM systems, providing an open-source registry of benchmarks and tools for systematic model assessment.

llm-evaluationbenchmarkevals +2
  • · Open-source registry of evals for testing different dimensions of LLM performance
  • · Custom eval creation using basic and model-graded templates without writing code
  • · Private evals support for evaluating LLM patterns in your workflow without exposing data
View full details → View on GitHub

Giskard

5.8k · Python
Active A+

An open-source evaluation and testing library for LLM agents providing automated model scanning, bias detection, performance benchmarking, and compliance checks.

evaluationtestingllm-safety +3
  • · Scenario API for creating evaluations that test non-deterministic LLM outputs
  • · Built-in checks including Groundedness, Conformity, and LLM-as-judge assessments
  • · Red-teaming vulnerability scanner generating adversarial test suites across OWASP LLM Top-10 categories
View full details → View on GitHub

Related Articles

安全Prompt InjectionOWASP

Agent Prompt Injection Defense: OWASP LLM01 in Practice

Based on OWASP LLM Top 10 engineering practice, this article systematically explains the seven layers of defense-in-depth for agent prompt injection: input sanitization, instruction isolation, least-privilege, output auditing, guardrails frameworks, continuous red-teaming, and kill switches -- with actionable code and toolchains.

Read more →
security-guardrailsred-teamprompt-injection

AI Agent Guardrails and Red Teaming in Practice: From Rule Engines to Adversarial Evaluation

Five-layer defense plus red-team loop, built on five open-source projects you can copy.

Read more →
NeMo GuardrailsGuardrails AILLM 安全

NeMo Guardrails vs Guardrails AI: Two Technical Routes for LLM Safety Guardrails

Systematic comparison of NVIDIA NeMo Guardrails (Colang-based procedural guardrails) and Guardrails AI (Pydantic-based structured output validation) across protection layers, interception timing, integration, and production maintainability. Includes a decision tree by scenario, scale, and compliance requirements.

Read more →