Screen prompts with Guardrails

A guardrail makes sure that each inference request to the model is secure and safe. You choose the topics to guard against. These topics include unsafe content, jailbreak attempts, and personally-identifiable information.

In Security → Guardrails, you can enable the topics for an AI Studio project. The guardrail then applies to all API keys of that project. You can change the topics at any time. Each change applies immediately to all API keys of the project.

Nebul enables guardrails for your organization. Then you choose which projects apply them, and which topics each project bans. To enable or disable a guardrail, or to change its topics, you must be a project owner or an organization owner. For more information, see Projects.

Why guardrails

If a public endpoint can get to a model, use guardrails. Examples are a customer chatbot, a support widget, or an internal assistant that is open to all employees.

Guardrails can do these tasks:

• Reject harmful prompts before inference, to prevent agent hijacking.
• Protect RAG-enabled endpoints, to make sure that the data available to the model does not leak to a malicious actor.
• Show what the guardrail screened, blocked, and allowed, with a history for each request.
• Trace a rejection back to the request that caused it.
• Keep prompt content private, because the system records only the outcome and the flagged topic.

How it works

The check runs on every request path

A blocked request returns HTTP 400:

{
  "detail": {
    "message": "prompt blocked by guardrails",
    "error_type": "bad_request"
  }
}

Two rules control what the guardrail screens:

• Every user message, in context. The guardrail screens each user message in the conversation, from the first message to the newest message. The guardrail reads each message together with the earlier user messages. For example, the guardrail knows what "How do I kill all of them?" refers to. The guardrail does not screen the system prompt, the assistant replies, or the tool messages.
• Only text. In a multimodal message, the guardrail screens the text. The guardrail does not screen the image parts or the audio parts.

Your application sends the whole conversation with each request. Thus, if the guardrail blocks a user message, the guardrail also blocks each later request that contains this message. To continue, start a new conversation, or remove this message from the history that you send.

The guardrail screens long prompts in full. Length does not create a blind spot.

What applies where

Nebul enables a guardrail for your organization. Then you decide which projects apply the guardrail, and which topics each project bans. Thus a strict production project and a permissive sandbox can exist together. Every API key in a project uses the setting of that project.

Outcomes

Every request ends in one of five outcomes. The Activity Log and the charts use the same five names.

Two outcomes stop a request: blocked and fail-closed. If you see fail-open or fail-closed, read When a check cannot run.

Topics

A guardrail screens for a fixed set of topics. You choose which topics apply to your project:

Politically sensitive content is not part of a topic. Thus it never blocks a request.

How to choose topics

To change the topics of a guardrail, open the guardrail from the list. If the guardrail is disabled, you cannot change its topics.

Two rules control which topics apply:

• When you enable a guardrail for the first time, it has no active topics. The guardrail blocks nothing until you select at least one topic.
• A topic that you disable cannot block a prompt, even when the guardrail flags that prompt.

If your product must permit a subject, disable the topic for that subject.

Disable a guardrail

To disable a guardrail in a project, set its switch to off in Security → Guardrails.

The project keeps the topics that you selected. The project also keeps the If this guardrail can't respond setting. When you enable the guardrail again, the same topics apply. Thus it is not necessary to select them again.

A disabled guardrail does not screen requests. If the project has no enabled guardrail, the Activity Log records each request as skipped.

When a check cannot run

A guardrail does not give a verdict in these cases:

• The model of the guardrail is unreachable.
• The model of the guardrail times out.
• The model of the guardrail returns an answer that the guardrail cannot read.

Each guardrail in a project has a setting for these cases, If this guardrail can't respond:

To change the setting, open the guardrail in Security → Guardrails. Select an option. Then save the change.

If an outage of the guardrail must not stop your application, select Allow the request unverified. In all other cases, keep the default.

If a guardrail flags an active topic, the check blocks the request with HTTP 400. If a different guardrail does not respond at the same time, this rule also applies.

In the product

Scanned counts the requests that the check ran for: ok, blocked, fail-open, and fail-closed. Scanned does not include skipped.

Each view takes a date range. The granularity of the activity data depends on the length of the range:

The Activity Log keys on request ID. If your application logged the ID of a request that returned a 400, you can find the row for that request. The row shows which topic the guardrail flagged. For more information, see Request IDs.