Guardrails
This is a beta feature. For more information and access requests, please contact us through the Support Portal.
A guardrail makes sure that each inference request to the model is secure and safe. You choose the topics to guard against. These topics include unsafe content, jailbreak attempts, and personally-identifiable information.
In Security → Guardrails, you can enable the topics for an AI Studio project. The guardrail then applies to all API keys of that project. You can change the topics at any time. Each change applies immediately to all API keys of the project.
Nebul enables guardrails for your organization. Then you choose which projects apply them, and which topics each project bans. To enable or disable a guardrail topic, you must be a project owner or an organization owner. For more information, see Projects.
Why guardrails
If a public endpoint can get to a model, use guardrails. Examples are a customer chatbot, a support widget, or an internal assistant that is open to all employees.
Guardrails can do these tasks:
- Reject harmful prompts before inference, to prevent agent hijacking.
- Protect RAG-enabled endpoints, to make sure that the data available to the model does not leak to a malicious actor.
- Show what the guardrail screened, blocked, and allowed, with a history for each request.
- Trace a rejection back to the request that caused it.
- Keep prompt content private, because the system records only the outcome and the flagged topic.
How it works
The check runs on every request path
A blocked request returns HTTP 400:
{"detail": {"message": "prompt blocked by guardrails","error_type": "bad_request"}}
Two rules control what the guardrail screens:
- Only the most recent user message. The guardrail does not screen earlier turns or the system prompt again. The request that sent each user message screened it already.
- Only text. In a multimodal message, the guardrail screens the text. It does not screen the image parts or the audio parts.
The guardrail screens long prompts in full. Length does not create a blind spot.
What applies where
Nebul enables a guardrail for your organization. Then you decide which projects apply the guardrail, and which topics each project bans. Thus a strict production project and a permissive sandbox can exist together. Every API key in a project uses the setting of that project.
Outcomes
Every request ends in one of four outcomes. The Activity Log and the charts use these same four words. Three of the four outcomes mean that the request went through.
Only blocked always stops a request. If you see fail-open, read When a check cannot run.
Topics
A guardrail screens for a fixed set of topics. You choose which topics apply to your project:
| Topic | Flags prompts that |
|---|---|
| Violent | Seek or promote physical violence against people or animals |
| Non-violent Illegal Acts | Seek or promote fraud, theft, hacking, trafficking, and similar acts |
| Sexual Content or Sexual Acts | Ask for or contain sexually explicit material |
| PII | Expose or ask for personal identifying information about a real person |
| Suicide & Self-Harm | Encourage, ask for, or give the means for suicide, self-harm, or disordered eating |
| Unethical Acts | Ask for help with deception, harassment, discrimination, or similar conduct |
| Politically Sensitive Topics | Push political propaganda, election disinformation, or extremist ideology |
| Copyright Violation | Ask for a copy of copyrighted or proprietary material |
| Jailbreak | Try to bypass, override, or extract the safety instructions of the assistant |
How to choose topics
To enable or disable individual topics, open the guardrail. Each change saves immediately.
Two rules control which topics apply:
- When you enable a guardrail from the list, all of its topics become active.
- A topic that you disable cannot block a prompt, even when the guardrail flags that prompt.
If your product must permit a subject, disable the topic for that subject.
When a check cannot run
If screening is not available for a short time, the request is not allowed through by default. The Activity Log records the request as fail-open.
To change this behavior, go to Project → Guardrails. Then clear the Allow requests when guardrails are degraded checkbox.
A flagged topic still blocks the request, even when the rest of the screening is degraded.
In the product
| Where | What you get |
|---|---|
| Security → Guardrails | Each guardrail available to your organization, the number of active topics (3 of 9), and an Active/Inactive switch. Counters show the total guardrails, the active guardrails, and the requests blocked in the selected period. |
| Guardrails → Activity Log | One row for each screened request. The row shows the request ID, the outcome, the timestamp, and the decision of each guardrail. To see only the blocked requests, filter by outcome. |
| Security → Overview | Totals for screened, blocked, allowed, and fail-open requests, with the change against the previous period. Activity over time. A safe-versus-flagged comparison. The most recent blocked requests. |
Guardrails log only the outcome of a request and the topics that the guardrail flagged. The system never stores the prompt. The prompt never appears in the Activity Log, on the charts, or in an export. A blocked row tells you that the guardrail flagged a request as Jailbreak. The row does not tell you what the request said.
Screened counts ok, blocked, and fail-open. These are the requests that a guardrail examined. Screened does not include skipped.
Each view takes a date range. The granularity of the activity data depends on the length of the range:
| Granularity | Longest range |
|---|---|
| Per minute | 7 days |
| Hourly | 90 days |
| Daily | 365 days |
The Activity Log keys on request ID. If your application logged the ID of a request that returned a 400, you can find the row for that request. The row shows which topic the guardrail flagged. For more information, see Request IDs.