Summary
When an output guardrail with kind: "keyword" and hook_point: "output" blocks an upstream response containing a forbidden literal, the gateway's caller-visible error message includes the matched literal verbatim — defeating the purpose of an output guardrail (which exists precisely to keep forbidden content from reaching the caller).
Repro
E2E test attached: drafted as tests/e2e/src/cases/guardrail-output-e2e.test.ts (held back from #151 batch 2 PR pending this fix).
Setup:
- Mock upstream returns assistant message containing the literal
"leakedsecret"
- Output keyword guardrail configured with
patterns: [{kind:"literal", value:"leakedsecret"}]
- Caller sends an innocent prompt
Observed:
```json
{
"error": {
"message": "content blocked by policy: output blocked by literal "leakedsecret"",
"type": "content_filter"
}
}
```
The forbidden literal is right there in error.message. Anyone who can trigger the guardrail can extract the policy's blocklist by inspecting error responses.
Expected:
```json
{
"error": {
"message": "response blocked by content policy",
"type": "content_filter"
}
}
```
Or any redacted equivalent that does NOT include the matched pattern.
Severity
MEDIUM-HIGH for output guardrails specifically:
- For input guardrails, leaking the matched pattern back is mostly cosmetic — the caller sent it, they already know.
- For output guardrails, leaking the matched pattern is a real information leak — the whole point is to prevent the model's forbidden output from reaching the caller. Echoing it in the error envelope is a partial bypass.
Proposed fix
Strip / redact the matched value from the error rendered to clients. Internal logs may keep the matched pattern for operator debugging, but the wire-level error.message and any other caller-visible field must not carry it.
Test attached
The held-back test asserts:
```ts
const errorBlob = JSON.stringify(caught.error ?? {});
expect(errorBlob).not.toContain(FORBIDDEN_WORD);
```
Will be added back to the e2e suite once this is fixed.
Summary
When an output guardrail with
kind: "keyword"andhook_point: "output"blocks an upstream response containing a forbidden literal, the gateway's caller-visible error message includes the matched literal verbatim — defeating the purpose of an output guardrail (which exists precisely to keep forbidden content from reaching the caller).Repro
E2E test attached: drafted as
tests/e2e/src/cases/guardrail-output-e2e.test.ts(held back from #151 batch 2 PR pending this fix).Setup:
"leakedsecret"patterns: [{kind:"literal", value:"leakedsecret"}]Observed:
```json
{
"error": {
"message": "content blocked by policy: output blocked by literal "leakedsecret"",
"type": "content_filter"
}
}
```
The forbidden literal is right there in
error.message. Anyone who can trigger the guardrail can extract the policy's blocklist by inspecting error responses.Expected:
```json
{
"error": {
"message": "response blocked by content policy",
"type": "content_filter"
}
}
```
Or any redacted equivalent that does NOT include the matched pattern.
Severity
MEDIUM-HIGH for output guardrails specifically:
Proposed fix
Strip / redact the matched value from the error rendered to clients. Internal logs may keep the matched pattern for operator debugging, but the wire-level
error.messageand any other caller-visible field must not carry it.Test attached
The held-back test asserts:
```ts
const errorBlob = JSON.stringify(caught.error ?? {});
expect(errorBlob).not.toContain(FORBIDDEN_WORD);
```
Will be added back to the e2e suite once this is fixed.