Skip to main content

Inline LLM Safety with the New Guardrail Job Type

· 3 min read

proxiML now offers a dedicated Guardrail job type that lets you deploy inline safety and prompt guarding in front of any LLM endpoint. It currently provides three configurable protection layers — content safety, jailbreak detection, and injection detection — and exposes an OpenAI-compatible API so you can drop it in front of your existing service with no code changes.

How It Works​

A Guardrail job wraps an existing HTTPS endpoint (the protected service) and intercepts all incoming requests and outgoing responses before they reach the model. You select which detection layers to enable, and the guardrail handles the rest.

  • Content Safety -- Uses models like Nemotron Content Safety, Llama Guard 3, or Qwen3Guard to evaluate both input and output text against safety policies covering hate speech, dangerous content, sexually explicit material, and harassment. Refusal messages can be returned in 9 languages for multilingual applications.
  • Jailbreak Detection -- Uses perplexity-based heuristics (length-per-perplexity and prefix/suffix perplexity) to detect adversarial prompts that attempt to bypass model safety measures and elicit harmful output.
  • Injection Detection -- Uses YARA pattern matching to detect exploitation attempts including Python code injection, SQL injection, Jinja template injection, and cross-site scripting (XSS). Designed for agentic systems as part of a defense-in-depth strategy.

The platform automatically provisions the managed serving job, encrypts API keys at rest, enables auto-restart, and configures the connection to your protected service — no custom Docker images, start commands, or checkpoint setup required.

Using the Web Platform​

From the job creation form, select Guardrails as the job type. The endpoint section presents:

  • Protected Service (required) -- Select the HTTPS endpoint you want to guard from your enabled project services.
  • Content Safety -- Toggle to enable input and output content safety checks. Optionally configure the detection service (service, API key, model).
  • Jailbreak Detection -- Toggle to enable adversarial prompt detection. Optionally configure the detection service (service, API key, model).
  • Injection Detection -- Toggle to enable injection attack detection for agentic systems.

Once the job reaches the running state, your Guardrail endpoint is live and accessible via the endpoint URL.

Using the SDK​

job = await proximl.jobs.create(
name="Full Guardrail Job",
type="guardrail",
gpu_type="cpu",
cpu_count=4,
endpoint=dict(
guardrail=dict(
protected_service=dict(
id="service_id",
api_key="your-api-key",
),
content_safety=True,
content_safety_service=dict(
id="moderation_service_id",
api_key="safety-service-key",
),
jailbreak_detection=True,
injection_detection=True,
),
),
)

The protected_service.id field is required; all detection toggles and service configurations are optional.