LLM Endpoint
LLM endpoints connect Eve to model providers such as OpenAI, Anthropic, OpenRouter, Ollama, and Azure OpenAI so traffic can be governed through one model path.
Endpoint Kind
Use llm as the endpoint kind when creating this endpoint through the API or dashboard.
What Eve Uses This Endpoint For
- Route model requests through Eve with policy, telemetry, and cost controls.
- Create governed model access for applications and agents.
- Apply PII, template lookup, content policy, access-boundary, budget, routing, and caching controls before requests reach providers.
- Compare provider latency, errors, and cost across traffic.
Basic Endpoint Shape
{
"endpoint": "openai-prod",
"kind": "llm",
"config": {
"read_conn": {
"provider": "OpenAI",
"inline_api_key": "secret",
"defaults": {
"model": "gpt-4o"
}
}
}
}Create it with the endpoint API:
curl -sS -X POST "$EDEN/endpoints" \
-H "$AUTH_HEADER" \
-H "Content-Type: application/json" \
-d '{
"endpoint": "openai-prod",
"kind": "llm",
"config": {
"read_conn": {
"provider": "OpenAI",
"inline_api_key": "secret",
"defaults": {
"model": "gpt-4o"
}
}
}
}'Configuration Fields
| Field | Purpose |
|---|---|
provider | Model provider, such as OpenAI, Anthropic, Ollama, OpenRouter, or Azure OpenAI |
inline_api_key | Provider API key when required |
model | Default model |
base_url_override | Optional compatible API base URL |
Operating Notes
- Use separate endpoints for materially different providers, environments, or routing policies.
- Avoid embedding provider secrets in client applications; route clients through Eve-issued access.
- Set model allowlists and budget controls before broad rollout.
- Add gateway governance before sending regulated traffic such as claim packets, research notes, support tickets, or tool-heavy agent work.
Gateway Governance Controls
LLM endpoints are usually paired with Eve-issued gateway API keys. A gateway key can apply additive governance before the request reaches the provider:
| Control | Use it for |
|---|---|
| Built-in PII detection | Common email, phone, SSN, and payment-card patterns. |
| Organization PII dictionary | Shared literal or regex terms across gateway keys. |
custom_pii_terms | Key-specific literals or regex patterns. |
pii_lookup_templates | Request-scoped PII from saved Eve Read templates, such as a customer or account lookup. |
| Content policy profiles | Reusable sensitive-content profiles for restricted projects, credentials, watchlists, or payment instructions. |
| Policy bindings | Assign profiles to organizations, users, agents, auth groups, gateway keys, endpoints, routes, or work metadata. |
| Access boundaries | Block disallowed subject/resource combinations before provider, template, or tool egress. |
| Feature audit and governance records | Store promptless decisions, simulations, approvals, evidence packs, replay jobs, and DLP/evaluation records. |
Example lookup configuration on a gateway key:
{
"pii_lookup_templates": [
{
"template_id": "lookup-customer-account-pii",
"input_fields": [
{
"name": "account_id",
"source": "user_message_regex",
"pattern": "(?i)account(?: id)?[:# ]+(?P<value>[A-Z0-9-]+)",
"required": true,
"value_type": "string"
}
],
"records_path": "",
"redaction_fields": [
{
"path": "/full_name",
"label": "customer",
"action": "redact"
},
{
"path": "/email",
"label": "email",
"action": "redact"
}
],
"max_rows": 1,
"error_mode": "block"
}
]
}Expected behavior: Eve extracts the configured inputs from the original request, runs the saved Read template in sensitive mode, maps selected template output fields to request-scoped PII terms, merges those terms with the existing scanners, and sends only the masked request upstream.