Skip to content
LLM Service Daemon

API Reference

For OpenAI/Anthropic SDK compatibility, see Gateway. This page covers the LLM Service Daemon’s (LSD’s) native endpoints, which add functions, variants, trips, and feedback on top.

POST /inference

Section titled “POST /inference”
{
  "function_name": "generate_summary",
  "input": { "messages": [{ "role": "user", "content": "Summarize this." }] },
  "trip_id": null,
  "continue_trip": false,
  "variant_name": null,
  "stream": false,
  "dryrun": false,
  "tags": { "team": "research" }
}
  • Exactly one of function_name or model_name is required. model_name skips function config entirely and calls a model directly through LSD’s native request/response shape (functions/variants/trips still apply, just with an implicit default function).
  • Omit trip_id on the first call of a multi-turn interaction; LSD generates one and returns it in the response. Pass it back on subsequent calls to group them into the same trip. Never generate a trip ID yourself.
  • continue_trip: true enables server-side trip memory: LSD fetches the trip’s latest stored inference, assembles the conversation context itself, and expects only the new messages in input. Requires a trip_id, works for chat functions only, and is rejected if the previous turn ended in tool calls (tool loops stay client-owned). On a trip’s first turn it behaves like a normal inference.
  • Omit variant_name unless you’re deliberately pinning a variant for testing; otherwise LSD selects one per Experimentation.
  • dryrun: true runs inference without persisting it.

POST /feedback

Section titled “POST /feedback”
{
  "inference_id": "01957bbb-44a8-7490-bfe7-32f8ed2fc797",
  "metric_name": "thumbs_up",
  "value": true
}
  • Exactly one of inference_id or trip_id is required, not both.
  • metric_name is either a metric defined in config, or the reserved names comment / demonstration.
  • value’s shape depends on the metric type: boolean, float, free-text comment, or a demonstration (a corrected output).

POST /batch_inference

Section titled “POST /batch_inference”

Submits many inference requests as one job (same Params shape as /inference, as an array; an optional trip_ids array assigns a trip per input). Poll and cancel with:

Terminal window
GET  /batch_inference/{batch_id}
GET  /batch_inference/{batch_id}/inference/{inference_id}
POST /batch_inference/{batch_id}/cancel

A batch’s status is pending, completed, failed, cancelled, or expired. See Gateway for which providers use a native batch API versus the synthetic fan-out backend.

Datasets

Section titled “Datasets”
Terminal window
POST   /v1/datasets/{dataset_name}/datapoints              # create
PATCH  /v1/datasets/{dataset_name}/datapoints               # update
DELETE /v1/datasets/{dataset_name}/datapoints               # delete
POST   /v1/datasets/{dataset_name}/from_inferences           # build a dataset from stored inferences
POST   /v1/datasets/{dataset_name}/list_datapoints
POST   /v1/datasets/{dataset_name}/get_datapoints
DELETE /v1/datasets/{dataset_name}

Datasets back evaluations and optimization jobs.

OpenAI/Anthropic-compatible

Section titled “OpenAI/Anthropic-compatible”
Terminal window
POST /openai/v1/chat/completions
POST /openai/v1/embeddings
POST /anthropic/v1/messages

Standard OpenAI/Anthropic request and response shapes; "model" resolves through a model’s routing list (see Gateway). These don’t expose functions, variants, or trips; use /inference for that.