Quickstart

The LLM Service Daemon (LSD) is a single Rust binary (gateway) backed by a single PostgreSQL database. Nothing else to stand up.

Prerequisites

To run LSD:

PostgreSQL 18+ with the pgvector extension
The pg_cron extension, for automated partition management, stats refresh, retention enforcement, and trip compaction (soft-required today; the gateway warns if it’s missing, and this is planned to become a hard requirement)
At least one model provider API key (OpenAI, Anthropic, etc.)

To build LSD from source (no pre-built binary or image is published for this fork yet):

Rust 1.97+ (managed with mise: mise use [email protected])

Steps

Build the gateway.
Terminal window
```
cargo build --release -p gateway
```
The binary is written to target/release/gateway.
Point LSD at Postgres.
Terminal window
```
export LSD_DATABASE_URL="postgres://user:pass@localhost:5432/lsd"
```
This is the only required environment variable. LSD stores its config, inferences, feedback, and statistics in this one database.
Run migrations.
Terminal window
```
gateway --run-postgres-migrations
```
This applies the consolidated baseline migration and exits.

Write a minimal config.

[gateway]
bind_address = "0.0.0.0:3000"

[models."gpt-5.4-mini"]
routing = ["openai"]

[models."gpt-5.4-mini".providers.openai]
type = "openai"
model_name = "gpt-5.4-mini"

Provider credentials are read from the provider’s usual environment variable (e.g. OPENAI_API_KEY) unless overridden in config.

(Optional) Create an API key.

Auth is off by default. To require a bearer token, set gateway.auth.enabled = true in your config and create a key:
Terminal window
```
gateway --create-api-key
```
The key is printed to stdout. Use it as Authorization: Bearer <key> on requests.
Start the gateway.
Terminal window
```
gateway --config-file lsd.toml
```
LSD logs the address it bound to, e.g. Lsd Gateway is listening on 0.0.0.0:3000. Binding to a non-loopback address without auth enabled prints a loud warning: don’t expose an unauthenticated gateway to the network.
Send a request, natively.

/inference is LSD’s own endpoint. Pass model_name to call a model directly, no function config required yet:
Terminal window
```
curl http://localhost:3000/inference \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "gpt-5.4-mini",
    "input": { "messages": [{"role": "user", "content": "Hello"}] }
  }'
```
The response includes a trip_id. Pass it back on the next call to group multi-turn requests into the same trip; add "continue_trip": true and LSD assembles the conversation context server-side, so you send only the new message instead of the full history. Neither is possible through the OpenAI-compatible endpoint below. This is also the endpoint that unlocks functions and variants, retries, cross-model fallback, and everything else native to LSD.

Add -H "Authorization: Bearer <key>" if you enabled auth.
(Or, if you’re migrating existing code) send an OpenAI-shaped request.

Point an existing OpenAI client at http://localhost:3000/openai/v1 and it works unchanged, no code changes, no native concepts required:
Terminal window
```
curl http://localhost:3000/openai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4-mini",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```
This is the easy on-ramp, not the destination: it’s a great way to try LSD with zero changes to existing code, but it can’t see functions, variants, trips, or retries. Move to /inference once you’re ready for those.

Next steps

Functions & Variants: LSD’s native abstraction over prompts, models, and sampling strategies.
Gateway: routing, fallbacks, and which providers are supported.
Experimentation: A/B test and adaptively route between variants.
Observability: what gets stored in Postgres and how to query it.
Optimization: turn collected data into fine-tuning and few-shot jobs.
Evaluations: score inferences and multi-step workflows.