Skip to content
LLM Service Daemon

Quickstart

The LLM Service Daemon (LSD) is a single Rust binary (gateway) backed by a single PostgreSQL database. Nothing else to stand up.

Prerequisites

Section titled “Prerequisites”

To run LSD:

  • PostgreSQL 18+ with the pgvector extension
  • The pg_cron extension, for automated partition management and stats refresh (soft-required today; the gateway warns if it’s missing, and this is planned to become a hard requirement)
  • At least one model provider API key (OpenAI, Anthropic, etc.)

To build LSD from source (no pre-built binary or image is published for this fork yet):

Steps

Section titled “Steps”

  1. Build the gateway.

    Terminal window
    cargo build --release -p gateway

    The binary is written to target/release/gateway.

  2. Point LSD at Postgres.

    Terminal window
    export LSD_DATABASE_URL="postgres://user:pass@localhost:5432/lsd"

    This is the only required environment variable. LSD stores its config, inferences, feedback, and statistics in this one database.

  3. Run migrations.

    Terminal window
    gateway --run-postgres-migrations

    This applies the consolidated baseline migration and exits.

  4. Write a minimal config.

    lsd.toml
    [gateway]
    bind_address = "0.0.0.0:3000"
    

    [models."gpt-5.4-mini"]
    routing = ["openai"]
    

    [models."gpt-5.4-mini".providers.openai]
    type = "openai"
    model_name = "gpt-5.4-mini"

    Provider credentials are read from the provider’s usual environment variable (e.g. OPENAI_API_KEY) unless overridden in config.

  5. (Optional) Create an API key.

    Auth is off by default. To require a bearer token, set gateway.auth.enabled = true in your config and create a key:

    Terminal window
    gateway --create-api-key

    The key is printed to stdout. Use it as Authorization: Bearer <key> on requests.

  6. Start the gateway.

    Terminal window
    gateway --config-file lsd.toml

    LSD logs the address it bound to, e.g. Lsd Gateway is listening on 0.0.0.0:3000. Binding to a non-loopback address without auth enabled prints a loud warning: don’t expose an unauthenticated gateway to the network.

  7. Send a request.

    LSD exposes an OpenAI-compatible endpoint, so existing OpenAI clients work unchanged by pointing their base URL at http://localhost:3000/openai/v1:

    Terminal window
    curl http://localhost:3000/openai/v1/chat/completions \
      -H "Content-Type: application/json" \
      -d '{
        "model": "gpt-5.4-mini",
        "messages": [{"role": "user", "content": "Hello"}]
      }'

    Add -H "Authorization: Bearer <key>" if you enabled auth.

Next steps

Section titled “Next steps”
  • Gateway: routing, fallbacks, and which providers are supported.
  • Observability: what gets stored in Postgres and how to query it.
  • Optimization: turn collected data into fine-tuning and few-shot jobs.
  • Evaluations: score inferences and multi-step workflows.