Skip to main content

LLM Gateway

Use the LLM Gateway in signageOS Cloud at https://llm.root.signageos.io to discover model aliases, submit Responses requests, and inspect credit consumption.

Feature entitlement: aiLLM · Availability: Explicit company LLM access required

Access requirements​

Your company must have the aiLLM company feature entitlement explicitly enabled to discover models or submit Responses. It is disabled by default and independent of MCP access: enabling MCP alone does not grant LLM access. Contact your signageOS representative to arrange access.

Models and Responses require LLM access. Successful enabled and disabled entitlement checks are cached for ten minutes; changes take effect after refresh, not necessarily immediately. Retained usage remains available within your role permissions when inference is disabled.

The gateway README is the canonical API and operator configuration reference. The public API provides only Models, Responses, and Usage; Messages and Chat Completions routes are not supported.

Authentication and company selection​

Send Authorization: Bearer <account JWT or account API token> on every endpoint. Organization credentials are not supported. Company and account attribution comes from your authenticated identity, not request body fields.

Use x-company-uid to select a company your account can access. You may omit it only when your account has exactly one accessible company. Multiple companies without a selector return 400 ambiguous_company; an unauthorized selection returns 403 forbidden.

Successful account/company discovery, including administrator privileges, is cached for five minutes. Membership and role changes may take that long to appear. The separate ten-minute entitlement cache is scoped to the API environment, credential, and company; identity refresh does not reset it. If entitlement refresh fails after expiry, inference is denied rather than using stale access.

Discover models​

Set GATEWAY_URL=https://llm.root.signageos.io, ACCOUNT_TOKEN to your account credential, and COMPANY_UID to your authorized company. Keep credentials out of source control, logs, and screenshots.

curl "$GATEWAY_URL/v1/models" \
  -H "Authorization: Bearer $ACCOUNT_TOKEN" \
  -H "x-company-uid: $COMPANY_UID"

The model aliases are sos/fast, sos/balanced, and sos/reasoning. Use an alias as your request's model; concrete model identifiers and routing overrides are not accepted.

Submit a response​

curl "$GATEWAY_URL/v1/responses" \
  -H "Authorization: Bearer $ACCOUNT_TOKEN" \
  -H "x-company-uid: $COMPANY_UID" \
  -H "Content-Type: application/json" \
  --data '{"model":"sos/fast","input":"Reply with OK."}'

You receive a JSON Responses payload. For incremental Server-Sent Events (SSE), use curl -N and add literal JSON "stream": true. Only that value enables client streaming.

Input structures and model-specific extensions pass through for upstream validation. Not every model supports every Responses option. Envelope model fields use the public alias; nested tool output and metadata are preserved.

Send conversation history inline. Requests reject background: true, non-null previous_response_id or conversation, input item_reference entries, top-level provider, provider_model, models, or route, and direct tool model selectors.

Request bodies are limited to 1 MiB; individual SSE events are limited to 20 MiB. Disconnecting stops delivery, not processing or usage accounting.

Credits and independent budgets​

Credits measure model-priced consumption, not a prepaid balance or invoice. Fractional credits are supported. Each request uses a fixed pricing snapshot; historical charges do not change when prices change.

Credits charge uncached input, cached input, non-reasoning output, and reasoning output separately, without counting overlapping categories twice. Cached input is part of input tokens, and reasoning is part of output tokens. Reported totalTokens is preserved, not recomputed from those categories.

Two independent rolling budgets control new inference:

  • Company budget: all accounts' credit consumption in that company. Switching accounts cannot bypass it.
  • Account budget: one account's consumption within the selected company. Exhausting it does not block another account while the company still has allowance; the same account's budgets in different companies remain separate.

Limits and rolling durations are operator-configured. New inference is rejected at or above either allowance and becomes available as recorded credits age out. Already-admitted work can exceed the allowance and still counts. A separate company request-count limit also applies.

View company and account usage​

Company administrators can view company totals and the per-account breakdown through Usage and the Box company page. In Box, select the date interval to see total credits and the per-user/account breakdown.

Company members can query Usage for their own account only. This API access does not add a member usage page in Box or expose company totals or other accounts.

Query the API with your own account credential:

curl "$GATEWAY_URL/v1/usage?from=2026-10-01T12:30:00.000Z&to=2026-10-02T15:45:00.000Z" \
  -H "Authorization: Bearer $ACCOUNT_TOKEN" \
  -H "x-company-uid: $COMPANY_UID"

from and to are required canonical UTC timestamps, including milliseconds and Z, such as 2026-10-01T12:30:00.000Z. Arbitrary times are supported; midnight is not required. The interval includes from, excludes to, must be positive, and cannot exceed 90 days. Date-only values and timezone-offset spellings are not accepted.

The administrator response contains:

  • from, to, and company credits.
  • inputTokens, cachedInputTokens, outputTokens, reasoningTokens, and totalTokens.
  • availableResponses and unavailableResponses, distinguishing reported usage from unavailable usage.
  • accounts: entries containing accountUid, credits, and the same token and availability counters, sorted by account UID. Account totals reconcile to company totals.

Member responses contain only from, to, and accounts, with at most your own accountUid entry and its credit, token, and availability counters. They contain no company aggregates. You cannot select another account through request parameters or headers.

There are no daily buckets or per-response ledger export. Empty administrator intervals return zero totals and no account entries; empty member intervals return accounts: []. Role-scoped usage remains available even when LLM access is disabled, entitlement refresh fails, or credit budgets are exhausted; company authorization still applies.

Usage is retained for 90 days. Valid terminal totals count for completed, incomplete, and failed responses. Missing optional token categories count as zero. Missing or invalid final usage contributes zero tokens and credits, marked unavailable, not measured zero consumption; it does not block later inference. The response can still fail for a proxy or processing error. No estimates fill missing usage, and failed or disconnected requests can still have recorded consumption.

Errors and retries​

ResultWhat to do
400 invalid_requestCheck the model alias, request restrictions, and canonical UTC usage range. Model-specific validation can also reject a request upstream.
400 ambiguous_companySupply an authorized x-company-uid.
401 authentication_requiredCheck your account bearer credential.
403 forbiddenCheck your account's access to the selected company.
403 llm_disabledAsk for the selected company's LLM entitlement to be enabled.
429 company_credit_limit_exceededThe company budget is exhausted; another account cannot bypass it.
429 account_credit_limit_exceededYour account's budget in this company is exhausted.
429 rate_limit_exceededThe separate company request-count limit is reached.
503 pricing_unavailableUsable model pricing is unavailable; inference has not been sent upstream. Retry later.
503 state_unavailableRequired authorization, entitlement, or usage state is unavailable. Retry later.
502 upstream_unavailable / 502 upstream_authentication_failedProcessing failed; do not assume the request consumed no credits.
504 upstream_timeoutThe upstream deadline expired; do not blindly retry inference.

Honor Retry-After on limit errors when present. For credit budgets, it assumes no additional completions consume credits before then.

After SSE headers are sent, failures can appear as response.failed events even with initial HTTP status 200. Read the terminal event rather than treating stream establishment as success. Retain the server-generated x-request-id when reporting a failure; omit credentials and prompt content.

Browser integrations​

Your browser origin must be enabled for the target environment. Localhost is not enabled by default. Origin rules are environment-specific; an explicit HTTPS subdomain policy does not include the domain apex, HTTP, or non-default ports.

Preflight allows POST for Responses, GET for Models/Usage, and authorization, content-type, and x-company-uid request headers. Valid preflight returns 204 but does not grant account/company access. Actual requests still require authentication, company authorization, and endpoint permissions.

Use bearer authentication, not cookies. Browsers may read x-request-id and retry-after. If preflight fails or a response lacks an origin grant, verify the exact scheme, hostname, and port with your environment administrator.