api documentation

TEE inference API docs

TEE inference on CPUs, sealed from the host inside AMD SEV-SNP or Intel TDX. The API is OpenAI-compatible. This preview is not live.

Preview: the host api.cpuinference.ai is a placeholder and is not live. Code samples below will not run against a real model yet.

Planned: fetch an attestation report and verify it before you send a prompt. See the Attestation page. That flow is not live.

Getting Started

curl -X POST https://api.cpuinference.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "lfm3-1.7b",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 512
  }'

Authentication

All API requests require authentication using a Bearer token. Include your API key in the Authorization header:

Authorization: Bearer YOUR_API_KEY

Single Unified Endpoint

Chat models use one OpenAI-compatible chat completions endpoint. Change the model parameter in your request body to switch between chat models. Embeddings, rerank, and speech to text are planned and are not live yet.

How it works: Specify which model to use via the model field in your JSON body. Example: "model": "lfm3-1.7b"

API Reference

Chat Completions

Creates a chat completion. This is the main endpoint for generating text responses.

Request Body Parameters

Code Examples

import requests

response = requests.post(
    "https://api.cpuinference.ai/v1/chat/completions",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json"
    },
    json={
        "model": "lfm3-1.7b",
        "messages": [{"role": "user", "content": "Hello!"}],
        "max_tokens": 512
    }
)

print(response.json()["choices"][0]["message"]["content"])

const response = await fetch("https://api.cpuinference.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: "lfm3-1.7b",
    messages: [{ role: "user", content: "Hello!" }],
    max_tokens: 512
  })
});

const data = await response.json();
console.log(data.choices[0].message.content);

Available Models

Use these model IDs in the model field of your API request. Each one is tagged not live yet. Chat examples on this page use lfm3-1.7b. Context lengths are omitted until we confirm them from the model cards.

Streaming Responses

Enable streaming to receive responses as they are generated. This provides a better user experience for real-time applications and chat interfaces.

const response = await fetch("https://api.cpuinference.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: "lfm3-1.7b",
    messages: [{ role: "user", content: "Write a story" }],
    stream: true
  })
});

const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  console.log(decoder.decode(value));
}

Rate Limits

Rate limits are TBD. We will publish them after we measure our own hardware. Nothing on this preview is live, so no limits are in effect.