Local API

Kalon for Mac can serve the models on your Mac over HTTP, so your own scripts, editors and tools can use them. The server runs inside the app. It is available while Kalon is running and stops when you quit. There is no server on iPhone or iPad.

Turn it on

1. Open Settings › Developer.
2. Turn on Local API server.
3. The status row changes to Running and shows the address, http://localhost:11434 by default.

While it is running, the Connect an external tool card lists both base URLs:

The server is off by default. If you leave it on, it starts again the next time you open Kalon.

Port

Change Port to any value from 1 to 65535. Changing it restarts the server. If another app holds the port, the server does not start and the switch turns itself off with an error that asks you to pick a different port.

Network access

By default the server listens on 127.0.0.1, so only programs on this Mac can reach it. Turning on Network access makes it listen on all interfaces so other devices on your network can use it.

The server has no authentication. Anyone who can reach the port can use your models. Turn on Network access only on a network you trust.

Devices on your network must address the Mac by its IP address, as in http://<mac-ip-address>:11434. Requests whose Host header is a name other than localhost, such as mymac.local, are rejected with 403. This protects against DNS rebinding.

Browsers and CORS

Requests without an Origin header, such as those from scripts, are always accepted. Browser pages get CORS headers only when their origin is localhost, 127.0.0.1, 0.0.0.0 or [::1] on any port, or an app://, file://, tauri://, vscode-webview:// or vscode-file:// origin. Preflight requests are answered with methods GET, POST, OPTIONS and headers Content-Type, Authorization.

Endpoints

The server has 15 routes.

Request bodies are JSON. No API key is checked. OpenAI client libraries need some key, so pass any placeholder.

Choosing a model

The model field takes a model id such as qwen2.5:7b. Only installed models can be used; /v1/models and /api/tags list them. Leaving model empty uses the model currently selected in the app.

A model other than the app's own is loaded into a pool on first use. See keep_alive for how long it stays.

Chat completions

POST /v1/chat/completions

Images: add an images array of base64 strings to a user message. Content parts of type image_url are not read. The model must be a vision model.

Fields not listed here, such as seed, n, response_format and tool_choice, are ignored.

Response

Field values in this example are illustrative.

{
  "id": "chatcmpl-…",
  "object": "chat.completion",
  "created": 1790000000,
  "model": "qwen2.5:7b",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "…" },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 12, "completion_tokens": 48, "total_tokens": 60 }
}

finish_reason is stop, or tool_calls when the model called a tool. Reasoning text is removed from the reply.

Streaming

With "stream": true, the response is text/event-stream. Each event is a data: line:

1. A first chunk with delta: {"role":"assistant","content":""}.
2. One chunk per piece of text, with delta: {"content":"…"}.
3. A final chunk with an empty delta and finish_reason: "stop", or with delta.tool_calls and finish_reason: "tool_calls".
4. data: [DONE].

The stream carries no usage chunk. If generation fails mid-stream, the final chunk's content begins with [error:.

Tool calling

Pass tools in the usual OpenAI form, {"type":"function","function":{"name":...,"description":...,"parameters":{...}}}. When the model calls one, the response carries:

{
  "index": 0,
  "id": "call_…",
  "type": "function",
  "function": { "name": "get_weather", "arguments": "{\"city\":\"Accra\"}" }
}

Send each result back as a message with role: "tool" and the result in content. Kalon does not read tool_calls or tool_call_id from earlier messages in the history.

Text completions

POST /v1/completions takes prompt (a string, or an array holding exactly one string) and the same sampling fields as chat. "raw": true sends the prompt to the model without a chat template. finish_reason is stop or length. Streaming follows the same data: format with choices[0].text.

Embeddings

POST /v1/embeddings

{ "model": "bge-small:en", "input": ["first text", "second text"] }

The response is {"object":"list","data":[{"object":"embedding","index":0,"embedding":[…]}],"model":"bge-small:en","usage":{…}}. Vectors are normalised to unit length. input can be a string or an array of strings.

POST /api/embed and POST /api/embeddings take the same model and input and return {"model":"…","embeddings":[[…],[…]]}.

Embedding runs in its own slot and never waits behind a chat request.

Native chat and generate

POST /api/chat takes model, messages, tools and keep_alive, with sampling in an options object: temperature, top_p, num_predict, stop and num_ctx. stream defaults to true here, and the response is newline-delimited JSON (application/x-ndjson):

{"model":"qwen2.5:7b","created_at":"2026-09-28T12:00:00Z","message":{"role":"assistant","content":"Hel"},"done":false}

The last line has "done": true, a done_reason of stop or length, and timing counts: total_duration, prompt_eval_count, prompt_eval_duration, eval_count and eval_duration, with durations in nanoseconds. Messages can carry images as base64 strings.

POST /api/generate takes prompt, an optional system string and raw. Each streamed line carries text in response. It does not accept images.

Loading and unloading

Send /api/chat with no messages, or /api/generate with an empty prompt, to load a model without generating. Add "keep_alive": 0 to unload it.

curl http://localhost:11434/api/generate -d '{"model": "qwen2.5:7b", "keep_alive": 0}'

The model selected in the app cannot be unloaded through the API.

keep_alive

keep_alive sets how long a pool model stays in memory after its last response.

Idle models are checked every 30 seconds. When a new model needs the memory, the least recently used idle pool model is unloaded first. If nothing can be unloaded, the request fails with not enough free RAM to load another model beside the ones in use; free up memory or switch models. Turning the server off unloads every pool model.

Status and planning

• GET /health returns status, the number of active, queued, admitted, completed and rejected requests, the models generating, max_queue (32), acquire_timeout_s (300) and version.
• GET /api/ps lists models in memory with expires_at. The app's own model reports 9999-12-31T23:59:59Z.
• GET /api/plan?model=<id> returns the load plan from Kalon's memory sizer without loading anything: the context window, GPU layers, whether experts would move to the CPU, a decision of fits, needs_eviction, downgraded or refuse, and the memory breakdown. See Models and memory.

Concurrency and errors

Requests to the same model run one at a time. Requests to different models run in parallel. Up to 32 requests can wait across all models.

Error bodies have the form {"error":{"message":"…","type":"kalon_error"}}.

Examples

curl

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5:7b",
    "messages": [{"role": "user", "content": "Summarise the edge in one line."}],
    "stream": true
  }'

Python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="kalon")

stream = client.chat.completions.create(
    model="qwen2.5:7b",
    messages=[{"role": "user", "content": "Write a haiku about Accra."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

JavaScript

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "http://localhost:11434/v1", apiKey: "kalon" });

const stream = await client.chat.completions.create({
  model: "qwen2.5:7b",
  messages: [{ role: "user", content: "Name three uses for a local model." }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Embeddings with fetch

const res = await fetch("http://localhost:11434/v1/embeddings", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ model: "bge-small:en", input: ["on-device", "in the cloud"] }),
});
const { data } = await res.json();
console.log(data.map((d) => d.embedding.length));

The kalon command-line tool

Settings › Developer › Command-line tool installs kalon at /usr/local/bin/kalon. macOS may ask for an administrator password.

pull, list and rm work on their own. run, ps and stop talk to the server, so Kalon must be running with Local API server on. Set KALON_HOST if you changed the port, for example KALON_HOST=127.0.0.1:8080.