Models and memory

The catalogue

Kalon offers a fixed catalogue of open-weight models. It has 405 entries: a short hand-picked list followed by a larger list checked against each model's repository. There is no import for arbitrary files: every download is a catalogue entry.

Chat entries range from falcon-h1:90m (90 million parameters, 59 MB) to kimi-k2:1t-thinking (one trillion parameters, 386.3 GB). The largest download is mistral-large3:675b-a41b at 412.1 GB.

The catalogue covers the major open families, including Gemma, Llama, Qwen, Mistral, Phi, DeepSeek, GLM, Kimi, MiniMax, gpt-oss, Granite, Falcon, LFM, SmolLM, InternLM, Hunyuan, ERNIE and MiniCPM, plus Whisper and Moonshine for speech and Kokoro and Qwen3 TTS for voices.

Reading a model card

Each card shows the model id, a one-line description, and parameters · quantization · size. A row of glyphs shows what the model takes in and what it gives back: text, images, video, audio, code or vectors.

• Model ids have the form family:tag, for example gemma4:4b or qwen2.5:7b. The same id is used by the app, the local API and the kalon command-line tool.
• Mixture-of-experts models show active parameters with an A, for example 30B-A3B: 30 billion parameters in total, about 3 billion used per token.
• Reasoning models think before they answer. The app shows that thinking separately from the reply.

Downloads

• Model files come from Hugging Face. The Moonshine speech models come from Moonshine's own download server.
• Downloads can be paused and resumed. An interrupted download picks up where it stopped.
• Models published in several parts are fetched as one download.
• A file is only kept if its size matches the size the server reports.
• Several models can download at the same time.

Files are stored in ~/Library/Application Support/Kalon/models on the Mac, one folder per model. On iPhone the models folder is excluded from iCloud and device backups.

To remove a model on the Mac, hover Installed on its card, click Uninstall, and confirm. On iPhone, swipe the row and tap Delete.

How Kalon sizes a model

When you load a model, Kalon reads its geometry from the file header without loading the weights. It then chooses a context window and how many layers go to the GPU. The setting that controls this is Automatic memory sizing in Settings › Models, and it is on by default.

1. Budget. Total memory minus a reserve for the system. On the Mac the reserve is 6 GB or one eighth of total memory, whichever is smaller.
2. Context. Kalon aims for 8,192 tokens, or 131,072 tokens for model files over 40 GB. It never exceeds what the model was trained for and never goes below 4,096. If memory is short, it lowers the context before anything else.
3. Fit check. The weights that must stay resident, the context cache and working memory must fit the budget. If they do not, Kalon declines to load the model.
4. GPU split. If the whole model does not fit the GPU's working set, Kalon keeps as many layers on the GPU as fit. For a mixture-of-experts model whose shared layers fit but whose experts do not, it moves only the experts to the CPU and keeps every attention layer on the GPU.

For a mixture-of-experts model, only the shared weights and the experts one token uses need to stay in memory. The rest is read from disk as needed. That is why some very large models can load on machines with less memory than their file size, though they run slowly.

When a model will not fit

A model that cannot fit is refused rather than loaded into constant swapping. The message names the model and the shortfall:

<model> is too large for this device's memory: about <amount> more RAM is needed. Try a smaller quantization.

Choose a smaller model, or a smaller quantization of the same one.

Turning Automatic memory sizing off on the Mac restores a fixed context and full GPU offload and never refuses a load. Use it only when you know the model fits.

iPhone and iPad limits

iOS gives each app a memory allowance that is smaller than the device's physical memory, and ends an app that goes over it. Kalon works within that allowance.

• The Models list offers chat and vision models only, and only those that fit. A model fits if its size plus 448 MB of working memory is within three quarters of the app's current allowance.
• No model that needs more than 12 GB is offered on any iPhone or iPad.
• The footer says how much memory the list was filtered against and how many models need more.
• Before a download starts, Kalon checks for free storage and warns with Not enough space if there is too little.
• Fit models to this iPhone's memory (or iPad) in Settings › Storage is on by default. Kalon then declines a model it judges too large to run. Turning it off loads the model anyway, and iOS may close the app if it runs out of memory.

Several models at once

The Mac can hold more than one model in memory. The model you chose in the app stays loaded. Models requested through the local API join a pool alongside it, and the least recently used idle model is unloaded when a new one needs the room.