Models at launch

Pick a model when you launch a Node Lodge Secure machine. The session is ready as soon as the machine is, and the model downloads in the background. Use the machine while it downloads. The session is billed from the moment it is ready, download included.

Pick a model

In Launch, choose a Node Lodge Secure machine. In the Environment step, enter the model in Model (optional):

• Use the name as Hugging Face lists it: org/name, like Qwen/Qwen3-8B, for the safetensors weights.
• Add :VERSION for one GGUF file of the repository: org/name:VERSION, like unsloth/Qwen3-8B-GGUF:Q4_K_M. See GGUF versions.
• Accept the license of the model. The launch carries your acceptance.
• Check Serve with to get an OpenAI-compatible endpoint on the model. See Serve the model.
• Leave the field empty to launch without a model.

With the API, send model (id and licenseAccepted: true) in POST /v1/sessions. With the Python SDK, pass model="Qwen/Qwen3-8B" to sessions.launch. With the TypeScript SDK, pass model: { id, licenseAccepted: true }. With the command line, add --model <org/name[:VERSION]> --accept-license to nodelodge launch. See API reference and CLI and SDK.

POST /v1/models/lookup checks a model before you launch: its revision, its size, its download time and, for a GGUF version, its format and file.

GGUF versions

A GGUF repository holds one file per version (quantization), like Qwen3-8B-Q4_K_M.gguf and Qwen3-8B-Q8_0.gguf. Write the version after the name, org/name:VERSION, and the machine downloads that file alone:

• The version matches the end of the file name, after a - or a ., upper or lower case alike: Q4_K_M picks Qwen3-8B-Q4_K_M.gguf.
• When several files end with it, like Q4_K_M and UD-Q4_K_M, the shortest name wins. Write UD-Q4_K_M for the other one.
• A file split in parts, like -00001-of-00002.gguf, downloads with all its parts.
• Each file is downloaded in parallel and checked against its SHA-256.

A launch with a version the repository does not have is refused before anything starts, with the versions it has:

This repository has no Q3_K_S version. Versions in this repository: Q4_K_M, Q6_K, Q8_0.

A repository without GGUF files is refused the same way:

This repository has no GGUF files. Enter it without :Q4_K_M to download its safetensors weights.

The estimated size and download time, the budget check and the disk space all count that one file.

Budget and download time

The launch form shows the estimated download time of the model. A launch is refused when your budget or maximum duration does not cover the estimate with a margin. The message says what to raise, with the amount that covers the estimate at the rate of the machine. For example, for Qwen/Qwen3-32B:

Your budget covers about 20 minutes. Downloading Qwen/Qwen3-32B and starting the machine takes about 29 minutes. Raise the budget to at least US$X.XX, or launch without a model.

The real duration depends on the load of the network. Once the session runs, its page shows the time left, measured from the download itself.

Your Hugging Face token

Save a Hugging Face read token once. Node Lodge uses it to look up and resolve every model you launch, public models included. A gated or private model needs it.

1. In the console, open Settings, then Integrations.
2. Paste a read token in Hugging Face and choose Save token. Hugging Face must accept it. With two-factor authentication on, enter a fresh code; without it, enter your Account password.
3. You receive an e-mail each time a token is saved, replaced or removed.

• A read token is enough.
• For a gated model, accept its license on Hugging Face with the account of the token first.
• The token is stored encrypted and never shown again: the page shows its last 4 characters and the date it was saved. Replace and Remove are on the same page. Sessions already started keep theirs.
• It does not appear in your session details.

For a gated or private model, the token is put on the machine: in $HF_HOME/token, readable by the ubuntu user only, and set as HF_TOKEN in every login shell. hf, vLLM and transformers find it without hf auth login. A machine launched with a public model gets no token.

To upload to the Hub from your machines, turn on Also put my token on every machine I launch (for uploads to the Hub) in Integrations. It is off by default. Every machine you launch with a model then gets the token, and anyone who can open these machines can read it. Turning it on asks for a fresh two-factor code, or your password without two-factor. On machines shows the setting: Gated and private models only or Every machine you launch with a model. With the API, send shareOnMachines in PUT /v1/integrations/hugging-face, or { "shareOnMachines": true } in PATCH /v1/integrations/hugging-face.

To use another token for one launch, send it as modelToken in POST /v1/sessions. With the Python SDK, pass model_token to sessions.launch; with the TypeScript SDK, pass modelToken. It takes the place of the saved token for that launch only, and it is put on the machine whatever the model. Without a saved token, the launch form asks for one under Hugging Face token (gated and private models), for that launch. POST /v1/models/lookup takes a token the same way; lookups with a token are limited to 10 per minute per organization.

With an API key (CLI, SDKs, MCP server, GitHub Action), launches use your saved token only when the key was created with Launches use my saved Hugging Face token checked, in API keys. Anyone holding such a key can read the token on the machines it launches. With any other key, send the token with the launch (modelToken) or create a key with the box checked.

What is loaded

For org/name: the weights in safetensors format, with the files that go with them: configuration, tokenizer, and license. Other formats of the same model, like GGUF or ONNX, are left out. A model published without safetensors weights is refused at launch: pick a GGUF version with org/name:VERSION when the repository has one.

For org/name:VERSION: the one GGUF file of that version, or all its parts.

The revision loaded is pinned when you launch, so a model updated on Hugging Face later does not change under your session. The session page shows the model and where it is on the machine.

Where the files are

On the machine, the model is in the Hugging Face cache:

• /data/huggingface when the machine has a data disk,
• ~/.cache/huggingface otherwise.

HF_HOME is set for every user and every shell from the first connection, so tools find the cache without further setup. The Model row of the session page shows the full path, with a copy button.

For safetensors weights, the session’s model.localPath is the snapshot folder of the model. Load it by name. Nothing downloads again:

vllm serve Qwen/Qwen3-8B

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B")

Run a GGUF file

For a GGUF version, model.localPath is the GGUF file itself, for example:

/data/huggingface/hub/models--unsloth--Qwen3-8B-GGUF/snapshots/<revision>/Qwen3-8B-Q4_K_M.gguf

model.format is gguf and model.id is unsloth/Qwen3-8B-GGUF:Q4_K_M. Copy the path from the Model row of the session page into MODEL_PATH, then give it to llama.cpp:

nodelodge-model wait && llama-server -m "$MODEL_PATH" --host 127.0.0.1 --port 8080

Or to Ollama, with a Modelfile that holds one line, FROM and the path:

echo "FROM $MODEL_PATH" > Modelfile
ollama create qwen3-8b -f Modelfile

For a file split in parts, the path is its first part. llama.cpp loads the other parts from the same folder.

Serve the model

Check Serve with under the model, and a server starts on it once it is downloaded, behind an OpenAI-compatible endpoint with its own key:

• vLLM for org/name (safetensors weights). Requests name the model org/name, like Qwen/Qwen3-8B.
• Ollama for org/name:VERSION (one GGUF file). The file is imported under a short name, like qwen3-8b:q4_k_m for unsloth/Qwen3-8B-GGUF:Q4_K_M. Ollama takes a GGUF file in one part: a version split in parts is refused at launch.

With the API, send serveWith (vllm or ollama) in POST /v1/sessions, with a model written to match. With the command line, add --serve to --model. With the SDKs, pass serveWith: 'vllm' (TypeScript) or serve_with="vllm" (Python). With the MCP server, pass serve: true. See CLI and SDK and MCP server.

Serve with runs on sessions through the Node Lodge relay. On other sessions, the launch is refused:

Serve with runs on sessions through the Node Lodge relay. Launch without it, then start the server over SSH.

The endpoint

The server listens on the machine only. Its endpoint is published at the address of the session on the relay:

https://<session>.<relay domain>/v1

• A port may follow the domain: copy the base URL as Node Lodge gives it.
• TLS runs all the way to the machine. The relay sees neither the key nor the requests.
• Each session has its own key, starting with nls_. Send it as Authorization: Bearer <key>. A request without a valid key gets 401.
• Paths under /v1/ are answered. Any other path is refused.
• The server and the endpoint run as a dedicated system user, nodelodge-serve, with no shell, never as root.
• The relay limits the open connections and the new connections per second of each session origin.
• The endpoint stops with the session, including when its budget is reached.

Read the endpoint:

• In the console, on the session page, the OpenAI-compatible endpoint panel shows Base URL, Model, API key (masked, with Copy key and New key) and a curl example.
• With the API, GET /v1/sessions/{id}/access returns inference: baseUrl, apiKey, model, engine and status.
• With the command line, nodelodge endpoint <session-id> prints the base URL, the model and the key.
• With the MCP server, get_session returns endpoint.

Server status

The session gives the status of the server in serve.status:

After a failure, connect over SSH and start a server yourself on the model. See Without Serve with.

Send a request

Set BASE_URL to the base URL and NODELODGE_ENDPOINT_KEY to the key, then:

curl "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $NODELODGE_ENDPOINT_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "Qwen/Qwen3-8B", "messages": [{"role": "user", "content": "Hello"}]}'

Any OpenAI client works. With the OpenAI Python library:

import os

from openai import OpenAI

client = OpenAI(base_url=os.environ["BASE_URL"], api_key=os.environ["NODELODGE_ENDPOINT_KEY"])
reply = client.chat.completions.create(
    model="Qwen/Qwen3-8B",
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)

With Ollama, name the short model, like qwen3-8b:q4_k_m. The key is a secret: keep it out of shared files, addresses and logs.

Get a new key

Choose New key in the panel, or call POST /v1/sessions/{id}/serve-key (sessions.newServeKey(id) in TypeScript, sessions.new_serve_key(id) in Python). The call returns the new key. The previous key stops within 15 seconds, and its open connections close.

Without Serve with

Nothing starts on the model. Connect with ssh and start the server yourself, on safetensors weights:

nodelodge-model wait && vllm serve Qwen/Qwen3-8B

On a GGUF file, with Ollama or llama.cpp: see Run a GGUF file.

Disk space

The machine needs room for the weights (for a GGUF version, its file) plus a margin. When it does not have enough, the launch is refused before anything starts, with the space the model needs and the space the machine has for models. Pick a machine with more disk, or a smaller model.

Follow the download

The Model row of the session page shows the progress and the time left. On the machine:

• nodelodge-model status prints the percentage downloaded and the size of the model.
• nodelodge-model wait returns when the model is loaded, with exit code 0. Any other exit code means the model was not loaded.
• A message at the opening of a shell tells you where the download stands.

Run nodelodge-model wait before you load the model or the GGUF file, so the server starts on complete files:

nodelodge-model wait && vllm serve Qwen/Qwen3-8B

Restarts and interruptions

A restart of the machine resumes the download. To resume a download that stopped, run:

sudo nodelodge-model resume

Files already verified stay in place and are not downloaded again.

When the model was not loaded

When the download does not finish, the session keeps running and its page says The model was not loaded. It gives the reason, for example:

• Hugging Face did not serve the files.
• The download took more than 6 hours.
• A downloaded file did not match its checksum.
• The disk ran out of space.
• The download failed.
• The download made no progress for 10 minutes.

The page shows the command that downloads the model. When the session is already running, you also receive an e-mail with the reason and the command. Open a terminal on the machine and run it, or run sudo nodelodge-model resume. The files land in the same cache.

For a gated or private model, the command signs you in to Hugging Face before it downloads: paste your read token when it asks. For a GGUF version, the command names its file: hf download org/name <file> --revision <revision>. sudo nodelodge-model resume uses the token in $HF_HOME/token. When that file is missing, sign in first, then resume:

/opt/nodelodge/jupyter/bin/pip install -q -U huggingface_hub && /opt/nodelodge/jupyter/bin/hf auth login
sudo nodelodge-model resume

The warning shows while the session runs. A session that ended or did not start has no machine to download to.

E-mails

You receive an e-mail when the model is ready, unless it arrives within 5 minutes of the session being ready: the console shows it. When the session is already running, you also receive an e-mail when the model could not be downloaded. Turn either off in Notification settings. See Notifications.

Community machines

Community environments that download a model do it at first launch, under your name, into /workspace. Models downloaded by Node Lodge in the background are on Node Lodge Secure machines. See Secure vs Community.