Carbon Halo Model Lab

Model testing command center.

Manage the local `llama-server`, upload GGUF files into the lab, switch active models, and inspect the current runtime without touching production Carbon Halo or Sarah services.

LLM

Service State inactive User service status for the managed `llama-server` process.

Health Check Degraded http://127.0.0.1:18010/v1/models

Model Inventory 1 GGUF payloads discovered under `models/`.

Chat Surface Embedded Native `llama-server` chat stays available below for live TPS testing.

Service Control

Start, restart, or stop the lab service while keeping an eye on the runtime health probe.

Control Plane

Enabled

disabled

Active

inactive

Health

Unavailable at http://127.0.0.1:18010/v1/models: <urlopen error [Errno 111] Connection refused>

Current Model

/opt/sarah-local-llm-lab/models/deepseek-coder/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf

◎ Status JSON Inspect the live API status payload returned by the lab backend. ◬ Live Models Endpoint Open the currently exposed OpenAI-compatible models listing directly.

Switch Active Model

Promote a different GGUF payload into the active `llama-server` configuration and restart the service in one step.

Runtime Config

Embedded Chat Bench

Use the native `llama-server` chat UI, including its token-per-second readout, for quick local model testing.

Performance Probe

Open chat in a full tab | http://tools.carbonhalo.us:18010

Current Config

Useful runtime details for the active lab target without dropping into the terminal.

Snapshot

Model Path

/opt/sarah-local-llm-lab/models/deepseek-coder/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf

Bind

0.0.0.0:18010

Context

4096

GPU Layers

-1

Extra Args

Upload GGUF

Stage new model files into the lab inventory, then promote them above into the active runtime config.

Model Intake

Uploads remain inside the Carbon Halo lab only. After upload, select the new model in the runtime config panel and restart the service.

Available Models

Inventory of GGUF payloads discovered inside the lab filesystem.

Model Catalog

Name	Path	Size
DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf	models/deepseek-coder/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf	9.65 GB

Recent Log Tail

Latest lines from the most recent lab log for quick runtime inspection.

Diagnostics

llama_server_18010_20260612T025627Z.log

0.00.013.451 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.013.455 I device_info:
0.00.189.104 I   - CUDA0   : NVIDIA GeForce RTX 3060 (11900 MiB, 10652 MiB free)
0.00.189.127 I   - CPU     : AMD Ryzen 7 5700G with Radeon Graphics (63647 MiB, 63647 MiB free)
0.00.189.242 I system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
0.00.189.249 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.189.296 I srv          init: running without SSL
0.00.189.340 I srv          init: using 15 threads for HTTP server
0.00.189.445 I srv         start: binding port with default address family
0.00.190.592 I srv  llama_server: loading model
0.00.190.606 I srv    load_model: loading model '/opt/sarah-local-llm-lab/models/deepseek-coder/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf'
0.00.190.640 I common_init_result: fitting params to device memory ...
0.00.190.641 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)