# Stage 1 training (train/)

Task: `docs/stage1-training-task.md`. State of the work: this file and `train/STATE.md` (later steps).

## Environment

- `train/.venv`: Python 3.11.17 (created with `uv`, Homebrew; the system Python 3.9 is not changed).
- `mlx-lm` 0.32.0, `mlx` 0.32.3.

## Base model

- Base model: `Qwen/Qwen3.8-27B` (architecture `qwen3_5`, 27.8B, dense; Apache 2.0). It is the same base
  model as the Ollama model `qwen3.8-27b-32k` that the harness used before.
- Weights: **`mlx-community/Qwen3.8-27B-4bit`** (Hugging Face), local path `~/models/Qwen3.8-27B-4bit`.
  Quantization: MLX affine, 4 bit, group size 64. Size 16.1 GB. Downloaded 2026-10-03.
- Why not the Ollama weights (`qwen3.8:27b-mlx`): they are NVFP4 (modelopt) with one global scale per
  layer. `mx.quantized_matmul` has no global scale, so `mlx_lm` cannot load them without a re-quantization
  (a different model). Kral approved the Hugging Face download (2026-10-03).

## Serving (`train/serve.sh`)

`mlx_lm.server` at `http://127.0.0.1:8080/v1` (OpenAI-compatible). Base model without adapter; after
training the same script with `--adapter-path`.

Settings, the same as the earlier Ollama runs (runs 103, 203):

| Setting | Ollama run | mlx_lm.server |
|---|---|---|
| Thinking | on, Ollama default level `medium` | `--chat-template-args '{"enable_thinking": true, "reasoning_effort": "medium"}'` (template default would be `xhigh`) |
| temperature | 0.2 (sent by the harness llm agent; overrides the Modelfile value 1) | 0.2 (sent by the agent; server default `--temp 0.2`) |
| top_p / top_k / min_p | 0.95 / 20 / 0 (Modelfile) | `--top-p 0.95 --top-k 20 --min-p 0` |
| presence / repeat penalty | 0 / 1 (neutral) | not set (neutral) |
| Output limit | none (context `num_ctx` 32768) | `--max-tokens 32768` (server default would be 512) |
| Context | 32768 | no fixed limit (memory) |

Thinking in the earlier Ollama runs: verified from run 103 (a turn with 3480 completion tokens and about
100 visible tokens).

Tool-call test (2026-10-03): one request with the harness system prompt and the MCP tool schemas returned
`sap_pull_source(objectType=INTF, objectName=ZIF_DEMO_CHECK)` in the OpenAI `tool_calls` format; the
arguments parse as JSON. The reasoning comes in a separate `reasoning` field. The chat template uses the
qwen3_coder XML tool format; `mlx_lm` parses it. First request: 86 s (prompt of 9.5k tokens).

## Eval subset

`train/subset.json` (copy: `runs/stage1/subset.json`): 25 tasks, balanced over the categories
(A, B, C, E, F: 3 each; D, G, H, I, K: 2 each), with T01. Use the same list before and after training.

## Baseline

- Runner: `python3 train/baseline.py --label baseline` (one task at a time; results
  `runs/stage1/baseline.json`, run directories `runs/stage1/baseline/`).
- The earlier T01 score 41.7 (run 103) used the Ollama NVFP4 weights. The baseline of 2026-10-03 with the
  MLX 4-bit weights is the new reference for stage 1.