abap-llm/docs/bf16-memory.md

bf16 memory test and GPU choice (2026-10-06, estimates, nothing was run)

No GPU job is started without Kral's go. Script: train/hf_train_bf16.py (--memory-test --sweep 16000,32000,48000 is the memory test, no data needed, 3 optimizer steps per length, stops at the first OOM, uploads the result as memtest_*.json to the output repo).

Model (from config.json of Qwen3.8-27B)

64 layers (48 linear attention, 16 full attention), hidden 5120, MLP 17408, vocab 248320, 27.8 B parameters. LoRA rank 16 on q/k/v/o, gate/up/down, in_proj_qkv, in_proj_z, out_proj: 107 M trainable parameters.

Memory by component at 48k tokens (GB; the estimate, not a measurement)

component	GB	how
weights bf16	51.8	27.8 B x 2 bytes
LoRA weights + grads + 8-bit Adam	1.0	107 M x 10 bytes
layer inputs for the backward pass (checkpoints)	29.3 on the GPU, about 0 with the Unsloth CPU offload	48k x 5120 x 2 bytes x 64 layers; the offload needs 29 GB of host RAM (all flavors have 142 GB or more)
recompute peak of one layer	9.2	MLP tensors 48k x 17408 x 2 bytes x 4 plus 3 GB for attention (assumption)
logits and loss	3 chunked, 67 with full logits	full logits: 48k x 248320 x (bf16 + fp32 upcast + grad); only a chunked or fused cross entropy is realistic

Fit by GPU (92 % of the card counted as usable)

seq	checkpoints	loss	estimated peak GB	1x A100 80 GB	1x RTX PRO 6000 96 GB	1x H200 141 GB	2x H200 282 GB (needs model parallel)
16k	CPU offload (Unsloth)	chunked	61	fits	fits	fits	fits
16k	CPU offload (Unsloth)	full logits	80	no	fits	fits	fits
16k	on the GPU	chunked	71	fits	fits	fits	fits
16k	on the GPU	full logits	90	no	tight	fits	fits
32k	CPU offload (Unsloth)	chunked	63	fits	fits	fits	fits
32k	CPU offload (Unsloth)	full logits	104	no	no	fits	fits
32k	on the GPU	chunked	82	no	fits	fits	fits
32k	on the GPU	full logits	124	no	no	fits	fits
48k	CPU offload (Unsloth)	chunked	65	fits	fits	fits	fits
48k	CPU offload (Unsloth)	full logits	129	no	no	fits	fits
48k	on the GPU	chunked	94	no	tight	fits	fits
48k	on the GPU	full logits	158	no	no	no	fits

Reading:

• The loss is the main risk, not the weights. With full logits only the H200 (141 GB, with the CPU offload) fits 32k and 48k; the A100 and the RTX PRO 6000 do not. The run needs a fused or chunked cross entropy (Unsloth has one for the architectures it patches; whether it covers qwen3_5 is not known). The memory test shows it at once: if seq_len 16000 already fails on an H200, the loss is the cause. Fallback (not built yet, small): compute the hidden states, then the loss over the 32 % labeled positions only, in chunks of 4k.
• A100 80 GB (2.50 USD/hour) is only possible with the CPU offload and a chunked loss (about 65 GB estimated at 48k: no margin). H200 141 GB (5 USD/hour) fits with margin; RTX PRO 6000 96 GB (2.75 USD/hour) fits with the offload and the chunked loss (est. 65 GB).
• Samples over 32k are a minority (p90 39560 tokens in the current data, p50 24.8k): if 48k does not fit, 32k would drop 15 of 52 samples (29 %; 10 of them DDLS), so test 48k first.
• Multi-GPU (a100x4, h200x2): possible only with model parallelism (device_map), one card works at a time, Unsloth multi-GPU is limited; not recommended before the single card test.

Proposed order

1. Memory test on h200 (about 30 minutes, 2.5 USD): sweep 16000,32000,48000 with the offload. If 48k fits with margin: stay on H200 or try rtx-pro-6000 for the real run.
2. If the loss is the problem: build the chunked loss (about 2 hours of work), repeat the test.
3. Real run after the stage 1 : stage 2 ratio decision (below).

Time and cost (estimate from the nf4 run: 221 tokens/s on an A100 at 6.75k tokens per document; bf16 is faster, H200 about 2 to 2.5 times an A100)

Tokens of the proposed run (1 epoch stage 1 and 3 epochs of the 47 stage 2 samples as built today: 2.53 M + 3 x 1.28 M = 6.4 M tokens): A100 about 5.9 h (about 15 USD), H200 about 2.7 h (about 14 USD). The real number comes from the memory test (step_seconds).