abap-llm/docs/bf16-memory.md

bf16 memory test and GPU choice (2026-10-06, estimates, nothing was run)

No GPU job is started without Kral's go. Decision 2026-10-06: wait. Run the memory test when the training data is near the size of the first SFT run (the real sample lengths and the count then decide the flavor; today there are 36 train samples). Script: train/hf_train_bf16.py (--memory-test --sweep 16000,32000,48000,64000: no data needed, 3 optimizer steps per length, stops at the first OOM, uploads the result as memtest_*.json to the output repo). The length limit (48k or 64k) is decided by this test (decision 3): the builder keeps --max-tokens 48000 until then; 64k would bring back the long CDS trajectories (today 5 of 19 are over 48k).

Model (from config.json of Qwen3.8-27B)

64 layers (48 linear attention, 16 full attention), hidden 5120, MLP 17408, vocab 248320, 27.8 B parameters. LoRA rank 16 on q/k/v/o, gate/up/down, in_proj_qkv, in_proj_z, out_proj: 107 M trainable parameters.

Memory by component (GB; estimate, not a measurement)

component	48k	64k	how
weights bf16	51.8	51.8	27.8 B x 2 bytes
LoRA weights + grads + 8-bit Adam	1.0	1.0	107 M x 10 bytes
layer inputs for the backward pass	29.3 on the GPU, about 0 with the Unsloth CPU offload (then 29 GB host RAM)	39.1 / about 0 (host RAM 39 GB)	seq x 5120 x 2 bytes x 64 layers
recompute peak of one layer	9.2	11.3	MLP tensors seq x 17408 x 2 bytes x 4 plus 3 GB for attention (assumption)
logits and loss	3 chunked, 67 full logits	3 chunked, 89 full logits	full logits: seq x 248320 x (bf16 + fp32 upcast + grad)

Fit by GPU (92 % of the card counted as usable)

seq	checkpoints	loss	estimated peak GB	1x A100 80 GB	1x RTX PRO 6000 96 GB	1x H200 141 GB	2x H200 282 GB (model parallel)
16k	CPU offload (Unsloth)	chunked	61	fits	fits	fits	fits
16k	CPU offload (Unsloth)	full logits	80	no	fits	fits	fits
16k	on the GPU	chunked	71	fits	fits	fits	fits
16k	on the GPU	full logits	90	no	tight	fits	fits
32k	CPU offload (Unsloth)	chunked	63	fits	fits	fits	fits
32k	CPU offload (Unsloth)	full logits	104	no	no	fits	fits
32k	on the GPU	chunked	82	no	fits	fits	fits
32k	on the GPU	full logits	124	no	no	fits	fits
48k	CPU offload (Unsloth)	chunked	65	fits	fits	fits	fits
48k	CPU offload (Unsloth)	full logits	129	no	no	fits	fits
48k	on the GPU	chunked	94	no	tight	fits	fits
48k	on the GPU	full logits	158	no	no	no	fits
64k	CPU offload (Unsloth)	chunked	67	fits	fits	fits	fits
64k	CPU offload (Unsloth)	full logits	153	no	no	no	fits
64k	on the GPU	chunked	106	no	no	fits	fits
64k	on the GPU	full logits	192	no	no	no	fits

Reading:

• The loss is the main risk, not the weights. With full logits only the H200 (141 GB, CPU offload) fits 32k and 48k, and not 64k. The run needs a fused or chunked cross entropy (Unsloth has one for the architectures it patches; whether it covers qwen3_5 is unknown). The memory test shows it at once: if 16000 already fails on an H200, the loss is the cause. Fallback (not built): hidden states, then the loss over the 32 % labeled positions only, in chunks of 4k.
• A100 80 GB (2.50 USD/hour) only with the CPU offload and a chunked loss (about 65 GB at 48k, 67 GB at 64k: no margin). H200 141 GB (5 USD/hour) fits with margin; RTX PRO 6000 96 GB (2.75 USD/hour) fits with the offload and the chunked loss.
• Multi-GPU (a100x4, h200x2): only with model parallelism, one card works at a time; not before the single card test.

Order when the test is run

1. Memory test on h200 (about 40 minutes with 64k, about 3.5 USD): sweep 16000,32000,48000,64000, offload on. The limit is the largest length that fits with margin.
2. If the loss is the problem: build the chunked loss (about 2 hours), repeat.
3. Real run with --s2-epochs 3 --s2-loss-share 0.6 (stage 1 epochs are computed: stage 2 carries 60 % of the loss tokens; weights of the own-test class are applied).

Time and cost (estimate from the nf4 run: 221 tokens/s on an A100 at 6.75k tokens per document; bf16 faster, an H200 about 2 to 2.5 times an A100)

Today's data: stage 2 is 36 samples (0.94 M tokens per epoch, 3 epochs = 2.8 M tokens) and, by the 60 % rule, about 0.2 epochs of stage 1 (0.5 M tokens): 3.3 M tokens in total. A100 about 3.1 h (about 8 USD), H200 about 1.4 h (about 7 USD). With three times the stage 2 data (the size that the restart plan aims at): 2.8 M tokens per epoch, 8.5 M in 3 epochs, plus about 0.6 epochs of stage 1 (1.5 M): 10 M tokens, H200 about 4.3 h (about 21 USD), A100 about 9 h (about 23 USD). The real number comes from the memory test (step_seconds).