erhan/abap-llm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Stage 1 on HF Jobs: Unsloth job script, PEFT to MLX converter, base valid loss 0.849, Qwen base model docs

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Kral
2026-10-04 18:51:40 +02:00
parent a2ba9e7b44
commit 5447874fd3
8 changed files with 196 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
train/config_test.yaml Normal file
View File

@@ -0,0 +1,25 @@
# Stage 1 short test: 20 iterations, same settings as config.yaml. Run: MLX_DISABLE_COMPILE=1 train/.venv/bin/mlx_lm.lora -c train/config_test.yaml
model: /Users/erhankeseli/models/Qwen3.8-27B-4bit
train: true
data: train/data
fine_tune_type: lora
num_layers: -1
lora_parameters:
rank: 16
scale: 20.0
dropout: 0.0
batch_size: 1
grad_checkpoint: true
iters: 20
learning_rate: 5.0e-5
lr_schedule:
name: cosine_decay
warmup: 2
arguments: [5.0e-5, 20, 0.0]
max_seq_length: 16384
steps_per_report: 1
steps_per_eval: 20
val_batches: 3
save_every: 20
adapter_path: train/adapters_test
seed: 20261003