Balanced generation under the pipeline, per-kind brake; STATE: type mix fix

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Kral
2026-10-05 19:27:43 +02:00
parent 55f4068330
commit 976274d48a
3 changed files with 42 additions and 8 deletions

View File

@@ -165,3 +165,13 @@ Training runs on HF Jobs with Unsloth, not on the Mac. No `mlx_lm` training.
- **Token length (for the bf16 memory test):** accepted samples, 20 tool schemas kept: p50 20k, p90 39k, **p95 48k, max 72k** (about 10k tokens are the tool schemas). The bf16 memory test must use a **sequence length of 48k** (covers 95 % of the samples; samples over 48k are cut or dropped; 72k is not needed in the test).
- **DDLS reject reasons** (11 CDS runs: 6 accepted, 5 rejected): tool budget 2 (old limit 60, before the 100 budget), empty response 3 (one turn reached the 32k output limit while the model wrote its own CDS test class). Activation 0, hidden tests 0, ATC 0: all 11 runs passed every gate and every hidden test. Failed CDS tasks get the second attempt (pipeline rule).
- **G1034 lock event (cause not found):** after `sap_push_source includeType=testclasses` the class stayed locked ("User KESELI is currently editing", SM12 entry needed; teardown said "You are already editing"). Not a harness bug: the lock outlived the MCP session and the run. Not reproduced in 3 tests on A4H (alone, 60 writes overlapping 2690 unit test calls of another session, writes with ATC + coverage + member listing + a second session; `push_element` on a test-class method). One case in about 90 runs. Details for the EPOD developer: `docs/epod-lock-leak.md`. The harness treats it as an infrastructure event (run not accepted, worker count 2 to 1, 3 equal events stop the pipeline) and lists the object in the dashboard.
### Object type mix fix (2026-10-05 19:30, Opus review item 1)
- **Cause:** plans 1 and 2 were built from `evalset.SLOTS`, which has only CLAS, FUNC, PROG and DDLS. The mix of CLAUDE.md section 1 (INTF, DDIC, MSAG, exception) was never in a plan, so no slot existed for those types and nothing was rejected. The harness also lacked G2 checks, message writing and mutants for them (`evalset.py` header said so).
- **Before (139 accepted tasks):** CLAS 49 %, FUNC 21 %, DDLS 19 %, PROG 9 %, INTF/TABL/STRU/MSAG 0, exception 2 %. Target: CLAS 28, INTF 7, DDLS 25, FUNC 15, PROG 10, TABL 8, STRU 2, MSAG 2.5, exception 2.5 (percent; `harness/mix.py`).
- **Fixes:** G2 for TABL/STRU fields and MSAG message numbers, `sap_push_message` in the oracle and setup, mutants for DDIC, interfaces, message classes and exception classes, G6 cascade fix (abaplint does not know CX_ super classes), format notes per type in the generator, `implements` may be a list. DTEL/DOMA are not made (DDIC share is TABL 8 + STRU 2).
- **First tasks per new type, all accepted with oracle 100, null 0 and mutation kill rate 100 %:** G1900 INTF (3 tries), G1901 TABL (1), G1902 MSAG (2), G1903 STRU (3), G1904 exception (mutation rerun after the exception mutants). Model mistakes fed back by the generator: unit/currency reference annotation in DDL needs `'table.field'`, RTTI length is in bytes.
- **Generation:** `trainset run --plan balanced` (3 workers, started by the pipeline): every slot takes the kind with the biggest deficit against the target; a kind is skipped with 6+ tries and under 20 % accepted, or with more than 8 accepted tasks waiting for a first trajectory run. 20 % error-targeted, 30 % hard (difficulty 3). K variants stay at 10 %.
- **Trajectories:** the next task is the one whose kind has the biggest deficit (accepted trajectories per kind). Summary now lists the mix and DDLS reject reasons.
- **Incident 19:25:** restarting the controller I started a second one by mistake (wrong `pgrep` pattern) and deleted the objects of a running run. Both controllers were stopped, leftovers cleaned, one controller runs. One table `Z4AJ50UB_PO_HEAD` kept a lock from the interrupted write (SM12 needed). Interrupting a write leaks the lock: never kill a controller during a run without checking.