Review notes: token length for bf16 test (p95 48k), DDLS reject reasons, G1034 lock analysis (not reproduced)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -159,3 +159,9 @@ Training runs on HF Jobs with Unsloth, not on the Mac. No `mlx_lm` training.
|
||||
- Tokens of accepted samples (20 tool schemas kept): p50 20147, p90 39458, p95 47694, max 71791, n 50.
|
||||
- Syntax hints (proxy syntaxCheck added): 1 in 1 runs.
|
||||
- Harness events: 2 ({'text:[LOCK]': 1, 'teardown': 1}); trajectory workers now 1.
|
||||
|
||||
### Notes after the Opus review of the first stage 2 summary (2026-10-05 18:30)
|
||||
|
||||
- **Token length (for the bf16 memory test):** accepted samples, 20 tool schemas kept: p50 20k, p90 39k, **p95 48k, max 72k** (about 10k tokens are the tool schemas). The bf16 memory test must use a **sequence length of 48k** (covers 95 % of the samples; samples over 48k are cut or dropped; 72k is not needed in the test).
|
||||
- **DDLS reject reasons** (11 CDS runs: 6 accepted, 5 rejected): tool budget 2 (old limit 60, before the 100 budget), empty response 3 (one turn reached the 32k output limit while the model wrote its own CDS test class). Activation 0, hidden tests 0, ATC 0: all 11 runs passed every gate and every hidden test. Failed CDS tasks get the second attempt (pipeline rule).
|
||||
- **G1034 lock event (cause not found):** after `sap_push_source includeType=testclasses` the class stayed locked ("User KESELI is currently editing", SM12 entry needed; teardown said "You are already editing"). Not a harness bug: the lock outlived the MCP session and the run. Not reproduced in 3 tests on A4H (alone, 60 writes overlapping 2690 unit test calls of another session, writes with ATC + coverage + member listing + a second session; `push_element` on a test-class method). One case in about 90 runs. Details for the EPOD developer: `docs/epod-lock-leak.md`. The harness treats it as an infrastructure event (run not accepted, worker count 2 to 1, 3 equal events stop the pipeline) and lists the object in the dashboard.
|
||||
|
||||
Reference in New Issue
Block a user