erhan/abap-llm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
0ba25faeeafe7dd9b894929acf2787294253e4a0
abap-llm/tasks_gen/eval
History
Kral 0ba25faeea Mutation in generator; category H (stop) scoring and judge; category K variants and tool schema variant; evalset plan 
- mutation.py: negation mutants, skip WHILE/DO blocks (endless loop blocked RFC ~8 min)
- generator: accept only after mutation check; survivors go back as repair feedback
- runner/judge.py: stop tasks scored 100/30/0; gap by keywords, else judge model
- proxy: tool schema variant generic_v0 (draft); generator make_k_variant
- evalset.py: 88 slots for step F (A-I, H, K)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 05:37:20 +02:00
..
G0001
Harness: runner, generator, tasks T01 T13 T14 T15, CLAUDE.md
2026-10-02 21:42:51 +02:00
G0002
Mutation in generator; category H (stop) scoring and judge; category K variants and tool schema variant; evalset plan
2026-10-03 05:37:20 +02:00
G0003
Mutation in generator; category H (stop) scoring and judge; category K variants and tool schema variant; evalset plan
2026-10-03 05:37:20 +02:00
G0004
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0005
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0006
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0007
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0008
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0009
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0010
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0011
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0012
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0013
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0014
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0015
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0016
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0017
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0018
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0019
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0020
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0021
Generator: pilot analysis and fixes; G0002-G0021 tasks; handover notes merged
2026-10-03 05:04:23 +02:00
G0022
Hard batch G0022-G0026: 5/5 accepted; budget floor, CDS checks, mutation check
2026-10-03 05:16:30 +02:00
G0023
Hard batch G0022-G0026: 5/5 accepted; budget floor, CDS checks, mutation check
2026-10-03 05:16:30 +02:00
G0024
Hard batch G0022-G0026: 5/5 accepted; budget floor, CDS checks, mutation check
2026-10-03 05:16:30 +02:00
G0025
Hard batch G0022-G0026: 5/5 accepted; budget floor, CDS checks, mutation check
2026-10-03 05:16:30 +02:00
G0026
Hard batch G0022-G0026: 5/5 accepted; budget floor, CDS checks, mutation check
2026-10-03 05:16:30 +02:00